Previously, we were only skipping zero-packet frames when we needed to resample because the source sampling rate was not set to 48kHz. This check should have also been applied in the case that a packet did not need a resampler to be built. Fixes #224.