1. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 1 Carnegie Mellon

2. Sound Compression

3. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 3 Carnegie Mellon Compression of Sound

4. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 4 Carnegie Mellon Motivation for Sound Compression  need to minimize transmission costs or provide cost efficient storage  demand to transmit over channels of limited capacity such as mobile radio channels  need to share capacity for different services (voice, audio, data, graphics, images) in integrated service network

5. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 5 Carnegie Mellon Compression Approaches Delta codingDelta coding Encode differences only Encode differences only Predictive codingPredictive coding Predict the next sample Predict the next sample Linear Predictive Coding (LPC) - mostly for speechLinear Predictive Coding (LPC) - mostly for speech Describe fundamental frequencies + ‘error’ Describe fundamental frequencies + ‘error’ CELP, RPE, cell-phone standards CELP, RPE, cell-phone standards Variable Rate EncodingVariable Rate Encoding Don’t encode silences Don’t encode silences regular signal=few bits, variable signal=many bits regular signal=few bits, variable signal=many bits Subband codingSubband coding Split into frequency bands each encoded separately + efficiently Split into frequency bands each encoded separately + efficiently Psycho-acoustical codingPsycho-acoustical coding drop bits where you can’t hear it drop bits where you can’t hear it

6. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 6 Carnegie Mellon Many (!) Compression Standards PCM (Pulse Code Modulation) u-LAW (Mu-law – logarithmic coding) LPC-10E (Linear Predictive Coding 2.4kb/s) CELP 4.8Kb/s – code excited LPC builds on LPC GSM (European Cell Phones, RPE-LPC) 1650 bytes/sec (at 8000 samples/sec) ADPCM (adaptive, delta PCM, 24/32/40 kbps) MPEG Audio Layers (builds on ADPCM) Layer-2: From 32 kbps to 384 kbps - target bit rate of 128 kbps Layer-3: From 32 kbps to 320 kbps - target bit rate of 64 kbps Complex compression, using perceptual models RealAudio, Windows Media Formats (builds on above, proprietary)

7. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 7 Carnegie Mellon MPEG Audio Encoding MPEG encodingMPEG encodingMPEG encodingMPEG encoding

8. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 8 Carnegie Mellon MPEG Audio Encoding Motion Picture Experts Group Audio EncodingMotion Picture Experts Group Audio Encoding Mpeg-1 layer 1 - 384 kbit/s Mpeg-1 layer 1 - 384 kbit/s Mpeg-1 layer 2 - 192 kbit/s Mpeg-1 layer 2 - 192 kbit/s Mpeg-1 layer 3 - 1128 kbit/s Mpeg-1 layer 3 - 1128 kbit/s Mpeg-2 Mpeg-2

9. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 9 Carnegie Mellon MPEG 1 layer 1 Audio Mapping the audio input into 32 subbands via equally spaced bandpass filtersMapping the audio input into 32 subbands via equally spaced bandpass filters Fixed block lengths (8 ms at 48 kHz sampling frequency) compressed into 12 samples eachFixed block lengths (8 ms at 48 kHz sampling frequency) compressed into 12 samples each Quantization providing very large dynamic range of more than 120 dB.Quantization providing very large dynamic range of more than 120 dB. Bit allocation algorithm selects a quantizer (out of 15) for bit rate requirement and best coding noise masking.Bit allocation algorithm selects a quantizer (out of 15) for bit rate requirement and best coding noise masking. In the official ISO/MPEG subjective tests Layer I codec showed an excellent performance at a stereo bit rate of 384 kbit/s.In the official ISO/MPEG subjective tests Layer I codec showed an excellent performance at a stereo bit rate of 384 kbit/s.

10. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 10 Carnegie Mellon MPEG 1 layer 2 Audio Similar to the Layer I coder but better performance through three modifications.Similar to the Layer I coder but better performance through three modifications. The input to the psychoacoustic model is a 1024-point FFT leading to a finer frequency resolution for the calculation of the global masking thresholds. The input to the psychoacoustic model is a 1024-point FFT leading to a finer frequency resolution for the calculation of the global masking thresholds. Exploiting redundancies between three adjacent 12- sample blocks. Exploiting redundancies between three adjacent 12- sample blocks. A finer quantization is provided. A finer quantization is provided. In the official ISO/MPEG subjective tests Layer II codec showed an excellent performance at a stereo bit rate of 256 kbit/s, and when using joint stereo, at 192 kbit/s.In the official ISO/MPEG subjective tests Layer II codec showed an excellent performance at a stereo bit rate of 256 kbit/s, and when using joint stereo, at 192 kbit/s.

11. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 11 Carnegie Mellon MPEG 1 layer 3 Audio (MP3) Hybrid (subband/transform) codingHybrid (subband/transform) coding Variable bit rate codingVariable bit rate coding Layer III has improved:Layer III has improved: frequency mapping frequency mapping an analysis-by-synthesis approach for the bit allocation an analysis-by-synthesis approach for the bit allocation an advanced pre-echo control an advanced pre-echo control Layer III employs nonuniform quantization with entropy coding.Layer III employs nonuniform quantization with entropy coding. In order to achieve a higher frequency resolution closer to critical band partitions:In order to achieve a higher frequency resolution closer to critical band partitions: the 32 subband signals are subdivided in frequency content by applying a six-point or 18-point modified DCT (MDCT) with 50% overlap to each of the subbands. the 32 subband signals are subdivided in frequency content by applying a six-point or 18-point modified DCT (MDCT) with 50% overlap to each of the subbands. the six-point MDCT is applied in case of expected pre-echoes. the six-point MDCT is applied in case of expected pre-echoes. the maximum number of frequency components is 32*18=576 each representing a bandwidth of only 24000/576 = 41.67 Hz. the maximum number of frequency components is 32*18=576 each representing a bandwidth of only 24000/576 = 41.67 Hz. At a stereo bit rate of 1128 kbit/s the ISO/MPEG subjective tests gave an increase in impairment score of around 0.6 over Layer II.At a stereo bit rate of 1128 kbit/s the ISO/MPEG subjective tests gave an increase in impairment score of around 0.6 over Layer II.

12. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 12 Carnegie Mellon MPEG 2 Audio 2 multichannel audio coding standards, one of which will be forward- and backwards-compatible with MPEG-1.2 multichannel audio coding standards, one of which will be forward- and backwards-compatible with MPEG-1. Forwards-compatible with MPEG-1: The multichannel decoder can properly decode MPEG-1 mono or stereo signals.Forwards-compatible with MPEG-1: The multichannel decoder can properly decode MPEG-1 mono or stereo signals. Backwards-compatible with MPEG-1: An MPEG-1 stereo decoder can deliver a correct (front left and front right) stereo signal when decoding a future MPEG-2 multichannel bit stream i.e., the decoder will reproduce a meaningful down-mix of the original five channel.Backwards-compatible with MPEG-1: An MPEG-1 stereo decoder can deliver a correct (front left and front right) stereo signal when decoding a future MPEG-2 multichannel bit stream i.e., the decoder will reproduce a meaningful down-mix of the original five channel. Nonbackwards-compatible (NBC) MPEG-2 codingNonbackwards-compatible (NBC) MPEG-2 coding The NBC multichannel coding mode will have the sampling rates, audio bandwidth, and channel configurations of MPEG-2 audio, but shall be capable to operate at bit rates from 32 kbit/s up to a bit rate sufficient for high quality audio.The NBC multichannel coding mode will have the sampling rates, audio bandwidth, and channel configurations of MPEG-2 audio, but shall be capable to operate at bit rates from 32 kbit/s up to a bit rate sufficient for high quality audio.

13. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 13 Carnegie Mellon Audio Encoding Issues Bitrate vs fidelity tradeoffBitrate vs fidelity tradeoff Computational effort (time) in encoding/decodingComputational effort (time) in encoding/decoding Delay and window sizeDelay and window size Recovery from lossy transmission, packet errorsRecovery from lossy transmission, packet errors Noisy environments (for speech, music encoders)Noisy environments (for speech, music encoders)

14. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 14 Carnegie Mellon Sound Editing GoldWave (www.goldwave.com)-GoldWave (www.goldwave.com)-www.goldwave.com requires a sound card. requires a sound card. digital audio sound player, recorder and editor digital audio sound player, recorder and editor can load, play and edit many different file formats can load, play and edit many different file formats.wav,.au,.voc,.snd, ….wav,.au,.voc,.snd, … displays separate graphics for the left and right channels displays separate graphics for the left and right channels very easy to use very easy to use good sound quality good sound quality CoolEdit (http://www.syntrillium.com/cooledit ) CoolEdit (http://www.syntrillium.com/cooledit )http://www.syntrillium.com/cooledit SonicFoundry SoundForge, SonicFoundry SoundForge, SonicFoundry SOX conversion tool SOX conversion tool Winamp Winamp MS SoundRecorder, Windows Media encoder MS SoundRecorder, Windows Media encoder

15. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 15 Carnegie Mellon Tips for Audio on the Web There is no generic audio standard on the Web Listening to 16-bit sounds on an 8-bit system results in strange effects Users will be annoyed if they spend a lot of time downloading a sound and they can’t play it Distribute only 8-bit sounds on your Web page Or, provide different sound files in both 8- and 16-bits Record in the highest sampling rate and size you can, and then process down to 8-bit Keep file size small downsampling to 8-bit use a lower sampling rate use mono sounds Describe what format those sounds are in WAVE, AIFF, or other format Providing the file size in the description is a politeness to help estimate download times If you need high sound quality and have large audio files: Use a smaller sound clip in downsampled format as a preview or for those who can’t to listen to the higher-quality sample. Check out http://music.acu.edu/www/iawm/articles/june96/techtips.htmlhttp://music.acu.edu/www/iawm/articles/june96/techtips.html

16. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 16 Carnegie Mellon Audio Encoding Reference Links Sub-Band CodingSub-Band CodingSub-Band CodingSub-Band Coding MPEG AudioMPEG AudioMPEG AudioMPEG Audio http://www.realnetworks.comhttp://www.realnetworks.comhttp://www.realnetworks.com esp. resources and tutorials esp. resources and tutorials

17. Speech Compression That’s all for today

18. © Copyright 2002 Michael G. Christel and Alexander G. Hauptmann 18 Carnegie Mellon