1. Multimedia – Sound Dr. Lina A. Nimri Lebanese University Faculty of Economic Sciences and Business Administration 1 st branch

2. What is sound? Sound is vibrations in the air; that is, a series of rising and falling pressures in the air, ▫deviating from the average, which is represented by atmospheric pressure. To prove this, you could place something loud (like an alarm clock) inside a vacuum chamber, and notice that the initially noisy object no longer makes a sound if it isn't surrounded by air anymore. The simplest way to create a sound is to make an object vibrate. ▫In this manner, a violin makes a sound when the bow makes its strikes vibrate, ▫a piano sounds a note when a key is struck, because a hammer struck a string and made it vibrate.

3. What is sound? This is how sound waves are produced; they can be represented in a diagram as changes in air pressure (or in the electricity level of the magnet) as a function of time. This can be represented as:

4. Sonogram A sonogram, on the other hand, depicts sound frequencies as a function of time. It should be noted that a sonogram shows fundamental frequency, on top of which higher frequencies, called harmonics, are superimposed. This is what allows us to distinguish between different sources of sound: low notes have low frequencies, while high notes have higher frequencies.

5. Sonogram

6. Sound sampling To play sound on a computer, it must be converted into a digital format, as this is the only kind of information computers can work with. ▫A computer program intersperses small samples of the sound (which amount to differences in pressure) at specific intervals of time. This is called sampling or digitizing sound. The period of time between two samples is called the sampling rate.

7. Sound sampling To reproduce audio which sounds continuous to the ear, it is required to sample at least once every few 100,000 ths of a second (10 micro sec) ▫it is more practical to go by the number of samples per second, expressed in Hertz (Hz).

8. Nyquist – Shannon theorem Sampling frequency must be high enough to preserve the form of the signal ▫the sampling rate must be equal to or greater than twice the maximum frequency contained in the signal. ▫Our ears can hear sounds up to about 20,000 Hz. Therefore, for a satisfactory level of sound quality, the sampling rate must be at least on the order of 40,000 Hz. There are several standardized sampling rates in use: ▫32 kHz: for digital FM radio (band-limited to 15 kHz) ▫44.1 kHz: for professional audio and compact discs ▫48 kHz: for professional digital multi-track recording, and consumer recording equipment (like DAT or MiniDisc)

9. Sound for computers Each sample (corresponding to an interval of time) is associated with a value, ▫The sample determines the air pressure value at that moment.

10. Sound for computers A computer works with bits, so the number of possible values that the sample could have must be determined. ▫This is done by setting the number of bits on which the sample values are encoded. With 8-bit coding, there are 2 8 (= 256) possible values. With 16-bit coding, there are 2 16 (= 65536) possible values. The second option clearly offers higher sound fidelity, but at the cost of using more computer memory.

11. Sound for computers Stereo sound requires two channels, with sound recorded individually on each one. One channel is fed into the left speaker, while the other is broadcast from the right speaker. In computer processing, a sound is therefore represented by several parameters: ▫The sampling rate ▫The number of bits in a sample ▫The number of channels (one for mono, two for stereo, and four for Quadraphonic sound)

12. Memory size to store a sound file It is easy to calculate what size an uncompressed audio sequence will be. ▫By knowing how many bits are used to code the sample, you know its size (as the sample size is the number of bits) To find out the size of a channel, all you need to know is the sample rate, and thus the number of samples per second, and from that the amount of space taken up by one second of music. ▫This comes to: Sampling rate x Number of bits

13. Memory size to store a sound file To find out how much space a sound extract several seconds long will take up, just multiply the preceding value by the number of seconds: ▫Sampling rate x Number of bits x Number of seconds Finally, to determine the actual file size of the extract, the above figure should be multiplied by the number of channels (it will be twice as large for stereo as for mono). The size, in bits, of a sound extract is equal to: ▫Sampling rate x Number of bits x Number of seconds x Number of channels

14. Audio files FrequencyQuality 48,000 HzDAT, DVD-audio 44,100 HzCD 22,000 HzRadio 8,000 HzTelephone CD stereo audio of one hour: 44,100 x 16 x 2 x 3600 = 635 Mb DVD quadraphonic audio of one hour: 48,000 x 16 x 4 x 3600 = 1,382 Gb

15. Psychoacoustics Human ear Audition - The ability to hear comes from acoustic process, mechanical, hydraulic, nervous and mental from the outer ear to the brain Apparatus - The main hearing are: ▫the ear canal (outer ear) ▫ossicular chain (middle ear) ▫the cochlea (inner ear)

16. Human ear

17. Cochlea ▫Basilar membrane: mechanical frequency analyzer ▫Corti Organ: vibration sensor membrane

18. Perception of sounds Frequency Range ▫The human ear, through the basilar membrane, collects a continuous range: [20 Hz - 20, 000 Hz] Sensitive frequencies ▫By the effects of filtering and carried resonance in the ear canal and the ossicles, the most sensitive frequencies are : [1, 000 Hz - 5, 000 Hz]

19. Perception of sounds Fletcher-Munson Curves: ▫Contours measuring perceived strength (loudness) of sound depending on the frequency  The lowest frequency is the Listening threshold and highest frequency is the threshold of pain  The curves evolve according to age

20. Perception of sounds Bass sounds ▫The first “La” of the piano: 27 Hz ▫The perception is disproportionally reduced

21. Perception of sounds Medium sounds ▫The standard “La” of the piano : 440 Hz ▫The hearing is very high in this range

22. Perception of sounds Treble Sounds ▫The last “La” of the piano: 3480 Hz ▫The hearing is highly variable and decreases with age

23. Masking Masking of frequencies ▫Two sounds of frequencies very close together are merged in a single frequency  Same time: one sound  Closer moments: beating

24. Masking Temporal Masking ▫A high amplitude sound will be perceived longer and hides the sounds that follow  Echo phenomenon  Exponentially decreases (10 - 50 ms)

25. Sound Compression – general method Division ▫The audio signal is divided into 12 samples per second Conversion ▫Each sample is converted into a spectrum of 32 bands of frequencies Filtering ▫The frequency spectrum is analyzed with the psychoacoustics model  Threshold: any frequency below the threshold of listening is eliminated  Masking: any frequency obscured by another is eliminated Coding ▫Determine the number of bits needed to quantify the amplitudes while minimizing the quantization noise

26. Sound Compression – MP3 MP3 encoding is a lossy method Level I ▫Uniform Quantization ▫Only the frequency masking is used Level II ▫Uniform Quantization ▫Frequency and temporal masking Level III ▫Adaptive Quantization ▫Frequency and temporal masking ▫Huffman coding

27. Sound Compression – MP3 MP3 (“MPEG1 Audio layer 3”) is a lossy audio data compression format, developed by the International Standardization Organization (ISO). ▫This format is used to compress normal audio formats (WAV or audio CD) at a rate of 1:12. MP3 format permits the equivalent of files of twelve CD music albums to take up the same space as one CD-ROM. Moreover, mp3 barely alters the sound quality perceptible by human ear.

28. MP3 - Context MP3 compression involves removing the data corresponding to inaudible frequencies by the average person under normal listening conditions. ▫This compression analyses the spectrometric components of an audio signal, and applies a psychoacoustic model to them so as to preserve only "audible" sound. ▫The average human ear is able to recognize sounds between 0.02 kHz and 20 kHz, with sensitivity being at its peak for frequencies between 2 and 5 kHz MPEG compression involves determining which sounds go unheard and can be deleted; it is therefore “lossy compression”

29. MP3 – The masking effect Gabriel Bouvigne explains : "When you look at the sun and a bird passes in front of it, you don't see it, because the light from the sun is too bright. Acoustics are like that. When there are loud sounds, you don't hear the quiet sounds. For example: When an organist isn't playing, you can hear whistling in the pipes, and when he is playing, you can't hear it anymore, because it's masked. This is why it isn't necessary to record every sound, and this is the main principle used in the MP3 format to save space.“

30. Huffman code The Huffman algorithm is an encoding (not compression) algorithm, which takes effect at the end of the compression process, by creating variable-length codes over a large number of bits. ▫The codes have the advantage of a unique prefix, but they may be correctly decoded despite their variable length, and this can be done quickly with the use of tables. ▫This type of encoding saves, on average, a little under 20% of the space taken up. When sounds are “pure” (that is, there is no masking), the Huffman algorithm is very effective, as digital audio contains many redundant sounds.

31. MIDI - Definition MIDI is a protocol for communication and control between electronic musical instruments ▫one or more of these "instruments" can be a computer An instrument that supports a standard MIDI has a MIDI-IN port and a MIDI-OUT port ▫instruments can be linked in cascade (or daisy-chain) Sometimes the device has a MIDI-THRU, which is a direct copy of the MIDI-IN, without the delay caused by the copy on the MIDI- OUT port