1. Unit 7 – Multimedia Element: Sound
Semester 2

2. Unit Outline In this unit, we will learn Use of Sound
Guidelines for Using Sound
Basic Principles of Sound
Sound Quality
Sample Rate
Audio Resolution
Digital Audio File Size
Audio Compression
Downloaded vs. Streamed
Sound Card
Digital Audio Software
Media Players
Audio File Formats

3. Introduction
The element of sound can be important to achieve the goals of a multimedia product.
Sound adds another dimension.
If used well, sound is an extremely powerful element which can stimulate emotional responses that would never be activated from text and graphics alone.

4. Introduction
Sound helps convey the intended message or complement the purpose of multimedia applications.
To help ensure effectiveness, it is important to incorporate sound files that are high quality, appropriate, and consistent with the goals of the multimedia product.

5. Use of Sound
Providing controls (such as skip, pause, mute, volume adjustment) is an effective way of keeping users involved and motivated.
The decision to incorporate sound into a multimedia product should have solid reasoning behind it.

6. Use of Sound Sounds are either content sounds or ambient sounds.
Content sounds furnish information
Narration, dialogue are content sounds.
Music and other sounds can be considered as content sounds if they are parts of the topic themselves.
Ambient sounds reinforce messages and set the mood
Background sounds and special effects are ambient sounds.
Special sound effects can reinforce or enliven a message.

7. Guidelines for Using Sound
Use the same style of music (if multiple sound files are needed) to maintain a sense of unity
Coordinate sound files with other media elements
Sound quality should be kept consistent
Record at a rate and resolution that is appropriate to the delivery mode 7

8. Guidelines for Using Sound
Use the same voice for narration and voiceovers, but different voices for different characters
Optimize files for background music
Use sound cues for specific events
During voice-overs, background music should be turned off or adjusted to a low volume such that the spoken words can be understood without difficulty 8

9. Basic Principles of Sound
When we speak, vibrations, called sound waves, are created.
Sound waves have a recurring pattern or an analog wave pattern called a waveform.
This analog wave pattern represents the volume and frequency of a sound.

10. Basic Principles of Sound
Amplitude: Distance between the valley and the peak of a waveform; determines volume
Volume is measured in decibels (dB)
Decibel (dB) is a logarithmic unit used to describe a ratio.
One dB is close to Just Noticeable Difference (JND) for sound level.
Frequency: Number of peaks that occur in one second measured by the distance between the peaks; determines pitch

11. Basic Principles of Sound
Analog sound is a continuous stream of sound waves. For sound to be included in multimedia applications, analog sound must be converted to digital form.
Digitizing (or sound sampling): the process of converting analog sound to numbers
Digital Audio: An analog sound that has been converted to numbers

12. Basic Principles of Sound
Sound sampling converts analog sound to digital audio.

13. Basic Principles of Sound
During digitizing, sound samples are taken at regular time instants.
Time instants are discrete.
Sound samples (the volumes of sound at time instants) cannot be stored precisely. Instead, only quantified values can be stored.
The feasible quantified values are known as quantization levels. 13

14. Basic Principles of Sound
The number of quantization levels is related to the quality of digital audio.
If more quantization levels are allowed, the difference between the original value and the quantified value will be smaller and we will get a better quality of the digital representation. However, this would also mean a higher cost for storage and processing of these values inside a computer (disks of larger capacity and more powerful CPUs are required) 14

15. Sound Quality
Factors that determine the sound quality of digital audio
sample rate
audio resolution
Sample rate
Number of sound samples taken per second
Also known as sampling rate
Measured in kilohertz (kHz),
Common values: 11 kHz, 22 kHz, 44 kHz
CD quality: 44 kHz

16. Sound Quality Audio resolution
Also known as sample size or bit resolution
Number of binary bits used to represent each sound sample
As the audio resolution increases, the quality of the digital audio also improves.
Audio resolution determines the accuracy with which sound can be digitized.
Common values: 8 bits, 16 bits
CD quality: 16 bits

17. Number of Quantization Levels
Sound Quality
Audio Resolution
Number of Quantization Levels
16-bit
216
65536 possible values for the sound sample
8-bit 28
256 possible values for the sound sample
4-bit 24
16 possible values for the sound sample
2-bit 22
4 possible values for the sound sample 17

18. Monophonic vs. Stereo Sound
Monophonic sounds: flat and unrealistic when compared to stereo sounds
Stereo sounds: much more dynamic and lifelike
Monophonic sound files are sometimes a more appropriate choice where storage and transfer time are major concerns.
Narration and voiceovers can effectively be saved in a monophonic format.
Music almost always must be recorded and saved in stereo.

19. Digital Audio File Size
File size of a digital audio recording (in bytes) (assume that there is no compression)
* Where number of channels, either 1 for monophonic or 2 for stereo
File size of a monophonic digital audio recording (in bytes) (assume that there is no compression)
File size of a stereo digital audio recording (in bytes) (assume that there is no compression)

20. Audio Compression
Similar to image compression (refer to Unit 5), mathematical algorithms are used to reduce file sizes.
File size is the primary consideration when using audio files on the Web.
Compression is beneficial for storing and transferring audio files.

21. Audio Compression
Different compression schemes are available for different file formats, and some file formats include the compression scheme within the format itself.
The compression scheme applied impacts both the quality of audio file and the file size.

22. Digital Audio File Size (Revisit)
(Recall from Unit 5)
Example: When a 6 MB audio file is compressed to 3 MB, the compression ratio is (6 / 3) = 2.
File size of a digital audio recording (in bytes) (assume that the compression ratio is CR) 22

23. Downloaded vs. Streamed
Web audio: downloaded or streamed
Downloaded audio file must be entirely saved to the user’s computer before it can be played.
Streaming: a more advanced process that allows audio file to be played as it is downloading (i.e. before the entire file is transferred to the user’s computer)
If we want our audio files to be streamed over the Internet, the web-hosting service must support streaming.

24. Downloaded vs. Streamed
If the user’s computer receives streaming audio data more quickly than required, the excess data will be stored in a buffer.
If the user’s computer receives streaming audio data slower than required, the data stored in the buffer will be used. If the buffer becomes empty, the user will experience a break. 24

25. Downloaded vs. Streamed
Two types of streamed audio:
On demand
Streamed audio is stored on a server for a long period of time, and is available for transmission at a user’s request
Live
Live streams are only available at one particular time
Example: live radio programme 25

26. Delivery
To play sound on a computer system, a sound card and speakers, or a headset are needed.
The digital audio file is sent through a digital-to-analog converter (DAC) so that it can be heard.
It is important to test sounds under a variety of different conditions. 26

27. Sound Card
Is an expansion board in a computer which interfaces with the processor through slots on the motherboard
Can be integrated on the motherboard
Externally it is connected to speakers for playback of sound
Main functions:
Playback
Digitizing
Recording
Compressing 27

28. Sound Card
Example: Creative Labs SoundBlaster 28

29. Sound Card Basic components of sound card: Self-Study Slide
Memory banks
for storing audio data during digitization and playback of sound files
DSP (Digital Signal Processor)
gives the sound card ability to perform functions like digital filtering, anti-aliasing, noise shaping and compression
DAC / ADC (Digital-to-Analog Converter, Analog-to-Digital Converter)
for digitizing analog sound and reconverting digital sound files to analog form for playback 29

30. Sound Card Self-Study Slide Input Ports: MIC, Line In
for feeding audio data to the sound card through a microphone connected to it; for feeding audio data from external CD players for recording or playback
Output Ports: Speakers, Line Out
for attaching speakers for playback of sound files; for connecting to external recording devices or an external amplifier
MIDI Input Port
for interfacing with an external synthesizer. Using this connection, MIDI songs can be composed on the PC using software and then can be sent to the sound modules of external synthesizers for playback. 30

31. Digital Audio Software
Traditional sound studio equipment is utilized to record, mix and edit audio. The cost can be tens of thousands of dollars.
With sound editing and mixing software, audio files can be produced for much less money.
Sound editing software is generally a much cheaper way to mix audio for a multimedia application than using traditional sound studio equipment.

32. Digital Audio Software
Examples:
Peak
for Macintosh platform
Sound Forge
for Windows platform
Adobe Audition
formerly called Cool Edit

33. Digital Audio Software
Self-Study Slide
Basic sound editing operations
Multiple tracks editing
Edit and combine multiple tracks and then merge the tracks and export them in a “final mix” to a single audio file
Trimming
Remove “dead air” or blank space from the front of a recording and any unnecessary extra time off the end
Splicing and assembly
Remove the extraneous noises that inevitably creep into a recording
Volume adjustments
Provide a consistent volume level 33

34. Digital Audio Software
Self-Study Slide
Format conversion
Import and save files in various formats
Resampling or downsampling
Increase or reduce sample rates
Fade-ins and fade-outs
Smooth out the very beginning and the very end of a sound file
Reversing sounds
Reverse all or a portion of a digital audio recording 34

35. Media Players Popular media players: Media players should:
Windows Media Player
QuickTime Player
RealPlayer
iTunes
Media players should:
be equipped to open and display most popular audio file formats
possess the ability to play live streaming audio 35

36. Audio File Formats MP3 (MPEG - Audio Layer 3) WAV (Waveform format)
Open standard technology
Uses file compression to create near-CD quality audio files
WAV (Waveform format)
for Windows platform
Real formats (RA, RAM, RM)
for streaming audio
WMA (Windows Media Audio)
Developed by Microsoft

37. Audio File Formats ASF (Advanced Systems Format)
Formerly known as Advanced Streaming Format
Mircosoft’s proprietary format for streaming
Stores audio and video information
Specially designed to run on networks
Content is delivered to users as continuous flow of data; little waiting time will be experienced before playback begins
AIFF or AIF (Audio Interchange File Format)
Macintosh waveform format

38. Audio File Formats MIDI (Musical Instrument Digital Interface):
Internationally accepted file format for storing MIDI data
Provides instructions on how to replay music; does not record the waveform (MIDI data is not digital audio)
Files of MIDI format are often smaller than those of most other audio file formats
If spoken dialog is needed, digital audio should be used instead 38

39. Reference
Part of this slide set is prepared or/and extracted from the following book:
Multimedia For The Web Revealed, Calleen Coorough & Jim Shuman, Thomson Learning, 2006, ISBN:
This set of slides is for teaching purpose only.
Self-study slide(s) is / are within the scope of the final examination.