1. Multimedia Building Block : Sound
References:
T. Vaughan, “Multimedia: Making It Work 5th Edition”, Osborne/McGraw-Hill, 2001
S. McGloughlin, “Multimedia: Concepts and Practice”, Prentice Hall, 2001
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

2. Why is sound important in Multimedia?
To reinforce message or theme
To set the mood
To catch the interest of the audience
To alert the audience
To include narration: effective for training and educational application.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

3. Types of Sound 2 types of sound in multimedia Music Speech Background
Attention grabber
etc..
Speech
Narration
Instruction
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

4. Sound consideration
While sound (music or narration) can be very useful, most of the time, they can be irritating as well.
Use appropriate music to reflect the mood or theme.
Give the users choice of turning off/on the sound.
In cases where the content of a page depends on sound (speech), consider providing transcript as the alternative. (for deaf people, non-multimedia pc)
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

5. Understanding Sound Sound travels as waves at 750 mph (at sea level)
Acoustic – a branch of physics that studies sound.
Sound pressure level (loudness or volume) is measured in decibels (dB)
Too much sound can have a damaging effects.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

6. Understanding sound
Amplitude – is the maximum displacement of the vibrating particles from their undisturbed positions
Frequency is measured in cycles-per-second, commonly known as Hearts (Hz)
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

7. Multimedia System Sound
Most target computers have sounds installed (e.g. default window sounds)
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

8. Multimedia System Sound
Macintosh and Windows systems have built-in sound recorders.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

9. Type of audio for multimedia application
Digital Audio
MIDI Audio (Musical Instruments Digital Interface)
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

10. Digital Audio - overview
Digital audio is a representation of the original sound.
Sound digitization is obtained through
Sampling
Quantization
Code-word generation process
Digitized sound is called sampled sound.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

11. Digital Audio - overview
Digital audio usually uses large storage space
Digital audio is less affected by hardware, thus providing consistent playback quality.
Audio track for multimedia project will sound as good in the end as it did in the beginning when it is created.
Require less musical talent to record.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

12. Digital Audio - Sampling
Every nth fraction of second, a sample of the original sound is captured and stored in digital format (binary digit in bits and bytes)
Sampling rate – describes how often the samples are taken.
Sampling size – describes the amount of information stored for each sample (bits and bytes)
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

13. Digital Audio - Sampling
Audio is digitally sampled thousands of times per second.
CD-quality audio is samples 44,100 times per second, thus sampling rate is 44.1 kHz
Other sampling rates commonly used in multimedia are kHz and kHz.
Sample size : 8 bits or 16 bits
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

14. Digital Audio - Quantization
The value of each sample is rounded off to the nearest integer (quantization).
If the amplitude is greater than the dynamic range available, clipping of the top and bottom of the wave occurs.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

15. Digital Audio – Quantization & Clipping
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

16. Preparing Digital Audio
1. Balance file sizes and quality
Audio resolution determine the accuracy with which a sound can be digitized i.e. how similar the regenerated sound to the original sound.
High quality sound audio requires high sampling rate and size, resulting in large file.
High quality sound requires large storage.
Choose between stereo or mono recording
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

17. Preparing Digital Audio
Stereo recordings are more lifelike and realistic
Mono recording are fine but sound flat.
To calculate file size (in byte):
Mono:
Sampling rate x duration of recording records x (sampling size / 8) x 1
Stereo:
Sampling rate x duration of recording records x (sampling size / 8) x 2
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

18. Preparing Digital Audio
2. Setting proper recording level (loudness)
Watch the meter in the recording device or software carefully to ensure the level of loudness are not too high or too low
Levels that are too high create distortion (sound alteration e.g. unpleasant crackling)
Levels that are too low cause the music to be too soft and overpower by the sound of the recording process itself.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

19. Preparing Digital Audio
3. Editing the recording
Trimming : removing blank space at the beginning or at the end of the recorded audio.
Splice: remove unnecessary noise (touch up)
Assembly: longer recorded audio can be build by cutting and pasting together a few shorter ones.
Convert to another audio format
Macintosh : SND and AIF files format
Windows : WAV files format
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

20. Preparing Digital Audio
Resampling or downsample: resample recorded audio file to a lower audio resolution. e.g. resample / downsample a 16 bit recorded sound to 8 bit.
Fade-in and fade-out: smooth out the very beginning and the very end of the sound file.
Time stretching: alter the length of the recorded sound
Digital Signal Processing (DSP): add special effects to the recorded sound.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

21. Sound Editing Tool
Sound Forge is one of the powerful sound editing tool for creating professional soundtracks.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

22. Sound Editing Tool
Apple’s QuickTime Player Pro allows you to change sound file formats and compression schemes and do simple cut-and-paste editing
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

23. MIDI Audio - overview
MIDI was developed in the 1980s as a standard communications protocol between electronic musical instruments and computer
MIDI file is not a digitized sound, it is the shorthand representation of music stored in numeric form (binary)
MIDI file contains instructional data (notes, sequences of notes, which instruments etc.) to other devices on how to generate an appropriate sound, it does not have the actual audio data, just the instruction to the instrument or sound card on how to reproduce it.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

24. MIDI Audio - overview
Offer opportunity for developers to compose their own music.
MIDI files are much more smaller than digital audio files, thus,
It requires less storage space.
Suitable to be used with web page.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

25. MIDI audio - overview Inconsistency in playback quality
Playback will only be accurate if the MIDI playback device is identical to the device used for production.
Not suitable for spoken dialogue
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

26. Preparing MIDI Audio You need:
Some musical talent or hire MIDI artists.
MIDI instruments (e.g. electronic keyboard)
MIDI Sequencer software:
allows you to record, edit and save music generated by MIDI instruments.
Computer Sound Hardware or Synthesizer
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

27. MIDI Sequencer Software
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

28. Preparing MIDI Audio Sound Synthesizer:*
07/16/96
Preparing MIDI Audio
Sound Synthesizer:
Is needed to translate information generated by the MIDI devices into sound (music).
May also be used to play back the MIDI file.
Typically built into the sound board on PC, but as an add-on board or peripheral for the Macintosh.
For more info, visit
That is the general concept behind a synthesizer: To put together parts and elements of a sound, from sound generating devices to sound processing devices, and form a whole sound.
To start with an electronically generated signal, process the signal through electronic methods, and emerge with a unique sound that often cannot be duplicated by any other instrument.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002 *

29. MIDI Audio System Components
MIDI Instrument
MIDI Instrument
                               
Synthesizer
Computer with MIDI Sound Board
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

30. Preparing MIDI Audio
MIDI can synthesize the sounds of over a hundreds different instruments.
How realistic the instruments sound is greatly affected by the hardware it is played on.
Each instrument is identified by a numeric General MIDI Instrument Sound code.
Table 9-3, page 218 – 224 in Multimedia: making it work.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

31. Choosing between MIDI and Digital Audio
In general, use MIDI data in the following circumstances
You only have small storage space, RAM, CPU processing power or bandwidth.
You have high quality MIDI sound source.
You have complete control over the playback hardware.
You do not need spoken dialogue.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

32. Choosing between MIDI and Digital Audio
In general, use digital audio in the following circumstances
You do not have control over the playback hardware
You have the computing resources and bandwidth to handle digital files.
You need spoken dialogue.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

33. Audio File Formats Macintosh : Windows :
Audio Interchange File Format (.AIFF) audio
Digitized Sound (.SND)
Windows :
Waveform Audio File (.WAV)
Both Macintosh and Windows can use MIDIs.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

34. Sound for the World Wide Web
Many audio format can now be found online.
Sound can be played over the Web in one of two ways.
Downloading: download and save the files to the computer first, then play it.
Streaming: sound is played while it is downloaded from the internet into the computer buffer portion by portion
Streaming quality is very sensitive to connection speed and internet traffic.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

35. Adding sound to Multimedia - tips
Before adding sound to your multimedia project, decide on:
What kind of sound is needed
When and where to use audio
Decide whether you want to use MIDI or digital
Acquire the source material
Edit the audio to suit your project
Test the sound.
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002

36. Copyright issues Secure rights for all material you use
Look for collections of royalty-free audio
CMPD273 Multimedia System
Prepared by Nazrita Ibrahim © UNITEN2002