Multimedia Digital Audio Lecture # 6 Razia Nisar Noorani

Outline Audio Basic Digital Audio Concepts Streaming Audio Web Audio Formats

Sound Sound comprises the spoken word, voices, music and even noise. It is a complex relationship involving a vibrating object (sound source), a transmission medium (usually air), a receiver (ear) and a perceptor (brain). Example banging drum. As the sound vibrates it bumps into molecules of the surrounding medium causing pressure waves to travel away from the source in all directions

Sound So, Sound are rapid vibrations that are transmitted as variations in air pressure.

Waveforms Sound waves are manifest as waveforms A waveform that repeats itself at regular intervals is called a periodic waveform Waveforms that do not exhibit regularity are called noise The unit of regularity is called a cycle This is known as Hertz (or Hz) after Heinrich Hertz One cycle = 1 Hz Sometimes written as kHz or kiloHertz (1 kHz = 1000 Hz)

Waveforms Time for one cycle distance along wave Cycle

The characteristics of sound waves Sound is described in terms of two characteristics: Frequency Amplitude (or loudness) the rate at which sound is measured Number of cycles per second or Hertz (Hz) Determines the pitch of the sound as heard by our ears The higher frequency, the clearer and sharper the soundthe higher pitch of sound

The characteristics of sound waves Amplitude Sound’s intensity or loudness The louder the sound, the larger amplitude. In addition, all sounds have a duration and successive musical sounds is called rhythm

The characteristics of sound waves Time for one cycle Amplitude pitch distance along wave Cycle

Example waveforms Piano Pan flute Snare drum

Capture and playback of digital audio Air pressure variations DAC Converts back into voltage Digital to Analogue Converter Captured via microphone Air pressure variations ADC Signal is converted into binary (discrete form) 0101001101 0110101111 Analogue to Digital Converter

The Analogue to Digital Converter (ADC) An ADC is a device that converts analogue signals into digital signals An analogue signal is a continuous value It can have any single value on an infinite scale A digital signal is a discrete value It has a finite value (usually an integer) An ADC is synchronised to some clock

The Analogue to Digital Converter (ADC) It will monitor the continuous analogue signal at a set rate and convert what it sees into a discrete value at that specific moment in time The process to convert the analogue to digital sound is called Sampling. Use PCM (Pulse Code Modulation)

Digital sampling Sampling frequency SCA 3103 - Introduction to Multimedia 22 April 2017 Digital sampling Sampling frequency Lecture 7

Digital sampling Sampling frequency SCA 3103 - Introduction to Multimedia 22 April 2017 Digital sampling Sampling frequency Lecture 7

Sampling Two parameters: Sampling Rate Frequency of sampling Measure in Hertz The higher sampling rate, higher quality sound but size storage is big. Standard Sampling rate: - 44.1 KHz for CD Audio - 22.05 KHz - 11.025 KHz for spoken - 5.1025 KHz for audio effect

Sampling Size sample The resolution of a sample is the number of bits it uses to store a given amplitude value, e.g. 8 bits (256 different values) 16 bits (65536 different values) A higher resolution will give higher quality but will require more memory (or disk storage)

Quantisation Samples are usually represented the audio sample as a integers(discrete number) or digital

Basic Terms Term Definition Bit Binary value 0 or 1 Kilobit (Kb) 1,000 bits (approx.) Byte 8 bits Kilobyte (KB) 1,000 bytes Megabyte (MB) 1,000,000 bytes Gigabyte (GB) 1,000,000,000 bytes Kbps Kilobits per second (1,000 bits in a second) KB/sec Kilobytes per second (1,000 bytes in a second)

Analog vs. Digital Analog: A signal that has a continuous nature rather than a pulsed or discrete nature. Note: Electrical or physical analogies, such as continuously varying voltages, frequencies, or phases, may be used as analog signals. Digital: A signal in which discrete steps are used to represent information.

Digital Signal Use Pulse Code Modulation (PCM) to represent an audio signal by digital data.

ADC & DAC Figure 4.3 Conversion from Analog to Digital requires an Analog-to-Digital Converter Figure 4.4 Conversion from Digital to Analog requires a Digital-to-Analog Converter

Basic Digital Audio Concepts Sampling rate Number of sample taken of a signal in a given time (usually one second) Bit depth Describes the accuracy of the audio data Channels E.g. Stereo has two channels Bit rate Measured in kilobits per second (Kbps) is a function of the bit depth and sampling rate

Sampling rate The more sample taken per second, the higher the accuracy. Typically measured in kilohertz (KHz). CD audio has 44,100 samples per second (44.1KHz). 8 KHz produces lower quality radio sound. Standard sampling rates include “8KHz”, “11.025KHz”, … The high-end 96K is used in DVD, but is not applicable to the Web.

Sampling Rate demo

Bit depth Also called “sampling resolution” or “word length”. The more bits, the better is the quality of the audio (and a larger file of course). Common bit depths are 8-bit (telephone like), 16-bit (CD quality), and 20, 24, 32, 48-bit depths. How many signal can a 8-bit and a 16-bit data represent? 0000 0000  1111 1111 0000 0000 0000 0000  1111 1111 1111 1111

Quantization demo

Channels Audio file can support one to six channels of audio formats. Mono – one channel Stereo – two channels Some others – three, four channels. Six channels – 5.1-channel surround sound. More multi-channel formats announce in the coming years.

Channel Examples

Bit rate Audio files are measured in terms of bit rate which is measured in kilobits per second (Kbps). It can be calculated by dividing the file size by the time (in second) to play the audio clip. E.g. 3Mb file play in 30 seconds 3000k / 30 = 100kbps. Quality at different compression rates

Optimizing for the Web Length of the audio clip Number of channels Keep the audio clip as short as possible. E.g. just keep the most sweetest part of your greeting. Number of channels A mono audio file is halved the space of a stereo file. Depends on your actual needs.

Optimizing for the Web Bit depth Sampling rate Audio file on the Web are usually 8-bits. Half the size of a 16-bit file. Sampling rate Half the sampling rate will also halve the space needed. Voice only audio file can be reduced to 8KHz. 22 KHz music clips are acceptable. Putting all things together: Mono, 8-bit, 22KHz, MP3 compression.

Calculate audio size 8-bit mono: seconds x KHz 16-bit mono: (seconds x KHz) x 2 8-bit stereo: (seconds x KHz) x 2 16-bit stereo: ([seconds x KHz] x 2)x2 E.g. the file size of 30 seconds of 16-bit, 11KHz mono audio: (30 x 11) x 2 = 660K.

Streaming Audio What is it? Advantages: Disadvantages: Reference Play almost immediately after the request, continues playing the transferring data. Advantages: Address the problem of long download time. Control distribution and protect copyright, because the user cannot get a copy of the file. Disadvantages: Sound quality may be be affected by low speed or unstable Internet connection. Reference

What is Streaming?

How does it work? Streaming audio Packet are sent to a buffer on the receiving Computer, the RealPlayer will play the sound File when buffer full Web browser Web server RealAudio Server Web browser request a RealAudio from the web server

Quality Comparison http://www.cit.cornell.edu/atc/itsupport/streamcompare.shtml

Web Audio Formats WAV/AIFF (.wav/.aif/.aiff) The Waveform Audio File format (.wav) was developed by Microsoft, supports arbitrary sampling rates and bit depths. The Audio Interchange File format (.aif, .aiff) was developed for Macintosh platform. They are less likely used on the Web, because people use “mp3” or “streaming”.

WAV/AIFF Try the Bell-lab synthesis link. http://www.nuance.com/prodserv/demo_vocalizer.html WAV/AIFF Try the Bell-lab synthesis link. http://www.bell-labs.com/project/tts/voices.html Select the Audio format

MP3 (.mp3) Able to maintain excellent sound quality at very small file sizes. The compression reduces an audio file to one-tenth of its original size. E.g. 40MB file  3.5MB MP3 is actually MPEG-1 Layer-III Good for distribution of HQ audio. Demo: www.mp3.com

What is MP3 digital encoding? MP3 is actually the file extension for MPEG, audio layer 3. Layer 3 is one of three coding schemes (layer 1, layer 2, and layer 3) for the compression of audio signals. Layer 3 uses perceptual audio coding and psychoacoustic compression to remove all superfluous information. (More specifically, it removes the redundant and irrelevant parts of a sound signal--the stuff the human ear doesn't hear anyway). It also adds a MDCT (Modified Discrete Cosine Transform) that implements a filter bank, increasing the frequency resolution 18 times higher than that of layer 2.

MP3 Players

Apple QuickTime Audio (.mov) QuickTime is a well-known video format, but it can create audio-only movies. QuickTime is a container format, which is able to store still images, movie formats, … Excellent compression, true streaming Netscape and IE have Plug-in now. Quicktime : demo

MIDI (.mid/.midi) MIDI stands for “Musical Instrument Digital Interface” which is developed for electronic musical instruments. MIDI files are very compact and very good for low-bandwidth delivery. Instruments are “piano, drums, bass, orchestral strings, …” It is very attractive for adding MIDI file to your website with very little download time. Demo: www.findmidis.com

MIDI http://www.findmidis.com/

RealMedia/RealAudio (.rm/.ra) RealAudio is a server-based streaming audio format. The RealServer responses to the requests and delivers the streaming packets, including the bandwidth negotiation. A robust RealServer can support thousands of simultaneous listeners. Good for continuous-playing audio and live broadcasts to a large group of people. Example: RTHK Radio

RealMedia/RealAudio

Windows Media (.wma/.asf) Windows Media is a streaming system. It wraps all media elements into a Active Streaming File (.asf). Audio may be saved as non-streaming Windows Media Audio format (.wma). Good for continuous-play audio. The encoder and player is Free, shipped with Windows OS.

Windows Media

Choosing an audio format Audio Needs Suggested formats Short voice greetings WAV, AIFF, MP3 News broadcasts Streaming solutions (RealAudio, Windows Media) Background music MIDI, WAV Music samples for some audience MP3 or QuickTime Radio-style or Live broadcasting RealMedia System Musical E-greeting card

Add Audio to your Web Page A simple link <A HREF=“audio/song.wav”>Play the song </A> <A HREF=“music.mp3”><IMG SRC=“buttons/play.gif”></A> Background Sound <BGSOUND SRC=“audio/song.mid” LOOP=3> Link to RealMedia <A HREF=“song.ram”>Link to the song</A> pnm://domainname.com/song.rm

Digital audio editing software One of the most powerful and professional PC-based packages is a tool called Sound Forge http://www.sonicfoundry.com/

Editing Digital Audio Trimming Splicing and assembly Volume adjustments Format conversion Resampling or downsampling Fade ins and fade outs Equalization Time Stretching Digital Signal processing Reversing Sounds

Editing Digital Audio Simple audio editing software allows: Recording of digital audio segments Trimming Splicing and assembly Volume adjustments of the entire segment Reversing Sounds Copy, cut, paste and delete segments of digital audio Others audio editing software: COOL Edit Pro Gold Wave PROSONIQ SonicWORX Samplitude Studio

How audio can be used effectively Examples of uses of audio: Cautions and warnings It is a good medium for alerting users to critical information. Some uses include: Sounding an alarm when a limit is reached Alerting users when data is entered incorrectly Music and Sound Effects These make multimedia interaction more real. Some uses include:

How audio can be used effectively Musical background for a video segment Birds Songs accompanying photographs in biological field training. Sound-related data. Some uses include: Helping mechanics diagnose engine trouble Training medical students to recognize different breathing sounds

How audio can be used effectively Direct voice communication. Some uses include: Leaving a voice message for other users of an application Consulting with an expert during a troubleshooting procedure.

Advantages and Disadvantages of using audio Sound adds life to any multimedia application and plays important role in effective marketing presentations. Advantages Ensure important information is noticed Add interest Can communicate more directly than other media

Advantages and Disadvantages of using audio Easily overused Requires special equipment for quality production Not as memorable as visual media

Summary There are two main types of digital audio Sampled audio Captured by sampling an analogue waveform at a set rate MIDI data Instructions on how to perform some musical composition Sampled audio requires more storage space than MIDI information