COM 597 Streaming Media Class 2 June 22, 2006

Today’s topics: The nuts and bolts of Audio The encoding and delivery process (sampling & resampling) Aliasing Audio Workshop Codecs and Players Compression demonstration Compression workshop

Audio How do you deal with legacy material? Audio sources: Records Open reel tape CD DVD Off-air recording Downloaded files Optical sound from a film Surround sound Embedded audio

Analog v. digital audio What is the difference? Analog material must be played in real time You will need to use a software tool or hardware device to convert these different physical representations of sound (grooves in a record for example) into a digital representation of the sound. This process can happen in a number of different ways during your production path. A-to-D converters (scanners)

Analog v. digital audio What is the difference? Sample rate will determine the fidelity of your final product Remember you cannot add detail to a sampled media file so it is best to start with a file that is as large as is reasonable. (RAW v. JPEG for example)

Raw Image.jpg Image Resolution Comparison Source v. Delivery

Raw Image.jpg Image Resolution Comparison Source v. Delivery

Sample rate is not bits-per-second, although they are related Sample rate is the number of “slices” of sound per second where bps is about how much data is begin delivered in the signal. Bps can be 22KHz or 96KHz or even 192KHz. For this class most of our work will be either in 44.1KHz or 48KHz.

When you sample a sound you need to record both the wave peaks and troughs Human ear is limited to detecting frequencies no higher than 22KHz. It was believed that simply doubling this number would give a sample rate that would reproduce all the sounds we can hear. Therefore, CDs run at 44.1KHz

Resampling a file And the problems inherent in the process

The difficulty with 44.1 as a standard is that it does not integrate easily into a standard production path. Most audio files are converted using integer- based averaging. This becomes apparent when you see that the common rates used for mastering are 48, 96 and 192 Khz. Moving down in quality involves simple averaging and not a complex resampling of the file.

Aliasing Artifacts

These quantization errors become evident when you convert between CD rates and film/television production rates. It is important to know your delivery before you start the process of sampling your sound. This is also why it is important to make certain all the tools in your production path are in agreement. (set to the same sample rate)

Audio formats you are likely to encounter Monoone track of audio Dual Mono (this is not stereo) identical audio on both tracks Stereomulti-channel audio on two tracks Mix Minusproduction audio with two mono tracks containing different sources (dialogue on one and effects on the other is one common example) 5.1 surroundHome theater is the most common application with three front speakers (left, right and center), two rear speakers and once low-frequency speaker (sub-woofer) AC3An alternative to Dolby Labs 5.1 surround format.

Mixing and blending of tracks needs to be carefully managed – phase and resampling errors Flash example Workshop

Codecs Which is best? There really is no one-stop solution. All of the major standards do a very good job of compressing most media The differences are subtle

There are real advantages to choosing one standard as your default distribution method Users will only need to rely on one playback solution Hardware will be optimized to that solution (servers and production tools) Licensing of technology is easier Staff is trained with emphasis on only one platform Support is easier

There are real disadvantages to choosing one standard as your default distribution method Not a large user base for the technology Format of files is not competitive with changes in the marketplace High licensing fees Flaky software Challenges with using the files on all types of hardware (WM on a Mac) You are tied to the fortunes of the company that created your codec

Format compatibilities and incompatibilities QuickTimeMac and PC are OK, no Linux support Windows MediaPC is great, Mac support is limited to older versions, no Linux Real NetworksSupports most every operating system FlashSupports most every operating system because it runs in a browser MPEG will play in many of the proprietary systems H.264 shows promise as a common standard.

How might you examine a codec to see if it will work for your needs? You might choose one compression solution for your production and another for delivery to your users.

PropertyProduction CodecDelivery Codec Compression Ratio Low usually between 1:1 and 4:1 High typically 50:1 or more QualityLossless (perfect) or close Good to acceptable, will denigrate to sub-VHS quality Lossy Zero for some codecs, for others very minimal Very lossy Bit rate25-50 and up to 150 Mbps As low as 300 Kbps for video and 24Kbps for audio File Size :20 seconds per GB (180 GB per hour) As low as.4 GB per hour EditableYes Not really, although many editors will accept a compressed file as a source. The quality will be dubious at best CompositableYesOnly if file has alpha channel Conversion to other formatsEasyAdditional degradation of quality is likely

General rule is that as compression schemas improve the files get smaller and the images either improve or stay the same.

Let’s look at how we compress a file to get an understanding of all the variables we can manage

Prior to compression you need to get your media in the best shape possible. Often it is easier in production to prep the media than it is to heal bad video during compression.

Video Preparation Deinterlacing the footage Frame rate agreement (2-3 cadence, 15fps, PAL v. NTSC, HD) Noise filters (dubious benefits at best) Grain removal

Video Preparation Color correction (gamma adjustments) Scaling (size your canvas can speed up compression significantly) Aspect ratio adjustment including anamorphic adjustments

Compression Example What the heck do those letters mean? And why do I care? I-frameIntra frame or key frame P-framepredicted difference frame B-framebi-directional difference frame Compression Demo