Digital Media Dr. Jim Rowan ITEC 2110 Video

Video Works because of persistence of vision http://en.wikipedia.org/wiki/Frame_rate Fusion frequency ~ 40 frames per second depends on the brightness of the image relative to the viewing environment Less than that flickering individual images appear losing the illusion of motion

Video Video vs Animation... Video Animation capture of frames in the world and then playback Animation create frames individually in the computer and then playback

Video... Computationally demanding Capture must be fast enough to capture sufficient frames to produce the illusion of motion Transport across the web must be fast enough to carry those captured frames at a rate fast enough to produce the illusion of motion Playback must be fast enough to play those captured frames at a rate fast enough to produce the illusion of motion

Video Transport and playback not fast enough? something’s got to give Video players (like quicktime) make compromises differently Attempt to “degrade gracefully” Some drop frames holding the last image effectively losing the illusion of motion but continuing the story as a slide show Some play lower resolution images Some continue to play audio

Video Standards NTSC America and Japan http://en.wikipedia.org/wiki/Ntsc North America and Japan 24 frames per second framesize different than PAL

Video Standards PAL Western Europe and Australia http://en.wikipedia.org/wiki/PAL 25 frames per second framesize different than NTSC

Video gets big fast At a 640 X 480 framesize Using 3 byte color (24 bits, one byte per color) each frame ~ 2 megabytes One second of video (uncompresssed) is 26 Megabytes One minute is 1.85 gigabytes

The effects of large size... Uncompressed? exceeds most home computer interface standards strains the internal speed of the home computer strains the storage capability of home computer WAY exceeds what can be carried by the net

What to do? Apply compression! On the capture side digitization & compression needs to be carried out by the hardware to be fast enough Can be done in the camera (VTR) Can be done in the computer (iSight cam)

Compression in the VTR Within the camera Mini DV or DV format 3 (at least) different formats internally with differing error correction and compatibility Recording on different media CD Tape Memory card Mini DV or DV format Connected to computer using firewire All 3 formats present the same stream of bits to the computer Artifacts that interfere with processing and recompression are created

Compression in the computer Analog is presented to the computer through a video capture card Compression is done (usually) in the video capture card Allows for a really small camera because the work is done elsewhere

Analog vs Digital An analog signal to the computer is susceptible to noise corruption Digital signal is not What’s the big deal? Consider compressing a video of a wall painted a solid color Analog noise will cause small fluctuations from pixel to pixel RLE can’t compress it because each pixel is a bit different

Consider compressing this using RLE analog signal !!!NOISE!!! computer iSight Camera video capture card compression digital signal iMovie miniDV the scene compression 640 x 480 = 307,200 bytes Noise makes each pixel a little different RLE: 307,200 bytes x RGB (3bytes) = 921600 bytes 640 x 480 = 307,200 307,200 can be represented by < 24 bits, call it 3 bytes RLE: 307,200 (3bytes) + RGB (3 bytes) = 6 bytes

hardware vs software compression Hardware conversion... user has no control over it... it is hardwired It is in the camera It can be in the video card Software conversion... is computationally expensive... it’s a slow process Provides for the most flexibility Can use different software coder-decoders (codec), picking and choosing what fits your needs better

Streaming Video Video is Similar to broadcast TV http://en.wikipedia.org/wiki/Streaming_media Video is transported across the web played as it arrives Similar to broadcast TV Can be part of video conferencing Network bandwidth is the enemy

Methods of Video delivery …NOT a mutually exclusive list …CAN be combined Streaming video similar to internet radio, when you “tune in” you listen to a show in progress Embedded video embedded inside something else like flash plays from the beginning usually does not allow download youTube, google video

(more) Methods of Video delivery Podcast download now and play later Progressive download starts playing before download is completed Pull Technology user requested (requested from the client) Push Technology server initiated (like chat)

Streaming video never stored on disk can be open-ended you don’t get a copy to play later solves copyright issue! can collect viewer stats without Nielsen ratings advertisers know how many viewers there are on some, you can advance ahead of where the download is complete and the play will start again at the new spot

Progressive download downloads some and then starts playing predicts when the download will complete and starts playing when there is enough downloaded to safely play without interruption you can get a copy to play later quicktime does this on some, you can advance ahead of where the download is complete and the download will start again at the new spot

The TV legacy Raster scan - 525 lines in US and Japan Not enough bandwidth to transmit the whole frame at a rate of 40+ frames per second Resulted in Interlacing fields Play half the Frame (and half the data) then play the other half of the Frame

525 lines total

525 lines total

The problem... The fields were played one after the other to avoid flicker BUT... The fields were also captured one after the other... There is a time difference between when they were captured

Why is this a problem? When you play them on a computer, the computer can refresh much faster and can display the entire frame at the same time To play it can put both fields in a frame buffer and displays them at the same time If the object is moving fast, the second frame shows the object in a different place Results in a “comb effect”

Can we solve the problem? You can average the two frames and construct a single frame You can toss out one of the fields and interpolate between them Neither is very good...

Converting Film to Video Problematic video is 30 frames per second film is 24 frames per second How do you make 30 frames from 24? The 3-2 pull down… (and others) http://en.wikipedia.org/wiki/Telecine

Next DV and MPEG

Questions?