1. Digital Media
Dr. Jim Rowan
ITEC 2110
Video

2. Video Works because of persistence of vision
Fusion frequency
~ 40 frames per second
depends on the brightness of the image relative to the viewing environment
Less than that
flickering
individual images appear losing the illusion of motion

3. Video Video vs Animation... Video Animation
capture of frames in the world and then playback
Animation
create frames individually in the computer and then playback

4. Video... Computationally demanding
Capture
must be fast enough to capture sufficient frames to produce the illusion of motion
Transport across the web
must be fast enough to carry those captured frames at a rate fast enough to produce the illusion of motion
Playback
must be fast enough to play those captured frames at a rate fast enough to produce the illusion of motion

5. Video Transport and playback not fast enough?
something’s got to give
Video players (like quicktime) make compromises differently
Attempt to “degrade gracefully”
Some drop frames holding the last image
effectively losing the illusion of motion but continuing the story as a slide show
Some play lower resolution images
Some continue to play audio

6. Video Standards NTSC America and Japan
North America and Japan
24 frames per second
framesize different than PAL

7. Video Standards PAL Western Europe and Australia
25 frames per second
framesize different than NTSC

8. Video gets big fast At a 640 X 480 framesize
Using 3 byte color (24 bits, one byte per color) each frame ~ 2 megabytes
One second of video (uncompresssed) is 26 Megabytes
One minute is 1.85 gigabytes

9. The effects of large size...
Uncompressed?
exceeds most home computer interface standards
strains the internal speed of the home computer
strains the storage capability of home computer
WAY exceeds what can be carried by the net

10. What to do? Apply compression!
On the capture side
digitization & compression needs to be carried out by the hardware to be fast enough
Can be done in the camera (VTR)
Can be done in the computer (iSight cam)

11. Compression in the VTR Within the camera Mini DV or DV format
3 (at least) different formats internally
with differing error correction and compatibility
Recording on different media CD
Tape
Memory card
Mini DV or DV format
Connected to computer using firewire
All 3 formats present the same stream of bits to the computer
Artifacts that interfere with processing and recompression are created

12. Compression in the computer
Analog is presented to the computer through a video capture card
Compression is done (usually) in the video capture card
Allows for a really small camera because the work is done elsewhere

13. Analog vs Digital
An analog signal to the computer is susceptible to noise corruption
Digital signal is not
What’s the big deal?
Consider compressing a video of a wall painted a solid color
Analog noise will cause small fluctuations from pixel to pixel
RLE can’t compress it because each pixel is a bit different

14. Consider compressing
this using RLE
analog signal
!!!NOISE!!!
computer
iSight
Camera
video capture card
compression
digital
signal
iMovie
miniDV
the scene
compression
640 x 480 = 307,200 bytes
Noise makes each pixel
a little different
RLE:
307,200 bytes x RGB (3bytes)
= bytes
640 x 480 = 307,200
307,200 can be represented
by < 24 bits, call it 3 bytes
RLE:
307,200 (3bytes) +
RGB (3 bytes) = 6 bytes

15. hardware vs software compression
Hardware conversion... user has no control over it... it is hardwired
It is in the camera
It can be in the video card
Software conversion... is computationally expensive... it’s a slow process
Provides for the most flexibility
Can use different software coder-decoders (codec), picking and choosing what fits your needs better

16. Streaming Video Video is Similar to broadcast TV
Video is
transported across the web
played as it arrives
Similar to broadcast TV
Can be part of video conferencing
Network bandwidth is the enemy

17. Methods of Video delivery …NOT a mutually exclusive list …CAN be combined
Streaming video
similar to internet radio, when you “tune in” you listen to a show in progress
Embedded video
embedded inside something else like flash
plays from the beginning
usually does not allow download
youTube, google video

18. (more) Methods of Video delivery
Podcast
download now and play later
Progressive download
starts playing before download is completed
Pull Technology
user requested (requested from the client)
Push Technology
server initiated (like chat)

19. Streaming video never stored on disk can be open-ended
you don’t get a copy to play later
solves copyright issue!
can collect viewer stats without Nielsen ratings
advertisers know how many viewers there are
on some, you can advance ahead of where the download is complete and the play will start again at the new spot

20. Progressive download downloads some and then starts playing
predicts when the download will complete and starts playing when there is enough downloaded to safely play without interruption
you can get a copy to play later
quicktime does this
on some, you can advance ahead of where the download is complete and the download will start again at the new spot

21. The TV legacy Raster scan - 525 lines in US and Japan
Not enough bandwidth to transmit the whole frame at a rate of 40+ frames per second
Resulted in Interlacing fields
Play half the Frame (and half the data) then play the other half of the Frame

22. 525 lines total

23. 525 lines total

24. The problem...
The fields were played one after the other to avoid flicker
BUT...
The fields were also captured one after the other...
There is a time difference between when they were captured

25. Why is this a problem?
When you play them on a computer, the computer can refresh much faster and can display the entire frame at the same time
To play it can put both fields in a frame buffer and displays them at the same time
If the object is moving fast, the second frame shows the object in a different place
Results in a “comb effect”

26. Can we solve the problem?
You can average the two frames and construct a single frame
You can toss out one of the fields and interpolate between them
Neither is very good...

27. Converting Film to Video
Problematic
video is 30 frames per second
film is 24 frames per second
How do you make 30 frames from 24?
The 3-2 pull down… (and others)

28. Next
DV and MPEG

29. Questions?