1. Dr. Jim Rowan ITEC 2110 Video Part 2
Digital Media
Dr. Jim Rowan
ITEC 2110
Video Part 2

2. Coping with Video Size Consider human vision limitations
1) Chrominance sub-sampling
Compression - two versions
2) Spatial
3) Temporal
differencing
vectoring

3. Coping with Video Size Consider human vision limitations
1) Chrominance sub-sampling
Compression - two versions
3) spatial
4) Temporal
differencing
vectoring

4. Chrominance sub-sampling
Humans can’t distinguish changes in color as well as they can distinguish luminance changes
In our cameras…
Of every 4 frames
store the luminance for each frame
only store a proportion of the color info
4:2:0

5. Chrominance sub-sampling Luminance, Cr, Cb

6. Coping with Video Size Aside from screen size and frame rate...
Consider human vision limitations
1) Chrominance sub-sampling
Compression - two versions
2) spatial
3) Temporal
differencing
vectoring

7. Coping with Video Size Spatial compression
Individual images can be compressed using the techniques discussed in the bitmapped section
Doesn’t result in very much compression for video
Doesn’t take into consideration the other frames that come before or after it
Our video cameras do this, compressing each frame to jpeg

8. Coping with Video Size Aside from screen size and frame rate...
Consider human vision limitations
1) Chrominance sub-sampling
Compression - two versions
2) spatial
3) Temporal
differencing
vectoring

9. Temporal Compression differencing
Use the Difference in two frames
A naive approach can result in good compression
Works well for a small amount of movement
Security cameras spend most of their time “seeing” the same thing all night long
A Tarantino film? not so much…
Most pixels change with nearly every frame
Saved
Frame 1
Saved
Frame 2
Captured
Frame 1
Captured
Frame 2

10. Image Differencing To subtract one image from the next
Do it one pixel at a time
red minus red
green minus green
blue minus blue
Store the difference
To play it back
Play frame one 1
Add frame 2 to frame 1
Next, an example in black & white

11. Example 1, the difference of two identical images

12. Example 1, the difference of two identical images
The result ===>

13. Example 2, the difference of two similar images

14. Example 2, the difference of two similar images
The result ===>

15. Temporal Compression vectoring
When an OBJECT moves
compute its trajectory
fill in the resulting exposed background
Captured
Frame 1
Captured
Frame 2
Stored
Background
Stored
Object
Movement
Vector
BUT there’s a problem...
why isn’t this an easy thing to do?

16. More on differencing
The differencing can happen in a forward manner and a backward manner
It might be more economical (in data size) to create a frame from a frame that follows it...

17. MPEG-2 iFrame pFrame bFrame GOP

18. iFrame: pFrame bFrame -a keyframe -spatially compressed
(a fully specified image)
pFrame
-predicted frame
-contains only the difference
between the current frame and
the previous iFrame
(smaller in size than iFrame)
bFrame
-bi-predicted frame
-contains difference between
current and both the preceding
and following iFrames
-even smaller in size than iFrame

19. 3 mpeg 2 video streams Group(s) Of Pictures (GOPs) The largest
Group(s) Of Pictures (GOPs)
The largest
All spatially (intra-frame)
compressed
Spatially compressed
and predictive
(difference)
Smaller…
but more
computation
Spatially compressed forward and backward predictive
(difference)
Smallest…
but more
computation
and it is
transported
out of order!
Play sequence:
Transmit sequence:

20. Mpeg compression technique 1
Frame capture sequence:

21. How does this ===> happen?
So…
How does this ===> happen?

23. Mpeg encoding It’s COMPLICATED…
It’s COMPLICATED…
Images are broken up and transmitted as macroblocks
They can be a variety of sizes
Typically 8X8 or 16X16
When missing (or there are video errors)
May use a lower resolution block (that was previously received)
May just use a solid color block instead

24. Video Compression What does this?
Coder/Decoder - Codec
encodes and decodes video
Can be symmetric
it takes as long to compress as decompress
Can be asymmetric
it takes longer to compress or decompress than it does to decompress to compress

25. Video Compression Can be very complex
Each image is just jpeg compressed
Each image is jpeg compressed; the p-frames
are the result of frames being subtracted from
the i-frames
Each image is jpeg compressed; the p-frames
are the result of frames being subtracted from
the i-frames; b-frames are the result of
frames being subtracted from i-frames and
future p-frames

26. A final worry... Codecs compress video to make it smaller
There are a variety of codecs to do this
Which to choose?
It is a tradeoff between compression technique, its computational complexity and its artifacts

27. So... How do codecs vary? compression and decompression complexity
affects the artifacts that are created
affects the time required to carry them out
affects the volume of the data stream created
affects the type and expense of the equipment used
affects whether or not it can be implemented in hardware of software
WMV, DivX, Cinepak, Intel Indeo & Sorenson

28. Which is the original? Notice the artifacts?

29. Center one is the original Left is “sharpened” Right is “blurred”

30. That’s It!