1. Digital Image and Video Coding 11. Basics of Video Coding H. Danyali hdanyali@ieee.org

2. This lecture is based on the material provided in: 2

3. Outline 3

4. 4

5. 5

6. 6

7. 7

8. 8

9. 9

10. 10

11. 11

12. 12

13. 13

14. 14

15. Global motion compensation In global motion compensation, the motion model basically reflects camera motions such as:global motion compensation –Dolly - moving the camera forward or backwards –Track - moving the camera left or right –Boom - moving the camera up or down –Pan - rotating the camera around its Y axis, moving the view left or right –Tilt - rotating the camera around its X axis, moving the view up or down –Roll - rotating the camera around the view axis It works best for still scenes without moving objects. There are several advantages of global motion compensation: –It models the dominant motion usually found in video sequences with just a few parameters. The share in bit- rate of these parameters is negligible. –It does not partition the frames. This avoids artifacts at partition borders. –A straight line (in the time direction) of pixels with equal spatial positions in the frame corresponds to a continuously moving point in the real scene. Other MC schemes introduce discontinuities in the time direction. Moving objects within a frame are not sufficiently represented by global motion compensation. Thus, local motion estimation is also needed. 15

16. 16

17. 17

18. 18

19. 19

20. 20

21. 21

22. Nearest neighbor interpolation Original zoomed A grid: Size: the same as the original image Pixels: the same as the zoomed image overlap Pixel enlargement 22

23. Linear interpolation 39 23

24. Bilinear interpolation –bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables on a regular grid. The key idea is to perform linear interpolation first in one direction, and then again in the other direction. –generates an image of smoother appearance that nearest neighbour –requires 3 to 4 times the computation time of the nearest neighbour method See interp2 function in Matlab 24

25. 25

26. 26

27. 27

28. 28

29. 29

30. 30

31. 31

32. 32

33. 33

34. 34

35. 35

36. 36

37. 37

38. 38

39. 39

40. 40

41. 41

42. 42

43. 43

44. 44

45. 45

46. 46

47. 47

48. 48

49. 49

50. 50

51. 51

52. 52

53. 53

54. 54

55. 55

56. 56

57. 57

58. The search for standards: the FCC & the NTSC 1953 - The NTSC standard had to be revised to adapt to color TV. Engineers split the signal into two components: luma, which contained the brightness information, and chrominance, which contained the color information. Field refresh rate of 60 Hz was slowed down by a factor of 1000/1001 to 59.94 Hz. Broadcast television downshifted from 30 to 29.97 frames per second

59. Same old standard (The song remains the same) Many improvements were made in cameras, production and broadcast gear, and in television receivers Despite advances, the quality of analog broadcast was still limited to the NTSC standard of 60 fields and 525 horizontal scan lines Stuck with more or less same standards created in 1941.

60. Same old standard (The song remains the same) By the 1980s, manufacturers had been developing and using both analog and digital HD systems It became clear that the replacement for analog would use digital television technology. Needed a new set of standards to ensure compatibility.

61. ATSC Formed in 1982 The Advanced Television Systems Committee is a not-for-profit organization whose purpose is to develop standards for the transition to DTV. Its published broadcast standards are voluntary unless adopted and mandated by the FCC. ATSC proposed DTV Standard (A/53) that specifies the protocol for high-definition broadcasting through a standard 6MHz channel

62. DTV In December 1996, the FCC adopted standards proposed by the ATSC, mandating that broadcasters begin broadcasting digitally. WRAL of Raleigh, North Carolina was the first station to broadcast in digital. FCC terminated analog broadcasting 2009

63. DTV, SDTV, & HDTV NTSC standards defined one analog format ATSC created a framework supporting multiple digital formats There is considerable confusion among consumers regarding SDTV, DTV and HDTV. Broadcaster do not have to broadcast in HD, just in DTV.

64. HDTV & SDTV Comparison

65. DTV formats HDTV/SD TV Horizontal lines Vertical lines Aspect Ratio Frame Rate SDTV6404804:360p, 60i, 30p, 24p SDTV7044804:3 and 16:9 60p, 60i, 30p, 24p HDTV128072016:960p, 30p, 24p HDTV1920108016:960i, 30p, 24p Note: Non-integer formats (eg. 29.97) omitted for clarity.

66. 66 During the transition from monochrome to color television, certain interference constraints needed to be satisfied among the horizontal, sound, and color frequencies. These constraints were resolved by reducing the 60.00 Hz field rate of monochrome television by a factor of exactly 1000 / 1001 to create the color NTSC field rate of about 59.94 Hz. This led to the dropframe timecode that is familiar to anyone that has been involved in videotape editing. http://www.poynton.com/notes/video/Timecode/index.htm l About non-integreer field rates in NTSC video

67. 67 When a transmitter broadcasts an NTSC signal, it amplitude-modulates a radio- frequency carrier with the NTSC signal just described, while it frequency- modulates a carrier 4.5 MHz higher with the audio signal. If non-linear distortion happens to the broadcast signal, the 3.579545 MHz color carrier may beat with the sound carrier to produce a dot pattern on the screen. To make the resulting pattern less noticeable, designers adjusted the original 60 Hz field rate down by a factor of 1.001 (0.1%), to approximately 59.94 fields per second. This adjustment ensures that the sums and differences of the sound carrier and the color subcarrier and their multiples (i.e., the intermodulation products of the two carriers) are not exact multiples of the frame rate, which is the necessary condition for the dots to remain stationary on the screen, making them most noticeable.beatintermodulation http://en.wikipedia.org/wiki/NTSC#Color_encoding

68. Digital Watermarking, Shiraz University of Technology, 89-2 68