1. Video Processing Dinesh Ganotra

2. Terminologies Frames = Images frame size : WxH Color depth : Pixel represented by fixed amount of bits Duration: T FPS: Frames per Second Pixel per frame: WxHxCD Bits per frame: Bit rate: Video Size: Low Level profile: Minimum video specification that any video decoder should be able to handle

3. Peak Signal-to-Noise Ratio (PSNR), a metric for the "quality" of compressed video images, while the horizontal axis shows the transmission bit rate.10 frames per second, 176x144 (QCIF) resolution, 10 seconds of video, (100 frames)

4. Microsoft AVI : Audio/Video Interleaved File Format ( Video for Windows: 1992) Do not have pixel aspect ratio information Hdrl tag Width, height, frame rate movi tag Audio, video data Idex1 tag Indexing info

5. Apple MOV: Quicktime File Format Container file that contains one or more tracks, each of which stores a particular type of data: audio, video, effects, or text (e.g. for subtitles). – Can also be data reference to the media stream located in another file – Suited for editing.

6. MPEG-1 color-space is transformed to YCbCr Chroma is also sub-sampled to 4:2:0, meaning it is reduced by one half vertically and one half horizontally Because of sub-sampling, YCbCr video must always be stored using even dimensions (divisible by 2), otherwise chroma mismatch ("ghosts") will occur, and it will appear as if the color is ahead of, or behind the rest of the video, much like a shadow.

7. Frame Types Intra-frames: can be decoded independently of any other frame. Effectively identical to baseline JPEG. GOP: group of pictures: typically 17-18 non I-frames are their between two I –frames Predicted-frames: improve compression by exploiting the temporal (over time) redundancy in a video. P-frames store only the difference in image from the frame (either an I- frame or P-frame) immediately preceding it. Bidirectional Frames: Can make prediction using both previous and future frames. Often avoided, not fully supported by hardware. D-frame: only DC component is encoded. Poor quality images, only used to find frames in videos. Obsolete.

9. MacroBlocks 8x8 blocks for quantization. However, because chroma (color) is subsampled by a factor of 4, each pair of (red and blue) chroma blocks corresponds to 4 different luma blocks. This set of 6 blocks, with a resolution of 16x16, is called a macroblock.

10. Motion Vector The encoder compares the current frame with adjacent parts of the video from the anchor frame (previous I- or P- frame) in a diamond pattern, up to a (encoder-specific) predefined radius limit from the area of the current macroblock. If a match is found, only the direction and distance (i.e. the vector of the motion) from the previous video area to the current macroblock need to be encoded into the inter-frame (P- or B- frame).

11. MPEG-2 Interlacing and high-definition

12. MPEG-4 Intellectual property protection Support B-Frames

13. DIVX Digital Video Express Interactive video menus Multiple subtitles (XSUB) Multiple audio tracks

14. WMV Window Media Video

15. 3GP Third Generation Partnership Project

16. FLV Flash Live Video Originally developed by Macromedia, now maintained by Adobe Systems Contain material encoded with codecs following the Sorenson Spark or VP6 video compression formats, protected by patents

17. Year Wise Progress YearStandardApplication 1984 H.120 1990 H.261 Videoconferencing, Videotelephony 1993 MPEG-1 ISO, IEC Video-CD 1995 H.262/MPEG-2DVD Video, Blu-Ray, Digital Video Broadcasting, SVCD 1996 H.263 T Videoconferencing, Videotelephony, Video on Mobile Phones (3GP) 1999 MPEG-4 Video on Internet (DivX, Xvid...) 2003 H.264/MPEG-4 Blu-Ray, Digital Video Broadcasting, iPod Video, HD DVD

18. Video Codecs are generally installed in the windows installation mrle Microsoft Run Length Encoding (msrle32.dll) msvc Microsoft Video 1 (msvidc32.dll) i420 Intel Indeo 4 codec (iyuv_32.dll) cvid Supermac Cinepak (iccvid.dll)

19. 8,8,16,16 All the video compression algorithms work on the DCT 8 cross 8 pixel blocks and macro- blocks are 16 cross 16 pixels block. So, its advisable to work with frames whose sizes are multiples of 16 and avoid exception handling algorithms and unnecessarily slowing down the compression/decompression algorithms.

20. Support for B- frames Variable bit rate audio Variable frame rate Captions (Subtitles) Video formats supported Audio formats supported Metadata/Tags supportedMenu Support 3GPYes 3GPP Timed Text H.263, MPEG-4 Part 2 and H.264/MPEG-4 AVC AMR-NB, AMR- WB, AMR-WB+, AAC, HE-AAC and HE-AAC v2?No FLVYes? No Sorenson, VP6, Screen Video, H.264/MPEG-4 AVC MP3, Nellymoser, ADPCM, Linear PCM, AAC[9], Speex[10]YesNo movYes Limited to what is available to the QuickTime codec manager Limited to what is available to Sound Manager or CoreAudioYes Can be done by using sprite tracks divxYes MPEG-4 Part 2 video encoded to meet DivX profilesMP3, PCM, AC-3?Yes wmvYes Almost anything through VFW or DMO; H.264/AVC is problematic Almost anything through ACM or DMO; Vorbis is problematicYes? AVIYes Almost anything through VFW; H.264/AVC is problematic due to the limited B- frame support Almost anything through ACM; Vorbis is problematic [5]not officiallyNo mpegYes NoMPEG-1, MPEG-2 MPEG-1 Layers I, II, III (mp3), other formats only in private streams: LPCM[15]No

21. Algorithm for m2v format BitStream []; BitStream = Alignbits(BitStream) BitStream = PutSequenceHeader(BitStream) BitStream = Alignbits(BitStream) BitStream = PutSequenceExtension(BitStream) BitStream = Alignbits(BitStream) BitStream = PutSequenceDisplayExtension(BitStream) BitStream = Alignbits(BitStream) BitStream = PutGOPHeader(BitStream)

22. frame_Y_trfmd(row:row+7,col:col+7) = floor(dct2(frame_Y(row:row+7, col:col+7)) + 0.5); BitStream = PutPictureHeader(BitStream); BitStream = Alignbits(BitStream); BitStream = PutPictureCodingExtn(BitStream); frame_Y_trfmd_Quant(row:row+7,col:col+7) = round((frame_Y_trfmd(row:row+7,col:col+7)./(intra_quant_mat.*Q UANT)).*tempmat); BitStream = Alignbits(BitStream); BitStream = PutSliceHeader(ind_slice, BitStream); % Each row is a slice