Chap 3: Encoding Video Content and Transportation 1

Learning Objectives After completing this learning object, you should be able to: Explain compression goals and characteristics. Describe the technical architecture of the MPEG-2, H.264/AVC, and VC-1 compression standards. 2

Compression Goal The goal of video compression is to reduce the quantity of data used to represent video content without substantially reducing the quality of the picture. Digitization Compression Decode NTSC/PAL Encode / / Transport Compressed Digital Bitstream Uncompressed Analog Video Sequence Film or Video Camera Analog TV Digital

About MPEG An acronym for Moving Pictures Experts Group Widely used by satellite, cable, and terrestrial TV systems The group has produced a family of major compression standard MPEG Format Description MPEG-1 The MPEG-1 file format was originally developed in 1988 and was primarily used to compress video data at bit rates of 1.5 Mbps. MPEG-1 content is used for such services as DAB (Digital Audio Broadcasting) and is the standard format on the Internet for quality video. MPEG-1 is also the basis of the MP3 standard, which is widely used for music on the Internet. MPEG-2 MPEG-2 builds on the powerful video compression capabilities of the MPEG-1 standard. MPEG-2 is widely used in the delivery of broadcast-quality television and storing video content on DVDs. A number of international television standards are based on this compression format. MPEG-4 (Part 2) MPEG-4, whose formal ISO/IEC designation is ISO/IEC 14496, was finalized in October 1998 and became an international standard in 2000. Part 2 of the standard is divided into a number of profiles that address the requirements of various video applications ranging from mobile phones to surveillance cameras. MPEG-4 Part 10 MPEG-4 Part 10 also called H.264/AVC has been designed to deliver broadcast and DVD-quality video at minimum data rates. Also produced specifications for describing A/V content, delivery, and consumption: MPEG-7 MPEG-21 4

Compression Key Concept – Remove Redundancy Compression Algorithms are able to reduce the size of a video bit stream significantly because video typically contains duplicate or redundant information both within and between frames. Image and video compression algorithms can take advantage of two primary types of redundancy to reduce the size of the resulting bit-stream. Spatial Redundancy Temporal Redundancy Spatial compression works on 1 image – Temporal compression works on several images 5

Spatial Compression - Overview Spatial Compression (intra-frame compression): Spatial compression is applied to a single picture Spatially compressed pictures are called I Frames or I Slices Spatial method is performed by taking advantage of the fact that the human eye is unable to distinguish small differences in color. Neighboring pixels in an image often have similar values: color or brightness of an object typically does not vary significantly over small areas. Instead of encoding each pixel individually, a compression algorithm could save bits by encoding only the difference between neighboring pixels. That difference is typically a smaller value than the full range of possible pixel values and therefore can be encoded with fewer bits. 6

Temporal Compression:Intro Temporal compression refers to the bit reduction between successive frames Typical frame rate: 25 or 30 frames/sec Change between two adjacent frames: 1/25 or 1/30 seconds In most case most of the picture is unchanged, while only parts of the picture are moving. There is an obvious redundancy between frames. Temporal compression only encodes the small changes or the direction that a part of the image moved between frames. These changes typically require fewer bits then representing the whole image again. Temporal compression 7

MPEG-2 Frame Types: I Frames I-Frame Characteristics: Starting point for a sequence Undergo minimal compression Independently encoded as a single image No reference to any past or future frames Encoding scheme used is similar to JPEG compression (.25 bits represents a single pixel, whereas 2.5 bits per pixel for higher quality) Self contained and used as a foundation to build other types of frames Effectively a JPEG image I frames are typically large ( 00’s to 000’s of IP packets per frame) I frame is aproximately 64,000 bytes One I-frame will occur approximately every 0.4 seconds of video runtime. More I-Frames make an MPEG stream ‘more editable’. I B B P B 8

MPEG-2 Frame Types: P Frames Forward predicted frames (P-frames) Characteristics: Based on past “I” or “P” frames Moderately compressed Not actually an encoded image Contains motion information (vectors) that allows the IPTVCD to rebuild the frame P-Frames require less bandwidth than I-Frames (Important for IPTV Deployments) P-frames are typically much smaller than I-Frames (0’s to 00’s of packets per frame) . 9

MPEG-2 Frame Types: B-Frames Bi-directional predicted frames (B-frames) Characteristics: Made up from information from both past and future “I” frames and “P” frames Encoding for B-frames is similar to P-frames, except that B-frames can specify two motion vectors (one to past and one to future) Extensive compression - B-frames occupy less space than I-Frames or P-Frames - (0’s to 00’s of packets per frame) A stream containing a high-density of B-frames requires less bandwidth compared to a digital stream built with a high density of I & P frames. When frame dropping occurs, B-frames get discarded first because they have the lowest impact on video quality, compared to I & P frames. 10

Group of Pictures (GOPs) GOP Characteristics: I, P & B images are combined to form a sequence of picture frames called a GOP. Each GOP must begin with a full reference I-Frame - mitigates propagation of errors to one GOP. All frames depend on the contents of the I-Frame GOPs vary in size – Average GOP for IPTV deployments is between 12 and 15 frames in length. Some GOP configurations can however include 250 frames. There are typically between 10 and 12 P- and B-frames occurring between each I-frame. [I B B B P B B B P B B B P B B B P] 2 4 6 8 10 I B P Relative amounts of data for each frame type in a typical MPEG GOP 11

MPEG-2 Shortcomings Although MPEG-2 has served the cable and satellite industries well for the past decade it has shortcomings when deployed on networks that have limited bandwidth capacities: A telephone network was not designed to carry MPEG coded video. New advanced compression schemes with better capabilities have been developed in recent years for the purpose of delivering video content over bandwidth constrained networks: MPEG-4 Part 10 AVC/H.264 Microsoft’s Windows Media Video (also known as VC-1) 12

Example MPEG-4 Compression A farmer herding cattle. The scene can be decomposed into a number of objects: The field and houses in the background The sky The farmer, son, and cattle walking along the road. The farmer’s voice Noises emanating from the cows H.264/AVC treats each one of these objects separately. Compression is applied to each object. 13

Others VC-1 Standardized by the Society of Motion Picture and Television Engineers (SMPTE) Most high profile implementations has been its adoption by Microsoft’s Windows Media Video (WMV) 9 multimedia coding platform A number of other international standards including the high-definition DVD formats HD-DVD and Blu-ray have also adopted VC-1. AVS China has developed a standard called AVS. Efficiency levels achieved by this standard are quite similar to the performance levels achieved by MPEG-4 Part 10. Also covers areas such as digital copyright and content management 14

Transportation Architecture 15

MPEG Streams 1/3 Elementary Streams Packetized Elementary Streams Transport Streams Program streams Video ES Video PES Audio PES Multiple Program Transport Stream Audio ES Video Encoder Audio Packetizer PSIP Data Transport Stream MUX 16

MPEG Streams 2/3 17

Program Streams 3/3 A Program Stream carries a single program In MPEG, a program is a combination of video, audio, and related data All information in the program stream must have a common time-base. Aimed for error-free environments like DVD files, etc Video PES + Audio PES 1 + Audio PES 2 = Program Stream 1 Pack Pack Header 18

PES Packet Overview In order for the audio, data, and video elementary streams to be transmitted over the digital network, each elementary stream is converted into an interleaved stream of time stamped PES packets. PES Packet Characteristics: A PES packet may be a fixed (or variable) sized block Up to 65536 bytes per packet Each PES packet has a header - 6 bytes Includes a number that identifies the ES, this is important when combining audio & video sources Also includes a time stamp for synchronization of ES’s Due to networking difficulties the order of video frames outputted from the IPTV data center can be different to the order that they are received by the IPTVCD. Thus to help synchronization, MPEG based systems often time stamp PES packets Remainder of PES packet used for video content 19

Time Stamping PES Packets There are two types of time stamps that can be applied to each PES packet: Presentation Time Stamps (PTS) It’s purpose is to define when and in what order the video should be presented/displayed to the viewer. 33-bit time value Set in the PES Header Field Decode Time Stamp (DTS) It’s purpose is to instruct the IPTVCD decoder when to process the packets. These time stamps are based on the encoders system time clock (STC). For synchronization to occur the STC needs to also be relevant at the IPTVCD decoder. Thus, the encoder uses STC to time stamp a program clock reference (PCR) in the MPEG-2 transport stream packets. These values are used by the decoder to synchronise its own 27MHz clock to the encoders STC. 20

Program Clock Reference (PCR) To assist the decoder in presenting programs on time, at the right speed, and with synchronization, programs usually periodically provide a Program Clock Reference, or PCR, on one of the PIDs in the program The PCR is a 42-bit number. The PCR time stamp carried by each program is derived from the 27 MHz source clock (STC) . This results in an accuracy of 1 in 27,000,000 or 37ns The adaptation field in the packet header is used periodically to include the PCR code that allows generation of a locked clock at the decoder. PCR’s should be spaced no more than 40 ms apart according to some international standards

MPEG TS Packets Overview A TS is formed by breaking up either elementary streams or program streams or both into MPEG TS packets. About MPEG TS Packets Formed by breaking up the PES packets Fixed-size of 188 bytes Each TS packet contains one of the three media formats Video Audio Data Thus; TS packets do not support a mix of media 22

TS MPEG Packet Structure 23

Structure of an MPEG TS Packet Field Name Description of Functionality Synchronization Byte The header starts with a well-known Synchronization Byte (8 bits). This has the bit pattern 0x47 (0100 0111) and is used to detect the start of the MPEG TS packet. Transport error indicator This single bit flag indicates an error in the associated transport stream. It is set by the encoder, when it detects corrupted source content. This purely identifies source content issue and is not an indicator of distribution network problems. Start indicator This flag indicates the start of the video payload. Transport Priority When set this flag identifies priority level of the video payload. Program Identifier (PID) The most important field of the header is the 13 bits that define the program identifier. This uniquely identifies the stream that the packet belongs to. All packets belonging to this stream will have the same PID value. This information is used by the demultiplexer in the IPTVCD to distinguish between different packet types. The bulk of the packets are video, followed by audio packets and null packets for unused space. Null packets are always assigned a PID value of 8191. Packets that have no PID values are typically discarded by the receiving IPTVCD. 24

Structure of an MPEG TS Packet Field Name Description of Functionality Transport scrambling control This two-bit field indicates the encryption status of the transport stream packet payload. Adaption field control This two-bit field indicates whether the associated transport stream packet header includes an adaptation field and payload. Continuity counter The continuity counter increments by one each time a transport stream packet with the same PID value is passed through the MPEG system. This helps to identify lost or duplicate packets, which could affect the quality of the video been viewed by the IPTV subscriber. Adaptation field The adaptation field contains a variety of data used for timing and control including the Program Clock Reference (PCR). The PCR is used to synchronise the IPTVCD clock with the source encoder clock. PCR values are 42-bits in length and increment according to a standard clock rate of 27 MHz. Once synchronization has taken place the decoding of the IPTV MPEG-2 stream can occur. 25

RTP This optional layer is used by a wide variety of IPTV applications It acts as an intermediary between the H.264/AVC, MPEG-2, or VC-1 encoded content in the higher layers and the lower sections of the IPTVCM. Real-time Protocol is the foundation block of this layer. Originally designed for real-time streaming of media content across an IP network 26

IPTVCM Transport Layer RTP packets form the input to the transport layer Possible to map MPEG-TS packets directly into the transport layer protocol payload; avoiding the RTP layer Transport Layer Characteristics: Hide the intricacies of the IP network structure from the upper-layer processes Provide for the reliability and integrity of the end-to-end communication link If video data is not delivered to the IPTVCD correctly, the transport layer can initiate retransmission Alternatively, it can inform the upper layers which can then take the necessary corrective action Two primary transport protocols: Transmission Control Protocol (TCP) User Datagram Protocol (UDP) 27

IP packet with MPEG2 TS video payload carried over Ethernet IP Encapsulation IP Encapsulation is the process of taking a data stream, formatting it into packets, and adding the headers and other data required. MPEG over IP Transport streams consist of a series of multiple MPEG TS packets packed inside UDP datagrams A typical IP video packet will contain 7 TS packets (188 x 7 = 1316 bytes) Add Ethernet, IP and UDP headers (64 bytes) 1316 bytes + 64 bytes _________________ = 1380 bytes Ethernet IP/UDP MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes MPEG2 TS Video Packet 188 bytes CRC IP packet with MPEG2 TS video payload carried over Ethernet

Traveling up the IPTVCM When data is received at the IPTVCD; the following occurs: Encapsulation process is reversed De-capsulation at the data link layer involves: Inspecting the packet Removing the Ethernet header and the CRC fields Examines type code & determines the packet needs to be processed by the IP protocol Packets are passed upwards to the IP layer De-capsulation at the IP layer involves: Inspects packet Removes the IP header Packets are passed upwards to the UDP layer This process continues until the packets reach the top of the IPTVCM and the raw video gets displayed on the viewers TV screen 29