1. Chap 3: Encoding Video Content and Transportation1

2. Learning Objectives
After completing this learning object, you should be able to:
Explain compression goals and characteristics.
Describe the technical architecture of the MPEG-2, H.264/AVC, and VC-1 compression standards. 2

3. Compression Goal
The goal of video compression is to reduce the quantity of data used to represent video content without substantially reducing the quality of the picture.
Digitization
Compression
Decode
NTSC/PAL
Encode
/ /
Transport
Compressed
Digital Bitstream
Uncompressed
Analog Video
Sequence
Film or
Video
Camera
Analog TV
Digital

4. About MPEG An acronym for Moving Pictures Experts Group
Widely used by satellite, cable, and terrestrial TV systems
The group has produced a family of major compression standard
MPEG Format
Description
MPEG-1
The MPEG-1 file format was originally developed in 1988 and was primarily used to compress video data at bit rates of 1.5 Mbps. MPEG-1 content is used for such services as DAB (Digital Audio Broadcasting) and is the standard format on the Internet for quality video. MPEG-1 is also the basis of the MP3 standard, which is widely used for music on the Internet.
MPEG-2
MPEG-2 builds on the powerful video compression capabilities of the MPEG-1 standard. MPEG-2 is widely used in the delivery of broadcast-quality television and storing video content on DVDs. A number of international television standards are based on this compression format.
MPEG-4 (Part 2)
MPEG-4, whose formal ISO/IEC designation is ISO/IEC 14496, was finalized in October 1998 and became an international standard in Part 2 of the standard is divided into a number of profiles that address the requirements of various video applications ranging from mobile phones to surveillance cameras.
MPEG-4 Part 10
MPEG-4 Part 10 also called H.264/AVC has been designed to deliver broadcast and DVD-quality video at minimum data rates.
Also produced specifications for describing A/V content, delivery, and consumption:
MPEG-7
MPEG-21 4

5. Compression Key Concept – Remove Redundancy
Compression Algorithms are able to reduce the size of a video bit stream significantly because video typically contains duplicate or redundant information both within and between frames.
Image and video compression algorithms can take advantage of two primary types of redundancy to reduce the size of the resulting bit-stream.
Spatial Redundancy
Temporal Redundancy
Spatial compression works on 1 image – Temporal compression works on several images 5

6. Spatial Compression - Overview
Spatial Compression (intra-frame compression):
Spatial compression is applied to a single picture
Spatially compressed pictures are called I Frames or I Slices
Spatial method is performed by taking advantage of the fact that the human eye is unable to distinguish small differences in color.
Neighboring pixels in an image often have similar values: color or brightness of an object typically does not vary significantly over small areas.
Instead of encoding each pixel individually, a compression algorithm could save bits by encoding only the difference between neighboring pixels.
That difference is typically a smaller value than the full range of possible pixel values and therefore can be encoded with fewer bits. 6

7. Temporal Compression:Intro
Temporal compression refers to the bit reduction between successive frames
Typical frame rate: 25 or 30 frames/sec
Change between two adjacent frames: 1/25 or 1/30 seconds
In most case most of the picture is unchanged, while only parts of the picture are moving. There is an obvious redundancy between frames.
Temporal compression only encodes the small changes or the direction that a part of the image moved between frames.
These changes typically require fewer bits then representing the whole image again.
Temporal compression 7

8. MPEG-2 Frame Types: I Frames
I-Frame Characteristics:
Starting point for a sequence
Undergo minimal compression
Independently encoded as a single image
No reference to any past or future frames
Encoding scheme used is similar to JPEG compression (.25 bits represents a single pixel, whereas 2.5 bits per pixel for higher quality)
Self contained and used as a foundation to build other types of frames
Effectively a JPEG image
I frames are typically large ( 00’s to 000’s of IP packets per frame)
I frame is aproximately 64,000 bytes
One I-frame will occur approximately every 0.4 seconds of video runtime.
More I-Frames make an MPEG stream ‘more editable’. I B B P B 8

9. MPEG-2 Frame Types: P Frames
Forward predicted frames (P-frames) Characteristics:
Based on past “I” or “P” frames
Moderately compressed
Not actually an encoded image
Contains motion information (vectors) that allows the IPTVCD to rebuild the frame
P-Frames require less bandwidth than I-Frames (Important for IPTV Deployments)
P-frames are typically much smaller than I-Frames (0’s to 00’s of packets per frame) . 9

10. MPEG-2 Frame Types: B-Frames
Bi-directional predicted frames (B-frames) Characteristics:
Made up from information from both past and future “I” frames and “P” frames
Encoding for B-frames is similar to P-frames, except that B-frames can specify two motion vectors (one to past and one to future)
Extensive compression - B-frames occupy less space than I-Frames or P-Frames - (0’s to 00’s of packets per frame)
A stream containing a high-density of B-frames requires less bandwidth compared to a digital stream built with a high density of I & P frames.
When frame dropping occurs, B-frames get discarded first because they have the lowest impact on video quality, compared to I & P frames. 10

11. Group of Pictures (GOPs)
GOP Characteristics:
I, P & B images are combined to form a sequence of picture frames called a GOP.
Each GOP must begin with a full reference I-Frame - mitigates propagation of errors to one GOP.
All frames depend on the contents of the I-Frame
GOPs vary in size – Average GOP for IPTV deployments is between 12 and 15 frames in length. Some GOP configurations can however include 250 frames.
There are typically between 10 and 12 P- and B-frames occurring between each I-frame.
[I B B B P B B B P B B B P B B B P] 2 4 6 8 10 I B P
Relative amounts of data for each
frame type in a typical MPEG GOP 11

12. MPEG-2 Shortcomings
Although MPEG-2 has served the cable and satellite industries well for the past decade it has shortcomings when deployed on networks that have limited bandwidth capacities:
A telephone network was not designed to carry MPEG coded video.
New advanced compression schemes with better capabilities have been developed in recent years for the purpose of delivering video content over bandwidth constrained networks:
MPEG-4 Part 10 AVC/H.264
Microsoft’s Windows Media Video (also known as VC-1) 12

13. Example MPEG-4 Compression
A farmer herding cattle.
The scene can be decomposed into a number of objects:
The field and houses in the background
The sky
The farmer, son, and cattle walking along the road.
The farmer’s voice
Noises emanating from the cows
H.264/AVC treats each one of these objects separately.
Compression is applied to each object. 13

14. Others
VC-1
Standardized by the Society of Motion Picture and Television Engineers (SMPTE)
Most high profile implementations has been its adoption by Microsoft’s Windows Media Video (WMV) 9 multimedia coding platform
A number of other international standards including the high-definition DVD formats HD-DVD and Blu-ray have also adopted VC-1.
AVS
China has developed a standard called AVS.
Efficiency levels achieved by this standard are quite similar to the performance levels achieved by MPEG-4 Part 10.
Also covers areas such as digital copyright and content management 14

15. Transportation Architecture15

16. MPEG Streams 1/3 Elementary Streams Packetized Elementary Streams
Transport Streams
Program streams
Video ES
Video PES
Audio PES
Multiple Program
Transport Stream
Audio ES
Video
Encoder
Audio
Packetizer
PSIP
Data
Transport
Stream
MUX 16

17. MPEG Streams 2/3 17

18. Program Streams 3/3 A Program Stream carries a single program
In MPEG, a program is a combination of video, audio, and related data
All information in the program stream must have a common time-base.
Aimed for error-free environments like DVD files, etc
Video PES
+ Audio PES 1
+ Audio PES 2
= Program Stream
1 Pack
Pack Header 18

19. PES Packet Overview
In order for the audio, data, and video elementary streams to be transmitted over the digital network, each elementary stream is converted into an interleaved stream of time stamped PES packets.
PES Packet Characteristics:
A PES packet may be a fixed (or variable) sized block
Up to bytes per packet
Each PES packet has a header - 6 bytes
Includes a number that identifies the ES, this is important when combining audio & video sources
Also includes a time stamp for synchronization of ES’s
Due to networking difficulties the order of video frames outputted from the IPTV data center can be different to the order that they are received by the IPTVCD.
Thus to help synchronization, MPEG based systems often time stamp PES packets
Remainder of PES packet used for video content 19

20. Time Stamping PES Packets
There are two types of time stamps that can be applied to each PES packet:
Presentation Time Stamps (PTS)
It’s purpose is to define when and in what order the video should be presented/displayed to the viewer.
33-bit time value
Set in the PES Header Field
Decode Time Stamp (DTS)
It’s purpose is to instruct the IPTVCD decoder when to process the packets.
These time stamps are based on the encoders system time clock (STC).
For synchronization to occur the STC needs to also be relevant at the IPTVCD decoder.
Thus, the encoder uses STC to time stamp a program clock reference (PCR) in the MPEG-2 transport stream packets.
These values are used by the decoder to synchronise its own 27MHz clock to the encoders STC. 20

21. Program Clock Reference (PCR)
To assist the decoder in presenting programs on time, at the right speed, and with synchronization, programs usually periodically provide a Program Clock Reference, or PCR, on one of the PIDs in the program
The PCR is a 42-bit number.
The PCR time stamp carried by each program is derived from the 27 MHz source clock (STC) .
This results in an accuracy of 1 in 27,000,000 or 37ns
The adaptation field in the packet header is used periodically to include the PCR code that allows generation of a locked clock at the decoder.
PCR’s should be spaced no more than 40 ms apart according to some international standards

22. MPEG TS Packets Overview
A TS is formed by breaking up either elementary streams or program streams or both into MPEG TS packets.
About MPEG TS Packets
Formed by breaking up the PES packets
Fixed-size of 188 bytes
Each TS packet contains one of the three media formats
Video
Audio
Data
Thus; TS packets do not support a mix of media 22

23. TS MPEG Packet Structure23

24. Structure of an MPEG TS Packet
Field Name
Description of Functionality
Synchronization Byte
The header starts with a well-known Synchronization Byte (8 bits). This has the bit pattern 0x47 ( ) and is used to detect the start of the MPEG TS packet.
Transport error indicator
This single bit flag indicates an error in the associated transport stream.
It is set by the encoder, when it detects corrupted source content.
This purely identifies source content issue and is not an indicator of distribution network problems.
Start indicator
This flag indicates the start of the video payload.
Transport Priority
When set this flag identifies priority level of the video payload.
Program Identifier (PID)
The most important field of the header is the 13 bits that define the program identifier.
This uniquely identifies the stream that the packet belongs to. All packets belonging to this stream will have the same PID value.
This information is used by the demultiplexer in the IPTVCD to distinguish between different packet types. The bulk of the packets are video, followed by audio packets and null packets for unused space.
Null packets are always assigned a PID value of 8191.
Packets that have no PID values are typically discarded by the receiving IPTVCD. 24

25. Structure of an MPEG TS Packet
Field Name
Description of Functionality
Transport scrambling control
This two-bit field indicates the encryption status of the transport stream packet payload.
Adaption field control
This two-bit field indicates whether the associated transport stream packet header includes an adaptation field and payload.
Continuity counter
The continuity counter increments by one each time a transport stream packet with the same PID value is passed through the MPEG system. This helps to identify lost or duplicate packets, which could affect the quality of the video been viewed by the IPTV subscriber.
Adaptation field
The adaptation field contains a variety of data used for timing and control including the Program Clock Reference (PCR).
The PCR is used to synchronise the IPTVCD clock with the source encoder clock.
PCR values are 42-bits in length and increment according to a standard clock rate of 27 MHz.
Once synchronization has taken place the decoding of the IPTV MPEG-2 stream can occur. 25

26. RTP This optional layer is used by a wide variety of IPTV applications
It acts as an intermediary between the H.264/AVC, MPEG-2, or VC-1 encoded content in the higher layers and the lower sections of the IPTVCM.
Real-time Protocol is the foundation block of this layer.
Originally designed for real-time streaming of media content across an IP network 26

27. IPTVCM Transport Layer
RTP packets form the input to the transport layer
Possible to map MPEG-TS packets directly into the transport layer protocol payload; avoiding the RTP layer
Transport Layer Characteristics:
Hide the intricacies of the IP network structure from the upper-layer processes
Provide for the reliability and integrity of the end-to-end communication link
If video data is not delivered to the IPTVCD correctly, the transport layer can initiate retransmission
Alternatively, it can inform the upper layers which can then take the necessary corrective action
Two primary transport protocols:
Transmission Control Protocol (TCP)
User Datagram Protocol (UDP) 27

28. IP packet with MPEG2 TS video payload carried over Ethernet
IP Encapsulation
IP Encapsulation is the process of taking a data stream, formatting it into packets, and adding the headers and other data required.
MPEG over IP Transport streams consist of a series of multiple MPEG TS packets packed inside UDP datagrams
A typical IP video packet will contain 7 TS packets (188 x 7 = 1316 bytes)
Add Ethernet, IP and UDP headers (64 bytes)
1316 bytes
+ 64 bytes
_________________
= 1380 bytes
Ethernet
IP/UDP
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
MPEG2 TS
Video Packet
188 bytes
CRC
IP packet with MPEG2 TS video payload carried over Ethernet

29. Traveling up the IPTVCM
When data is received at the IPTVCD; the following occurs:
Encapsulation process is reversed
De-capsulation at the data link layer involves:
Inspecting the packet
Removing the Ethernet header and the CRC fields
Examines type code & determines the packet needs to be processed by the IP protocol
Packets are passed upwards to the IP layer
De-capsulation at the IP layer involves:
Inspects packet
Removes the IP header
Packets are passed upwards to the UDP layer
This process continues until the packets reach the top of the IPTVCM and the raw video gets displayed on the viewers TV screen 29