by Abhishek Hassan Thungaraj Supervisor- Dr. K. R. Rao

Outline Basics of a Digital Video Need for compression Areas for compression Video codecs Introduction to HEVC Features of HEVC Present and Proposed algorithms Experimental conditions and Results Conclusions

Digital Video What makes a video? o Group of Pixels > Images o Series of Image at certain speed > Video What are its features? o Types – 2-D, 3-D, HDR o frames per second o Resolution o Video size

Need for compression Internet and its ability to stream video o VoIP – Online streaming, online video games, video conference, internet TV Introduction of faster hand-held devices Bandwidth limitations

What to compress? Discard what cannot be seen o Pixel Representation - Eye perceives Intensity better than Color Discard Redundancies o Spatial o Temporal

Areas of compression Spatial redundancies o Large homogenous regions

Areas of compression Temporal redundancies o Between every second there are 30 frames! - Implies adjacent frames are almost identical

Who exploits them? – Video Codecs

High Efficiency Video Coding-HEVC Latest video coding standard by Joint Collaborative Team on Video Coding (JCT-VC) in January, 2013 Can address all applications of H.264/MPEG-4 AVC [8] 50% bitrate reduction over H.264/AVC with same quality Supports parallel processing architectures

Encoder block[8]

Partitioning in HEVC

Coding Unit (CU)

Coding Tree Unit (CTU)

Prediction Unit and Transform Unit

Partitioning in a Video frame

Inter-Prediction in HEVC Motion Estimation Motion Compensation

Result of Motion Estimation Mode information indicating the mode Reference indices indicating the Reference Picture Motion Vector i.e. horizontal and vertical displacement values which directs to the reference PU in the reference picture

Motion Vector in a video frame

Motion Information after Motion Estimation

Motion Information after Motion Merge

Motion Merge in HEVC Objects in images have homogenous motion Effectively cluster of PUs could have same Motion Information ! Solution – o Indicate such cluster of PUs to follow one base PU thus reducing redundancy in motion information

Spatial Merge Mode in HEVC Evaluation of 4 spatial candidates among 5 candidates at different positions in the order A 1 - B 1 - B 0 - A 0 - B 2 If a candidate has identical motion information - mark it as the true candidate Encode all five candidates

Limitations and Motivations The candidates must have motion information and itself cannot be in a merge mode Hence all candidate PUs must obtain motion information from their root PU which increases the time overhead Limits the maximum size of a merging block to neighboring areas

Proposed algorithm Step 1: Evaluate if the current CU is greater than the threshold size.  If yes proceed to step 2 else go to step 7 Step 2: Check whether the candidate PU is in merge mode.  If yes proceed to step 3 else go to step 4 Step 3: Locate the base PU of the candidate PU and term it as the candidate PU

Proposed algorithm (cont.) Step4: Check if the Motion Information matches with PU  If Yes, Proceed to Step5 else go to step 6 Step 5: Select the PU as the base PU and terminate future evaluations. Step 6: Select the next candidate PU and go to step 1 Step 7: Select all of its descending PUs to follow merge mode and select the top left PU as the base PU

Test conditions Source code: HEVC Reference software HM 13.0 [38] Platform: Windows 7 64-bit OS on 16 GB RAM at 3.70 GHz on Intel Xenon E v2 processor Profile: ‘random access profile’ GOP length: 8 CTB size: 64x64 with minimum CU size of 8x8. Quantization Parameters: 22, 27, 32, 37

Test Sequences

Metric - Encoding time Indicated the time taken by the encoder in terms of seconds. Indicates the fastness of the codec and its underlying algorithm Depends on availability of resources

Encoding time gain (1)

Encoding time gain (2)

Metric- Bitrate Rate of the bitstream generated by the codec Measured in kilo bits per sec (kbps) Indicates the compression performance of the codec and the underlying algorithm

Bitrate vs. QP (1)

Bitrate vs. QP (2)

Metric - Peak Signal to Noise Ration (PSNR) Useful signal among the total signal Measured in terms of decibels (dB) Indicates the quality of the encoded data generated by the codec and the underlying algorithm

PSNR vs. QP (1)

PSNR vs. QP (2)

Bjontegaard Delta metrics (BD-metrics) Useful for comparing two codecs or two different algorithms used in a codec Makes an Rate-Distortion (R-D) comparison using the generated bitstream and its effectiveness BD-PSNR (dB): +ve value indicates an improvement BD-bitrate (%): -ve value indicates an improvement

BD-PSNR vs. QP (1)

BD-PSNR vs. QP (2)

BD-bitrate vs. QP (1)

BD-bitrate vs. QP (2)

Quality and bitrate comparison The quality of the original and the proposed algorithm in terms of PSNR in decibels (dB) The data size of the original and the proposed algorithm in terms of bitrate in kilo bits per sec (kbps) Provides a bird view of gains against losses of the proposed algorithm over original algorithm

PSNR vs. Bitrate (1)

PSNR vs. Bitrate (2)

Summary of Results Reduction in encoding time by 13% - 24 % Reduction in bitrate by 2% - 7% Slight drop in PSNR of 2% - 6% Positive value of BD-PSNR ranging from 0.29 to 0.56 dB Dip in BD-bitrate ranging from -31% to -65%

Conclusions Reduction in complexity has lead to reduction of encoding time  making the codec faster Reduction in bitrate as a result of larger merge blocks  making it easier to transmit the codec data Slight drop in quality as a tradeoff BD metrics suggests proposed algorithm as an improvement over existing algorithm as the gains are greater than losses

Future work Single Processor used – can be made much faster using Parallel Processors like GPUs Integrating with improved algorithms of Intra/Inter prediction produces faster and better compression Associating with Scalable HEVC (SHVC) provides wide range of applications Can be extended to Intra frames and temporal merging

Acronyms AVC - Advanced Video Coding AMVP – Advanced Motion Vector Prediction BD - Bjontegaard Delta CABAC – Context Adaptive Binary Arithmetic Coding CB – Coding Block CBF – Coding Block Flag CFM – CBF Fast Mode CTU – Coding Tree Unit CTB – Coding Tree Block CU – Coding Unit DCT – Discrete Cosine Transform DST – Discrete Sine Transform

Acronyms HDTV - High Definition Tele Vision HDR - High Dynamic Range HDRI - High Dynamic Range Imaging HEVC – High Efficiency Video Coding HM – HEVC Test Model HVS – Human Visual System ISO – International Standards Organization ITU – International Telecommunications Union JCT-VC - Joint Collaborative Team on Video Coding MB – Macroblock MC – Motion Compensation ME – Motion Estimation

Acronyms MPEG – Moving Picture Experts Group NAL – Network Abstraction Layer PB – Prediction Block PSNR – Peak Signal to Noise Ratio PU – Prediction Unit QP – Quantization Parameter RDOQ – Rate Distortion Optimization Quantization RGB – Red Green Blue RMD – Rough Mode Decision SATD – Sum of Absolute Transform Differences SD – Standard Definition SSIM – Structural Similarity

Acronyms TB – Transform Block TU – Transform Unit URQ – Uniform Reconstruction Quantization VCEG – Video Coding Experts Group VPS – Video Parameter Set WQVGA – Wide Quarter Video Graphics Array WVGA – Wide Video Graphics Array

References

Thank you