1. 1000449352 Shiba Kuanar Shiba.kuanar@mavs.uta.edu
Analysis of Motion Estimation Algorithm (HEVC), using Multi-core processing
Shiba Kuanar
EE5359 – Interim Report

2. CONTENTS:
OVERVIEW OF HEVC
SEARCH ALGORITHEM
RESULTS
REFERENCES

3. HEVC
High Efficiency Video Coding (HEVC) is the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group.
The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards in the range of 50% bit-rate reduction for equal perceptual video quality.
Video Coding Standards: Always have a trade between
1) Picture Quality
2) Compression Ratio
3) Computation complexity

4. Fig 1:HEVC Encoder

5. HEVC Partitioning – into non overlapping macro blocks
Prediction - Forward/Backward based on current, past and future frame
Error signal - transformed and quantized
Entropy coded

6. Figure 2: HEVC decoder block diagram [2]

7. Fig 3: Partitioning of a Macro block
CTB->CBs andTBs
Solid lines indicate CB boundaries
and dotted lines indicate TB boundaries
Corresponding QUAD TREE

8. Motion Estimation(ME)
Block Based ME algorithms are proposed to reduce the computation time. Algorithms will be discussed the current proposal
“Test Zone (TZ) search”
“Full search algorithm”
Proposed work is to analyze the existing algorithm and try to reduce motion estimation computation time using multicore programming.
“Motion estimation consumes more than 84% encoding time and coding complexity to encode
HM13.0 – latest HEVC software.
Block Matching Criterions used in BBME
- Mean of Square Error
- Mean of absolute difference (No multiplication)
- Matching Pixel Count.

9. Figure: 4. [22], Percentage “execution time” distributions of the HEVC encoder steps, for an HD sequence

10. Block based Motion Estimation
In HEVC video coding, motion estimation (ME) plays a vital role in temporary redundancy reduction between the frames.
In BBME, the current frame is divided into N ×N pixel size macro blocks (MBs) and for each MB a certain area of the reference frame is searched, to minimize a block difference measure (BDM).
The block difference measure is usually a sum of absolute differences (SAD) between the current and the reference frame MB.
The displacement within the search area which gives the minimum BDM value is called a motion vector (MV).
MVs together with transformed and quantized block differences (residua) are entropy coded into the video bit stream.
To decrease such a huge computational burden, many fast BBME algorithms have been proposed. On the current proposal only two algorithms are going to be analyzed 1) Algorithm based on the search position number and 2) algorithm based on reduction of fast full search.

11. Motion Estimation (ME)[19]
“Pel by Pel” or “Block by Block motion” algorithm used for ME.
ME improves the prediction accuracy between current frame and reference frame.
In Block matching algorithm (BMA), motion of a block of pels (M*N) within a frame interval is estimated.
The range of the motion vector is constrained by the search window.
Best match between the (M*N) block on current frame and corresponding block in previous frame calculated within a search window (M+2*m2) * (N+2*n1) (figure 5)
BMA uses minimum mean absolute error, mean squared error cost functions to find the best match.

12. Figure 5: Motion Estimation of an (M
Figure 5: Motion Estimation of an (M*N) Block in the previous frame within an [19] [(M+2*m2) * (N+2*n1)] search window. MV range is [-n1 to n1] pels and [-m2 to m2] rows
(2d+ 1)² search points (SPs)

13. Motion Estimation
The cost function computed every pel and line displacement within search window & compared to find the location of optimal cost function
To find the minimum cost function in figure 5, requires (2m2+1)*(2n1+1) times computation for every possible displacement constrained by the MV range.
Once ME carried out based on integer pel MV resolution surrounding the best match then MV can be extended to half a pel /line by interpolating the fractional pels.
So ME calculation computationally intensive and many ME algorithms are developed.

14. Figure 6: Motion Estimation at the eight fractional pels surrounding the integer pel locations of the MV (pel a)

15. Fig 7: BBME Algorithm for a) 2DLOG, b) TSS and c) NTSS search

16. Fig 8: Logarithmic Search algorithm

17. Figure 9: Various Search Patterns: a) Diamond search, b) Square Search, c) Horizontal Hexagonal search, d) Vertical Hexagonal Search, e)Rotating Hexagon Type - 1 and f) Rotating Hexagon Type - 2 [17, 20]
a) Diamond search b) Square Search c) Horizontal Hexagonal d) Vertical Hexagonal
e) Rotating Hexagon Type – f) Rotating Hexagon Type – 2

18. Test zone (TZ) Search Algorithm Process (4 steps)
TZ Search algorithm is one of the fast search algorithms in HEVC motion estimation and reduces the encoding time.
1) Motion Vector Prediction: TZS algorithm employs median predictor, left predictor, up predictor, upper right predictor. The minimum of these predictors is selected as a starting location for further search steps.
2) Initial Grid Search: In this step, the algorithm searches the search window in using diamond or square patterns with different stride lengths ranging from1 through 64, in multiples of 2.
3) Raster Search: The raster search is a simple full-search on a down-sampled version of the search window.
4) Raster/Star Refinement: The Raster refinement is a fine refinement of the motion vectors obtained from the step three.

19. Figure 10 Flowchart of TZS Algorithm [28]

20. Test Sequence 1 -BasketballDrill [24] Resolution = 832x480 and frame = 50

21. Test Sequence 2RaceHorses [24], Resolution = 832x480 and frame = 50

22. BasketballDrill : TZ search(Fast Search algorithm) 4:2:0 video format PSNR(Y, U, V) = [6 * PSNR(Y) + PSNR (U) + PSNR (V)] / 8 QP 24 32 41 49
PSNR (dB)
Bitrate (Kbps)
Time (Sec)
sec
sec
sec
sec.

23. BasketballDrill: Full search (Fast Search algorithm) 4:2:0 video format PSNR(Y, U, V) = [6 * PSNR(Y) + PSNR (U) + PSNR (V)] / 8 QP 24 32 41 49
PSNR (dB)
40.957
31.893
27.936
Bitrate (Kbps)
Time (Sec)
sec.
sec.
sec
sec

24. Project Results
The simulation conducted on HM Software 13.0 using BasketballDrill and RaceHorses test sequences [24] and with a single core system.
The PSNR (dB), encoding time in sec and bitrate (kbps) are calculated by varying quantization parameter. The results are calculated for both TZ search and Full search algorithms.
The PSNR calculated using PSNR(y, u, and v) = [6 * PSNR(y) + PSNR (u) + PSNR (v)] / 8, for 4:2:0 sample video format.
The “PSNR – Bitrate” and “PSNR – Quantization Parameter” plotted for the TZ search & Full search.

25. BasketballDrill PSNR – Bitrate Plot (TZ Search and Full Search)

26. BasketballDrill QP – PSNR Plot (TZ Search and Full Search)

27. Future Work
Further simulation will be conducted on HM Software 13.0 using multi core processors for the TZ search and Full search algorithms.
The PSNR (dB), encoding time in sec and bitrate (kbps) will be calculated by varying quantization parameter.
The PSNR – Bitrate Plot and PSNR – QP Plot will be compared with the single core processor for BasketballDrill and RaceHorses test sequences

28. Acronyms AVC: Advanced Video Coding
BBME: Block Based Motion Estimation
BDM: Block Difference Measure
BMA: Block matching algorithm
CABAC: Context Adaptive Binary Arithmetic Coding
CB: Coding Block
CSVT: Circuits and Systems for Video Technology
CTB: Coding Tree Block
CTU: Coding Tree Unit
CPU: Central Processing Unit
CU: Coding Unit
CUDA: Compute Unified Device Architecture
DCT: Discrete Cosine Transform

29. Acronyms GPU: Graphic Processor Unit
HEVC: High Efficiency Video Coding
ISO: International Organization for Standardization
ITU-T: International Telecommunication Union – Telecommunication Standardization Sector
JCT-VC: Joint Collaborative Team on Video Coding
MC: Motion Compensation
MCP: Motion Compensated Predication
ME: Motion Estimation
MPEG: Moving Picture Experts Group
OPENMP: Open Multiprocessing
PB: Prediction Block

30. Acronyms PCM: Pulse Code Modulation PSNR: Peak signal-to-noise ratio
PSNR: Peak signal-to-noise ratio
PU: Prediction Unit
SAD: Sum of Absolute Differences
SOA: Sample Adaptive Offset
SIMD: Single Instruction Multiple Data
TB: Transform Block
TU: Transform Unit
VCEG: Video Coding Experts Group
VBSME: Variable Block Size Motion Estimation
2DLOG: Two dimensional logarithmic search procedure
TSS: Three step search
NTSS: New three step search

31. References Reference thesis
1 Thesis by S.Gangavati on “Complexity reduction of H.264 using parallel programming” which describes significant speed-up in encoding time on GPU using CUDA and CPU combined than on CPU by data and task parallelization, 2012.
2 Thesis proposed by Pratik Meheta on “Complexity reduction for intra mode selection in HEVC using OpenMP”
References
[1] G. J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp , Dec
[2] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC). J. Sel. IEEE Journal of topics in Signal Processing” vol. 7, pp , Dec. 2013
[3] F. Bossen et al, “HEVC complexity and implementation analysis”, IEEE Trans. CSVT, vol. 22, pp , Dec
[4] H. Zhang and Z. Ma, “Fast intra prediction for high efficiency video coding”, Pacific Rim Conf. on Multimedia, PCM2012, Singapore, Dec
[5] C.C. Chi et al, “Parallel scalability and efficiency of HEVC parallelization approaches”, IEEE Trans. CSVT, vol. 22, pp , Dec

32. References
[6] Introduction to parallel computing URL -
[7] T. Wiegand and G.J. Sullivan, "Overview of the H.264/AVC video coding standard," IEEE Trans. on Circ. Sys. for Video Tech., vol. 13, pp , no. 7, July 2003. 
[8] Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, "Comments on Motion Estimation Algorithms in Current JM Software (JVT-Q089)", Joint Video Team Document, 17th Meeting: Nice, FR, Oct., 2005. 
[9] N.Purnachand, L.N. Alves, A. Navarro “Improvements to TZ search motion estimation algorithm for multiview video coding” 19th International Conference on Systems, Signals and Image Processing IWSSIP, pp , 2012.
[10] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar available on ​
[11] J. Ascenso, C. Brites and F. Pereira, "Improving Frame Interpolation with Spatial Motion Smoothing for Pixel Domain Distributed Video Coding", in Proc. EURASIP Conference on Speech and Image Processing, Multimedia Communication and Services, Slovak Republic, June-July 2005. 
[12] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE International Conference on Consumer Electronics, pp , Jan  

33. References
[13] G. Bjontegaard, "Calculation of average PSNR difference between RD curves", VCEG-M33, 2001.
[14] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012.
[15] P. Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Applications of digital image processing XXXV , vol. 8499, paper , Aug
[16] M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG - 4 AVC standards for low delay applications”, SPIE Applications of digital image processing XXXV , vol. 8499, paper , Aug
[17] L.N.A. Alves and A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc IEEE International Conf. on Consumer Electronics -ICCE Berlin , Germany , vol.11 , pp , Sep. , 2012
[18] X. Wang et al, Paralleling Variable Block Size Motion Estimation of HEVC on Multicore CPU plus GPU platform, IEEE International Conference on Image Processing (ICIP 2013), Melbourne, Australia, Sep.15-18, 2013.
[19] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards” – Springer 2014.
[20] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey,” Journal of Opto-Electronics Review, vol. 21, pp , Mar

34. References
[21] A. Abdelazim,W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding (HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, , Feb. 17, 2014.
[22] M. Shafique et al,"An adaptive workload management scheme for HEVC encoding",Image Processing (ICIP),IEEE International Conference PP: , Sept. 2013
[23] Software repository for HEVC -
[24] Video test sequences – or
[25] HM Software - or
[26] Y.S. Ho and K.J. Oh, “Overview of Multi-view VideoCoding,” IWSSIP and EC - SIPMCS -Proc. 14th Int.Workshop on Systems, Signals and Image Processing and 6th EURASIP Conf. Focused on Speech and ImageProcessing, Multimedia Communications and Services, pp.5-12, 2007.
[27] X.L. Tang, S.K. Dai, C.H. Cai, “An Analysis of TZSearch Algorithm in JMVC,” 1st InternationalConference on Green Circuits and Systems ICGCS, pp , 2010.

35. References
[28] JVT of ISO/IEC MPEG, ITU -T VCEG, MVC software Reference Manual- JMVC 8.5, Mar
[29] HM Software Manual -
[30] Special issue on emerging research and standards in next generation video coding, IEEE Transactions on CSVT, vol. 22, pp , Dec 2012
[31] Special issue on emerging research and standards in next generation video coding. IEEE Transactions on CSVT, vol. 23, pp , Dec 2013
[32] IEEE Journal of selected Topics in Signal Processing, vol. 7, pp , Dec 2013

36. THANK YOU