1. Hierarchical Systolic Array Design for Full-Search Block Matching Motion Estimation Noam Gur Arie,August 2005

2. Noam Gur ArieMotion Estimation2 Faculty of Electronic Engineering, University of Nis Professor IC: –Ivan Milentijevic PhD Instructors: –Vladimir Ciric –Oliver Vojinovic

3. Noam Gur ArieMotion Estimation3 Contents Introduction to Video Compression The Problem and objective of the project Development of motion estimation Systolic arrays approach to the problem conclusion

4. Noam Gur ArieMotion Estimation4 Project Roadmap Finish line Theory MATLAB Paper Simulations VHDL

5. Noam Gur ArieMotion Estimation5 Video Compression One of the key elements of many video compression schemes is motion estimation. A video sequence consists of a series of frames. To achieve compression, the temporal redundancy between adjacent frames can be exploited. That is, a frame is selected as a reference, and subsequent frames are predicted from the reference using a technique known as motion estimation.

6. Noam Gur ArieMotion Estimation6 Video Compression (cont’) Motion estimation is used in many applications such as Video-on-Demand (VOD), High definition TV (HDTV) and communication applications.

7. Noam Gur ArieMotion Estimation7 Video Compression (cont’) Block Matching (BM) is the most common method of motion estimation. Typically each macroblock ( 16×16 pels) in the new frame is compared with shifted regions of the same size from the previous decoded frame, and the shift which results in the minimum error is selected as the best motion vector for that macroblock. The motion compensated prediction frame is then formed from all the shifted regions from the previous decoded frame.

8. Noam Gur ArieMotion Estimation8 Original Frame Reference frame is 1 frame away Reference frame is 5 frames awayReference frame is 10 frames away

9. Noam Gur ArieMotion Estimation9 The Problem BM can be very computationally demanding if all shifts of each macroblock are analyzed. For example, to analyze shifts of up to ±15 pels in the horizontal and vertical directions requires 31×31=961 shifts, each of which involves 16×16=256 pel difference computations for a given macroblock. This is known as exhaustive search BM.

10. Noam Gur ArieMotion Estimation10 Objective In This project we were set to analyze and implement a Hierarchical Approach, thus allowing system performance manipulation by pipelining and broadcasting some elements of the process More over, the design strategy allows modifying time and hardware complexities at each level of the hierarchy in order to match hardware and design specifications

11. Noam Gur ArieMotion Estimation11 Objective (cont’) The proposed hierarchical design methodology for deriving hierarchical systolic array architectures for FBMA possess the fallowing advantages: –Adaptable sampling rate –Adaptable processor complexity –Low memory bandwidth –Low I/O pin count –High throughput

12. Noam Gur ArieMotion Estimation12 Development Mean Absolute difference - MAD

13. Noam Gur ArieMotion Estimation13 Development (cont’) Possible movement of one macro block (2x2)

14. Noam Gur ArieMotion Estimation14 Development (cont’) We are trying to get from pixel map to MAD map 6 6

15. Noam Gur ArieMotion Estimation15 Development (cont’) MAD Value of one pixel into 1D block Part of frame 1D Block MAD Value

16. Noam Gur ArieMotion Estimation16 Development (cont’) MAD Value of 1D block into 2D block 1D Block MAD Value 2D Block MAD Value

17. Noam Gur ArieMotion Estimation17 Development (cont’) 1D block movement Current blockSearch area -2 0 +1+2 MAD Values Relative movement

18. Noam Gur ArieMotion Estimation18 Development (cont’) Mapping MAD Values Row 1 Row 2 Row 3 Row 4 Row 5 Row 6

19. Noam Gur ArieMotion Estimation19 Development (cont’) Mapping MAD Values of 2D Blocks Row 1 Row 2 Row 3 Row 4 Row 5 Row 6

20. Noam Gur ArieMotion Estimation20 Development (cont’) Row 1 Row 2 Row 3 Row 4 Row 5 Row 6 Mapping MAD Values of 2D Blocks +++++ Block 1 Block 2 Block 3 Block 4 Block 5

21. Noam Gur ArieMotion Estimation21 Development (cont’) Mapping Minimum MAD Values of 2D Blocks Block 1 Block 2 Block 3 Block 4 Block 5

22. Noam Gur ArieMotion Estimation22 Development (cont’) Mapping MAD Values, finding minimum MAD Value (-1,-1)

23. Noam Gur ArieMotion Estimation23 Development (cont’) The minimum values calculated from the MAD table are translated in to a vector using a coding of the inputs, thus allowing us to determined the values of the vector directly. 1 0 0 2 6 7 8 22 7 2 Code - 100

24. Noam Gur ArieMotion Estimation24 Pixel projections Sub sampling Predicative search ML Log search FBMA Systolic Array Design

25. Noam Gur ArieMotion Estimation25 Systolic Array Design (cont’) Using pipelining in order to calculate the MAD values more efficiently PE -2 D -2 r s PE -1 D -1 PE 0 D0D0 PE 1 D1D1 PE 2 D2D2

26. Noam Gur ArieMotion Estimation26 Systolic Array Design (cont’) T(-2)T(-1)T(0)T(1)T(2)T(3) PE(-2) r0r0 r1r1 s 0 - r 2 s 1 - r 3 s 2 - r 4 s 3 - r 5 PE(-1) r0r0 s 0 - r 1 s 1 - r 2 s 2 - r 3 s 3 - r 4 PE(0) s 0 - r 0 s 1 - r 1 s 2 - r 2 s 3 - r 3 PE(1) s 1 - r 0 s 2 - r 1 s 3 - r 2 PE(2) s 2 - r 0 s 3 - r 1 Systolic MAD Calculation, allowing us to have results in parallel

27. Noam Gur ArieMotion Estimation27 Systolic Array Design (cont’) After each 2 time steps we receive all results of one row 2 time steps after we receive all results of one row of other reference block Row, block 2 Row, block 1

28. Noam Gur ArieMotion Estimation28 Systolic Array Design (cont’) Now we add the results together in order to create a MAD table simultaneously

29. Noam Gur ArieMotion Estimation29 Conclusions The main importance of the work was to implement and develop an efficient algorithm based on systolic arrays. Most of the work done was directed in general implantation of the algorithm designed. Additional time should be invested in order to fix edge effects and simulation the results.

30. Noam Gur ArieMotion Estimation30 Conclusions (cont’) 1. Simulating and planning data flow 2. Further research of edge effects 3. Generalizing for different block sizes 4. Synthesis with FPGA device

31. Noam Gur ArieMotion Estimation31 Optimization and algorithm Development Optimization and algorithm Development Simulation and synthesis VHDL implementation VHDL implementation Phase 1 Phase 2 Phase 3

32. Thank you