1. ENEE 408G Multimedia Signal Processing Video Stabilization for Pocket PC Application Professor: Dr. Liu Group 4 Student: Hamed hamed@glue.umd.edu Hsiu-huei Yen tamiyen@glue.umd.edu Yuan-heng Lo ylo@glue.umd.edu 2002, May 10

2. ENEE 408G Multimedia Signal Processing Outline  Motivation & Application  Specification  Video Stabilization Algorithm  Performance & Demo  Conclusion & Future Work

3. ENEE 408G Multimedia Signal Processing Motivation & Application --In the near future, we can walk on the street and conference people with pocket pc wirelessly. --Video conferencing with the pocket PC will be one of the future most attractive aspects of this device. --Use global stabilization skills to remove hand shacking effect. --Achieve the high performance video---stabilized, but low bit rate, for the pocket PC.

4. ENEE 408G Multimedia Signal Processing Specification HardwareCompaq iPAQ Pocket PC H3760 Series Creative PC camera with resolution 640x480 pixels SoftwareEmbedded Visual C++ Test patternTranslational camera motion with resolution 160x160 pixels in RGB format. Captured frames saved as BMP files. Frame rate : 10 frames/sec AlgorithmBased on Global Stabilization Algorithm: Motion Estimation, Motion Smoother and Motion Correction

5. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (i) Block Diagram of Video Stabilization Algorithm Input video output stabilized sequence video sequence Motion Estimation Motion Smoother Motion Correction

6. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (ii) Motion Estimation (ME) Obtain optical flow information, motion vectors, for successive frames. Adopt Block –matching method with local 3-step search strategy. Take the median motion vector as movement.

7. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (iii) Motion Estimation (ME) Fig. 1 (a) quiver plot for motion vectors (b) histograms of motion vectors

8. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (iv) Motion Smoother (MS) Remove the unwanted (rapid) camera motion. Convolve the original motion vectors with the Moving Average filter. Moving Average filter: where, for n=0,1,..N-1, otherwise x[n] is the original motion vector N is the length of MA filter can be set arbitrarily, we choose N=10 here.

9. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (v) Motion Smoother (MS) Fig2. Original motion vector and smoothed motion vector (a) horizontal direction (b) vertical direction

10. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (vi) Motion Correction (MC) Perform motion compensation by using the smoothed motion vector. Block Diagram: Original frames Motion vectors Smoothed frames Video Stabilization Algorithm (vi) O1O2O3O4 S2S3S4 MV3MV1MV2 MA Filter (Smoother) SMV1SMV2SMV3

11. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (vii) Motion Correction (MC) Error propagation control: 1. Use threshold / for each frame i / if stabilized frame(i+1) =MC(original frame i, smoothed MV i) else stabilized frame(i+1) =MC(smoothed frame i, smoothed MV i) The trade off about setting threshold value should be considered.

12. ENEE 408G Multimedia Signal Processing Video Stabilization Algorithm (viii) Motion Correction (MC) 2. Use synchronization We can also prevent the error propagation by forcing frames synchronize every certain period. Original frames Motion vectors Smoothed frames O1O2O3O4 S2S3S4 MV3MV1MV2 MA Filter (Smoother) SMV1SMV2SMV3 O5 MV4 SMV4 S5

13. ENEE 408G Multimedia Signal Processing Performance & Demo Test Pattern Frame Rate 10 frames/sec Processing Time Frames reading time : 180 sec Frames processing time : 120 sec Total processing time : 300 sec for 30 frames 10 sec/ frame

14. ENEE 408G Multimedia Signal Processing Performance & Demo (ii) Pocket PC Device Demo

15. ENEE 408G Multimedia Signal Processing Conclusion & Future Work We have verified our algorithm which can remove hand shacking effect with good performance for translational video motions. We have implemented the algorithm with embedded visual C++ on pocket pc device successfully. In the future, we want to achieve the real time processing. In the future, we want to make algorithm used for stabilizing videos in various conditions, and integrated for wireless purpose applications.