Yu Liu and King Ngi Ngan Department of Electronic Engineering

Slides:



Advertisements
Similar presentations
Arc-length computation and arc-length parameterization
Advertisements

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
Fast Algorithms For Hierarchical Range Histogram Constructions
FTP Biostatistics II Model parameter estimations: Confronting models with measurements.
Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.
{ Fast Disparity Estimation Using Spatio- temporal Correlation of Disparity Field for Multiview Video Coding Wei Zhu, Xiang Tian, Fan Zhou and Yaowu Chen.
An Improved 3DRS Algorithm for Video De-interlacing Songnan Li, Jianguo Du, Debin Zhao, Qian Huang, Wen Gao in IEEE Proc. Picture Coding Symposium (PCS),
Young Deok Chun, Nam Chul Kim, Member, IEEE, and Ick Hoon Jang, Member, IEEE IEEE TRANSACTIONS ON MULTIMEDIA,OCTOBER 2008.
Yen-Lin Lee and Truong Nguyen ECE Dept., UCSD, La Jolla, CA Method and Architecture Design for Motion Compensated Frame Interpolation in High-Definition.
Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,
Novel Point-Oriented Inner Searches for Fast Block Motion Lai-Man Po, Chi-Wang Ting, Ka-Man Wong, and Ka-Ho Ng IEEE TRANSACTIONS ON MULTIMEDIA, VOL.9,
Outline Introduction Introduction Fast Inter Prediction Mode Decision for H.264 – –Pre-encoding An Efficient Inter Mode Decision Approach for H.264 Video.
Real-time Embedded Face Recognition for Smart Home Fei Zuo, Student Member, IEEE, Peter H. N. de With, Senior Member, IEEE.
Mobile Motion Tracking using Onboard Camera Lam Man Kit CEG Wong Yuk Man CEG.
New Sorting-Based Lossless Motion Estimation Algorithms and a Partial Distortion Elimination Performance Analysis Bartolomeo Montrucchio and Davide Quaglia.
Robust Real-time Object Detection by Paul Viola and Michael Jones ICCV 2001 Workshop on Statistical and Computation Theories of Vision Presentation by.
Segmentation Divide the image into segments. Each segment:
Scalable Wavelet Video Coding Using Aliasing- Reduced Hierarchical Motion Compensation Xuguang Yang, Member, IEEE, and Kannan Ramchandran, Member, IEEE.
FAST MULTI-BLOCK SELECTION FOR H.264 VIDEO CODING Chang, A.; Wong, P.H.W.; Yeung, Y.M.; Au, O.C.; Circuits and Systems, ISCAS '04. Proceedings of.
Multi-Frame Reference in H.264/AVC 卓傳育. Outline Introduction to Multi-Frame Reference in H.264/AVC Multi-Frame Reference Problem Two papers propose to.
1 An Efficient Mode Decision Algorithm for H.264/AVC Encoding Optimization IEEE TRANSACTION ON MULTIMEDIA Hanli Wang, Student Member, IEEE, Sam Kwong,
Motion Estimation Using Low- Band-Shift Method for Wavelet- Based Moving Picture Hyun-Wook Park, Senior Member, IEEE, and Hyung-Sun Kim IEEE Transactions.
A Low-Power VLSI Architecture for Full-Search Block-Matching Motion Estimation Viet L. Do and Kenneth Y. Yun IEEE Transactions on Circuits and Systems.
AN IMPROVED MULTILEVEL SUCCESSIVE ELIMINATION ALGORITHM FOR FAST FULL- SREACH MOTION ESTIMATION Tae Gyoung Ahn, Youg Ho Moon*, and Jae Ho Kim Department.
Wavelet-Based Multiresolution Matching for Content-Based Image Retrieval Presented by Tienwei Tsai Department of Computer Science and Engineering Tatung.
Frame by Frame Bit Allocation for Motion-Compensated Video Michael Ringenburg May 9, 2003.
1 Efficient Reference Frame Selector for H.264 Tien-Ying Kuo, Hsin-Ju Lu IEEE CSVT 2008.
Presented by Tienwei Tsai July, 2005
Wavelet-based Coding And its application in JPEG2000 Monia Ghobadi CSC561 final project
The Application of The Improved Hybrid Ant Colony Algorithm in Vehicle Routing Optimization Problem International Conference on Future Computer and Communication,
Chin-Yu Huang Department of Computer Science National Tsing Hua University Hsinchu, Taiwan Optimal Allocation of Testing-Resource Considering Cost, Reliability,
MOTION ESTIMATION IMPLEMENTATION IN RECONFIGURABLE PLATFORMS
Guillaume Laroche, Joel Jung, Beatrice Pesquet-Popescu CSVT
Advances in digital image compression techniques Guojun Lu, Computer Communications, Vol. 16, No. 4, Apr, 1993, pp
2005/12/021 Content-Based Image Retrieval Using Grey Relational Analysis Dept. of Computer Engineering Tatung University Presenter: Tienwei Tsai ( 蔡殿偉.
An efficient Video Coding using Phase-matched Error from Phase Correlation Information Manoranjan Paul 1 and Golam Sorwar IEEE.
IEEE Transactions on Consumer Electronics, Vol. 58, No. 2, May 2012 Kyungmin Lim, Seongwan Kim, Jaeho Lee, Daehyun Pak and Sangyoun Lee, Member, IEEE 報告者:劉冠宇.
Segmentation of Vehicles in Traffic Video Tun-Yu Chiang Wilson Lau.
Page 11/28/2016 CSE 40373/60373: Multimedia Systems Quantization  F(u, v) represents a DCT coefficient, Q(u, v) is a “quantization matrix” entry, and.
3-D Direction Aligned Wavelet Transform for Scalable Video Coding Yu Liu 1, King Ngi Ngan 1, and Feng Wu 2 1 Department of Electronic Engineering The Chinese.
1 Yu Liu 1, Feng Wu 2 and King Ngi Ngan 1 1 Department of Electronic Engineering, The Chinese University of Hong Kong 2 Microsoft Research Asia, Beijing,
By Dr. Rajeev Srivastava CSE, IIT(BHU)
BLOCK BASED MOTION ESTIMATION. Road Map Block Based Motion Estimation Algorithms. Procedure Of 3-Step Search Algorithm. 4-Step Search Algorithm. N-Step.
Motion tracking TEAM D, Project 11: Laura Gui - Timisoara Calin Garboni - Timisoara Peter Horvath - Szeged Peter Kovacs - Debrecen.
1שידור ווידיאו ואודיו ברשת האינטרנט Dr. Ofer Hadar Communication Systems Engineering Department Ben-Gurion University of the Negev URL:
Content Based Coding of Face Images
Dense-Region Based Compact Data Cube
Experience Report: System Log Analysis for Anomaly Detection
Michael Gormish Prepared by:Emin Sinan Mimaroglu
Dr. Ofer Hadar Communication Systems Engineering Department
Singular Value Decomposition and its applications
Design and Implementation of Lossless DWT/IDWT (Discrete Wavelet Transform & Inverse Discrete Wavelet Transform) for Medical Images.
Compressive Coded Aperture Video Reconstruction
Adaptive Block Coding Order for Intra Prediction in HEVC
UNIVERSITY OF MASSACHUSETTS Dept
Advanced Wireless Networks
Injong Rhee ICMCS’98 Presented by Wenyu Ren
Wavelets : Introduction and Examples
Principal Component Analysis (PCA)
SoC and FPGA Oriented High-quality Stereo Vision System
Department of Computer Science
A Novel TDS-FDMA Scheme for Multi-user Uplink Scenarios
Anisotropic Double Cross Search Algorithm using Multiresolution-Spatio-Temporal Context for Fast Lossy In-Band Motion Estimation Yu Liu and King Ngi Ngan.
Fully Scalable Multiview Wavelet Video Coding
Reseeding-based Test Set Embedding with Reduced Test Sequences
HALO-FREE DESIGN FOR RETINEX BASED REAL-TIME VIDEO ENHANCEMENT SYSTEM
Wavelet Analysis Objectives: To Review Fourier Transform and Analysis
Scalable light field coding using weighted binary images
Image Registration  Mapping of Evolution
An image adaptive, wavelet-based watermarking of digital images
Presentation transcript:

Fast Lossless Multi-Resolution Motion Estimation for Scalable Wavelet Video Coding Yu Liu and King Ngi Ngan Department of Electronic Engineering The Chinese University of Hong Kong ISCAS2006, May 21-24, Island of Kos, Greece

Outline Introduction Background Proposed Algorithm Experimental Results Conclusion

Introduction Motion Estimation in Critically-Sampled Wavelet Domain Pro: basically free form the blocking effects Con: inefficient in high bands Motion Estimation in Shift-Invariant Wavelet Domain Pro: perform ME more precisely and efficiently Con: computational complexity e.g. low-band-shift (LBS) and complete-to-overcomplete DWT (CODWT) ME/MC in wavelet domain has received much attention due to its superior performance by comparing to conventional ME/MC in spatial domain. ME/MC in wavelet domain is basically free form the blocking effects due to the global nature of wavelet transform. However, ME/MC in critically sampled wavelet domain is very inefficient in high bands because of the shift-variant property of wavelet decomposition. To overcome the shift-variant property of wavelet transform, LBS and CODWT are proposed. These methods avoid the shift-variant property of the wavelet transform and perform ME more precisely and efficiently. However, a major disadvantage of these methods is the computational complexity which mainly comes from full search algorithm.

Background Motion Estimation in Shift-Invariant Wavelet Domain (1) Two Level Shift-Invariant Wavelet Decomposition by using Low-Band-Shift (LBS) or Complete-to-Overcomplete DWT (CODWT) First, let’s look at some background. This is an example of two level SIDWT by using LBS or CODWT. For LBS method, these other shifted subbands are obtained by shifting the LL Band of each level, then followed by normal wavelet transform. For CODWT method, these other shifted subbands are obtained by making a direct link between the critically-sampled subbands and shift-invariant subbands using the complete-to-overcomplete prediction filters.

Background Motion Estimation in Shift-Invariant Wavelet Domain (2) Generation of Wavelet Blocks For ME in shift-invariant wavelet domain, the coefficients of each wavelet tree rooted in the lowest subband are rearranged to form a wavelet block. The purpose of the wavelet block is to provide a direct association between the wavelet coefficients and what they represent spatially in the image. Related coefficients at all scales and orientations are included in each wavelet block. The v-pixel-shifted or {dx,dy}-pixel-shifted coefficient of the pth wavelet block of the reference frame t’ can be represented by (dx%2l, dy%2l) means which shifted subband it is, (i+[dx/2l],j+[dy/2l]) means where the pixel is located for the shifted subband. l denotes decomposition level, k denotes subband type, such as LL/HL/LH/HH subband type. The coefficient of the pth wavelet block of the current frame t can be represented by The difference between these two representations of the reference and current frames is that, for ME/MC, we just want to use the shift-invariant wavelet coefficients of the reference frame to predict the critically-sampled wavelet coefficients of the current frame. The v-pixel-shifted or {dx,dy}-pixel-shifted coefficient of the pth wavelet block of reference frame t’ can be represented by The coefficient of the pth wavelet block of current frame t can be represented by

Background Motion Estimation in Shift-Invariant Wavelet Domain (3) The sum of absolute difference (SAD) of the pth wavelet block for the motion vector v is computed as follows: The wavelet blocks in the search window in the reference frame are compared to current wavelet block, and a reference wavelet block that leads to the best match is selected. The sum of absolute difference (SAD) of the pth wavelet block for the displacement vector v is computed as follows: The optimum motion vector v∗ of the pth wavelet block, which has minimum displacement error, is given by: However, computational complexity of full search ME in SIWD is so time consuming because of the extra shifted subbands that limits its practical applications. For this reason, alternative and faster techniques should be developed. The optimum motion vector v∗ of the pth wavelet block, which has minimum displacement error, is given by:

Background Partial Distortion Elimination Partial Distortion Elimination (PDE) is a fast algorithm which has identical quality as that of FSA. The partial SAD (PSAD) is used to eliminate impossible candidates before the complete calculation of the SAD: The partial distortion elimination (PDE) is a fast algorithm which has identical quality as that of the full search algorithm (FSA). The partial sum of differences (PSAD), which is used to eliminate impossible candidates before the complete calculation of the SAD, is calculated as follows: where m is the number of differences needed to reach the SADmin which has already been found so far, and O (n) represents a generic matching order in the wavelet block.

Proposed Algorithm Wavelet Matching Error Characteristic To improve the computational saving of PDE, if the expected values of the matching error dp+v(i, j) in the search window w fulfills the following criterion: First, let’s look at the proposed fast lossless ME algorithm in SIWD. Since the partial distortion elimination (PDE) is a good way to find a fast lossless ME algorithm, our proposed method is also based on PDE. To improve the computational saving of PDE, if the expected values of the matching error dp+v(i, j) in the search window w fulfills the following criterion: then we assume that this matching order will be generally effective for all of the candidate vectors. In order to obtain the predicted matching error ep(i, j), we consider then we assume that this matching order will be generally effective for all of the candidate vectors in the search window. Therefore, to fulfill the above objective, we use ep(i, j) to predict the matching error.

Proposed Algorithm Wavelet Matching Error Characteristic To obtain the predicted matching error ep(i, j), solve this equation We finally can obtain an approximate solution of Eq. (6): To solve this equation, we have We finally can obtain an approximate solution of Eq. (6): The solution of the predicted matching error consists of two parts: one is the wavelet coefficient magnitude in the current wavelet block, another is the expected value of wavelet coefficient magnitude in the reference wavelet block. Since the expected value of the wavelet coefficient magnitude in a search window varies smoothly, the second part can be considered as a constant for different ep(i,j). The predicted matching error is proportional to the wavelet coefficient magnitude in the current wavelet block. That is to say, larger wavelet magnitude in the current wavelet block tends to produce larger matching error. Larger wavelet coefficient magnitude in the current wavelet block tends to produce larger matching error

Proposed Algorithm MR-WMEC-PDE Three key factors which affect the performance of PDE the Searching Order in which the wavelet blocks are tested during the searching phase. the Matching Order in which the coefficients within a wavelet block are picked up to compute the SAD. the Comparison Interval in which comparison between PSAD and SADmin is performed. Three new strategies for PDE by using wavelet matching error characteristic (WMEC) are proposed. Observed from the PDE equation in previous slide, we can sum up three key factors which affect the performance of PDE. the Searching Order, If a good SADmin is found early, then many more successive tests have a tighter distortion bound and may be skipped. the Matching Order, If the highest contributions to SAD are found early, then the distortion bound may be reached after a small number of differences and the PSAD can be stopped. the comparison interval, If a good tradeoff between the cost of comparisons and the number of useless differences is achieved, then the number of comparisons and differences can be reduced. In order to make use of these three factors, the wavelet matching error characteristic (WMEC) is employed to develop three new strategies for PDE. Searching Order Strategy based on Wavelet Multi-Resolution Property Matching Order Strategy based on Wavelet-tree Grouping Scheme Comparison Interval Strategy based on Adaptive Sub-blocks Checking Unit

Proposed Algorithm MR-WMEC-PDE Searching Order Strategy based on Wavelet Multi-Resolution Property Instead of the spiral search order, the proposed searching order strategy uses the normalized partial SAD in LL subband level as the estimated SAD (ESAD) Because the LL subband actually corresponds to the lowest resolution version of original frame, the motion information in LL subband of a wavelet block is highly correlated with that of the whole wavelet block. This property can be used for a new searching order strategy to get a good match sooner. Therefore, Instead of the spiral search order, the proposed searching order strategy uses the normalized partial SAD in LL subband level as the estimated SAD (ESAD). Then, sort the ESAD using the counting sort algorithm in ascending order to obtain the searching order SO = {vn | n = 0, ...,w−1}. Then, sort the ESAD using the counting sort algorithm in ascending order to obtain the searching order SO = {vn | n = 0, ...,w−1}.

Proposed Algorithm MR-WMEC-PDE Matching Order Strategy based on Wavelet-tree Grouping Scheme A wavelet-tree grouping scheme according to spatial self-similarity property and matching error clustering property of wavelet coefficient Sort Ep(Bl,bl,m) using the quick sort algorithm in descending order to obtain the matching order of level l: MOl = {bl,m | m = 0, ...,M − 1} Since the wavelet coefficient magnitude tends to be larger in the higher decomposition level, the matching error also tends to be larger in the higher decomposition level according to above reasoning. Therefore, the trend of matching order is from the higher decomposition level to the lower decomposition level. In addition, the matching errors with similar magnitude in the same level tend to appear in clusters. Thus, we propose a wavelet tree grouping scheme which groups the coefficients into sub-blocks according to the spatial self-similarity property and the matching error clustering property of wavelet coefficients. To determine the matching order of sub-blocks, the estimated matching error of each sub-block is calculated by Sort Ep(Bl,bl,m) using the quick sort algorithm in descending order to obtain the matching order of level l: MOl The figure shows (a) Wavelet-tree grouping scheme in a wavelet block, (b) an example of adaptive matching order based on wavelet-tree grouping scheme

Proposed Algorithm MR-WMEC-PDE Comparison Interval Strategy based on Adaptive Sub-blocks Checking Unit In conventional PDE methods, fixed comparison interval, such as eight-pixels or sixteen-pixels checking unit, is usually adopted. Combined with the wavelet-tree grouping scheme, an adaptive comparison interval strategy is proposed. In the proposed strategy, every 2l−1 sub-blocks in the decomposition level l are used as the checking unit. In conventional PDE methods, fixed comparison interval, such as eight-pixels or sixteen-pixels checking unit, is usually adopted. Combined with the wavelet-tree grouping scheme, an adaptive comparison interval strategy is proposed. In the proposed scheme, every 2l−1 sub-blocks in the decomposition level l are used as the checking unit, that is, the comparison between PSAD and SADmin is performed every 2l−1 sub-blocks for the decomposition level l. So that, the cost of comparisons remains at a reasonable level.

Experimental Results (1) Simulation results are reported in the following ways: operation number : used to compute the partial distortion speed-up ratio : for motion estimation including the required overheads for comparison. Simulation results are reported in the following ways: operation number per block used to compute the partial distortion; execution time per frame for motion estimation including the required overheads for comparison. For the performance comparison, we tested the following algorithms: FSA, Spiral-PDE, CPME-PDS, MR-WMEC-PDE. For performance comparison with other algorithms Full Search Algorithm (FSA) Spiral-PDE [5] CPME-PDS [6] proposed MR-WMEC-PDE

Experimental Results (2) Average Operation Number per Block for Tested Algorithms This table lists the Experimental Results of Average Operation Number per Block for Tested Algorithms On average speed-up ratio in terms of operation number, MR-WMEC-PDE is better than Spiral-PDE and CPME-PDS by about 92% and 34%, respectively. On average speed-up ratio in terms of operation number, MR-WMEC-PDE is better than Spiral-PDE and CPME-PDS by about 92% and 34%, respectively.

Experimental Results (3) Average Execution Time per Frame for Tested Algorithms This table lists the Experimental Results of Average Execution Time per Frame for Tested Algorithms On average speed-up ratio in terms of execution time, MR-WMEC-PDE is better than Spiral-PDE and CPME-PDS by about 84% and 63%, respectively. The overhead of MR-WMEC-PDE algorithm still can be tolerable since the loss of performance that comes from the overhead is only 10% on average. On average speed-up ratio in terms of execution time, MR-WMEC-PDE is better than Spiral-PDE and CPME-PDS by about 84% and 63%, respectively.

Conclusion Fast Lossless Multi-Resolution Motion Estimation Algorithm Wavelet Matching Error Characteristic (WMEC) Three New Strategies for PDE Searching Order Strategy based on Wavelet Multi-Resolution Property Matching Order Strategy based on Wavelet-tree Grouping Scheme Comparison Interval Strategy based on Adaptive Subblocks Checking Unit In this paper, we proposed a fast lossless multi-resolution motion estimation algorithm, MR-WMEC-PDE. We take advantage of the new discovery about wavelet matching error characteristic to develop the three new strategies for PDE: Searching Order Strategy based on Wavelet Multi-Resolution Property Matching Order Strategy based on Wavelet-tree Grouping Scheme Comparison Interval Strategy based on Adaptive Subblocks Checking Unit