True Motion Estimation Techniques Part II YY

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm A Novel TME Algorithm Iterative TME Comparison and summary

A Novel TME Algorithm Modified matching function Ik the frame at time k, λ the smooth factor, Nn the causal neighbors of block n Spatial and temporal neighbors “A Novel True-Motion Estimation Algorithm and Its Application to Motion-Compensated Temporal Frame Interpolation” , Salih Dikbas, Yucel Altunbasak, IEEE, Transactions on image image processing

A Novel TME Algorithm Spatiotemporal neighbors are clustered and outputs are served as predictors for current block. A global predictor with penalty term is also used. k-pixel away 8-neighbors of n 8X8 ME stage predictor with lowest cost is selected for 8x8 ME. Costs of are calculated and the lowest cost one is selected.

A Novel TME Algorithm Each 8x8 block is then divided extensive check with 1 pixel away if necessary are checked extensive check with 3 pixel away if necessary > coarse MV of the 8x8 block Each 8x8 block is then divided into 4x4 block, and the coarse MV is assigned as initial prediction of 4x4 block.

A Novel TME Algorithm 4x4 ME stage minimum cost of 4 diagonal neighbors is selected as final predictor ME is performed on neighbors > MV with sub-sample accuracy is obtained.

A Novel TME Algorithm

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm A Novel TME Algorithm Iterative TME Comparison and summary

Iterative TME Define MV reliability by linear checking between UFME & UBME. Iteratively construct reliable MVF. NxN block size NxN ME for unreliable block only 3Nx3N overlapped block size MV for the center Inter出來的blk是可靠的if 通過interp. Blk的UFMV和UBMV是線性(precisely,差異不大) Large blk->noisy or homo- area, small blk-> object boundary or small obj “Iterative True Motion Estimation for Motion-Compensated Frame Interpolation”, DongYoon Kim, Hyungjun Lim, and HyunWook Park, IEEE transactions on circuits and systems for video technology

Iterative TME Unidirectional ME with cost function SAD Iterative ME scheme UFME in RSA_f UBME in RSA_b Correct irregular MVs with median filter

Iterative TME Median filter the center point of block(m,n) in previous frame S the search range

Iterative TME Linearity checking reliable & if < TH_MVD RSA is update as the set of unreliable blocks Assumption block is 1-1 mapped st. unreliable MVs point to a unreliable blocks

Iterative TME

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm A Novel TME Algorithm Iterative TME Comparison and summary

Comparison and Summary Alg property 3DRS+FC OBJ/Block based Novel TME Iterative TME Search method Predictive Hierarchical UME Candidate 5 4 Adaptive cluster + 1 global NO Boundary info. Corner detection Segmentation MLV in BDM Block size Fixed size Multiple sizes Smoothness Candidate + penalty Motion model Part of cost function Median MV filter All of the algorithms introduced can’t guarantee correct MVs in occlusion area, so what they can do is to reduce the error in occlusion area by different interpolation methods .

Appendix Block distortion measure (BDM) ρ the matching criteria, Ikx the image at time kx ,x=1,2 d the displacement of MV Weight function w(n,l) g(n) the MLV, ED(l) the conventional BDM (SAD), Th the reliability threshold

Appendix Maximum local variance (MLV) n the block location, |.| the cardinality, the mean of set Structural similarity image measure (SSIM)

Appendix Adaptive clustering algorithm K=6 D=6