True Motion Estimation Techniques Part I YY

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

True motion estimation Finding true motion of objects in a frame TMVF usually differs from MVF that has lowest residual. Minimum match error is no guarantee for TMF. 因為目前DVC架構再找HF prediction, 用TME可能較好 Full Search with SAD 3DRS

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

3-D recursive search Construct a small set of candidate vector based on spatiotemporal predictions. Assumption Objects are larger than block size -> neighbor MVs are reliable Objects have inertia. Three kind of candidate Spatial candidate Temporal candidate Update candidate adding small random vector (u) to spatial candidate FS可能找不到TMV的原因是太多candidate

3DRS ρi is the relative position to current block 4 candidate per block 2 spatial candidate, 1 temporal candidate, 1 update candidate Update cand. 對converge來說畢要,and to correctly track variable obj motion. 靠Update cand. 來找新MV, op需要converge到TMF 連續的frame有很高的MF關係,所以前一個的MVF只需小modify, Implicitly take care of this by temporal candidate.

3DRS Match error with penalty pi is minimized. βi the block width/height, d the MV, x the block position, n the frame number, CS the candidate set Prefernce to spatial candidate , least prefer to update candiddate.

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

3DRS with Feature Correspondences Proposed to deal with drawbacks of 3DRS discontinuous changes or MVs convergence situation distance fast convergence requires long update length but results in more noise in MVF Discont. Change : 新出現\被擋住的出現, 需要數個blk,itrs,frames去converge 因為update length小

3DRS with Feature Correspondences Take the found correspondences as initial estimates, and use 3DRS to construct MVF. reliable 2-D MV -> corner feature point (fp) correlation between 2 fp x the fp, Bfp the block to which fp belongs, μ(x), σ(x) the fp block mean & stddev Blk size = 8x8

3DRS with Feature Correspondences Maximum correlation of a given fp S the search range Additional candidate from fp Cth the correlation threshold S=+-120 for x , +-20 for y Cth=0.85

3DRS+FC vs. 3DRS

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

Block / Object based Algorithm BMA problem small matching window -> random MVs large matching window -> inaccurate MV near object boundary Proposed hybrid algorithm for smooth, dense TMF 1.Initial ME : improved hierarchical BMA 2.Image segmentation : object based 3.Wrong MV detection & correction

Block / Object based Algorithm Improved hierarchical BMA Adaptive technique to propagate MV between levels MSE Duplication\bilinear 是low level傳MV給high level 的方法 Hierarchial BMA的問題: 在obj邊緣的MV不准, 因為 1.low level 相對大的blk size可能重疊多個物體 2.high level小的SR不夠大無法refine區別不同物體如果在低接用小blk或高階用大SR->MVF 不smooth 切4 child原因:為了處理可能不同obj

Block / Object based Algorithm Correct the incorrect MV in area of motion occlusion by image segmentation. multiscale gradient algorithm Segment the image into objects Correct MVs in occlusion area with objects. BMA 無法只用兩張frame處理occlusion problem

Block / Object based Algorithm Assumption of wrong MV detection & correction occlusion area is small part of an object TMF is continuous & smooth within an object detection r the spatial coordinate, u(.) the estimated MVF, (.) the smooth motion field model, T the threshold

Block / Object based Algorithm Object-adaptive motion field correction

outline True motion estimation algorithms Comparison and summary 3-D Recursive Search Block Matching 3DRS with Feature Correspondences Block/Object-Based Algorithm Novel TME Algorithm Iterative TME Comparison and summary

Appendix Local inconsistency △(x,n) Spatial inconsistency (SI) W the measurement window (exclude boarder blocks)

Appendix M2SE 3DRS TMV MC