1. True Motion Estimation Techniques Part IYY

2. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

3. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

4. True motion estimation
Finding true motion of objects in a frame
TMVF usually differs from MVF that has lowest residual.
Minimum match error is no guarantee for TMF.
因為目前DVC架構再找HF prediction, 用TME可能較好
Full Search with SAD
3DRS

5. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

6. 3-D recursive search
Construct a small set of candidate vector based on spatiotemporal predictions.
Assumption
Objects are larger than block size -> neighbor MVs are reliable
Objects have inertia.
Three kind of candidate
Spatial candidate
Temporal candidate
Update candidate
adding small random vector (u) to spatial candidate
FS可能找不到TMV的原因是太多candidate

7. 3DRS ρi is the relative position to current block
4 candidate per block
2 spatial candidate, 1 temporal candidate, 1 update candidate
Update cand. 對converge來說畢要,and to correctly track variable obj motion.
靠Update cand. 來找新MV, op需要converge到TMF
連續的frame有很高的MF關係,所以前一個的MVF只需小modify,
Implicitly take care of this by temporal candidate.

8. 3DRS Match error with penalty pi is minimized.
βi the block width/height, d the MV,
x the block position, n the frame number,
CS the candidate set
Prefernce to spatial candidate , least prefer to update candiddate.

9. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

10. 3DRS with Feature Correspondences
Proposed to deal with drawbacks of 3DRS
discontinuous changes or MVs
convergence situation distance
fast convergence requires long update length but results in more noise in MVF
Discont. Change : 新出現\被擋住的出現, 需要數個blk,itrs,frames去converge 因為update length小

11. 3DRS with Feature Correspondences
Take the found correspondences as initial estimates, and use 3DRS to construct MVF.
reliable 2-D MV -> corner feature point (fp)
correlation between 2 fp
x the fp, Bfp the block to which fp belongs,
μ(x), σ(x) the fp block mean & stddev
Blk size = 8x8

12. 3DRS with Feature Correspondences
Maximum correlation of a given fp
S the search range
Additional candidate from fp
Cth the correlation threshold
S=+-120 for x , +-20 for y
Cth=0.85

13. 3DRS+FC vs. 3DRS

14. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

15. Block / Object based Algorithm
BMA problem
small matching window -> random MVs
large matching window -> inaccurate MV near object boundary
Proposed hybrid algorithm for smooth, dense TMF
1.Initial ME : improved hierarchical BMA
2.Image segmentation : object based
3.Wrong MV detection & correction

16. Block / Object based Algorithm
Improved hierarchical BMA
Adaptive technique to propagate MV between levels
MSE
Duplication\bilinear 是low level傳MV給high level 的方法
Hierarchial BMA的問題: 在obj邊緣的MV不准, 因為 1.low level 相對大的blk size可能重疊多個物體 2.high level小的SR不夠大無法refine區別不同物體如果在低接用小blk或高階用大SR->MVF 不smooth
切4 child原因:為了處理可能不同obj

17. Block / Object based Algorithm
Correct the incorrect MV in area of motion occlusion by image segmentation.
multiscale gradient algorithm
Segment the image into objects
Correct MVs in occlusion area with objects.
BMA 無法只用兩張frame處理occlusion problem

18. Block / Object based Algorithm
Assumption of wrong MV detection & correction
occlusion area is small part of an object
TMF is continuous & smooth within an object
detection
r the spatial coordinate, u(.) the estimated MVF,
(.) the smooth motion field model, T the threshold

19. Block / Object based Algorithm
Object-adaptive motion field correction

20. outline True motion estimation algorithms Comparison and summary
3-D Recursive Search Block Matching
3DRS with Feature Correspondences
Block/Object-Based Algorithm
Novel TME Algorithm
Iterative TME
Comparison and summary

21. Appendix Local inconsistency △(x,n) Spatial inconsistency (SI)
W the measurement window (exclude boarder blocks)

22. Appendix
M2SE
3DRS TMV MC