1. Frame Decimation for Structure and Motion Young Ki Baik CV Lab.

2. Contents Motivation Motivation Frame Selection method Frame Selection method Conclusion and Future works Conclusion and Future works

3. Motivation Profit from video sequence Profit from video sequence Correspondence Correspondence To save large number of inlier correspondence from small motion To save large number of inlier correspondence from small motion X O

4. Motivation Problem #1 Problem #1 Different amount of motion Different amount of motion Fairly small motion to allow automatic matching Fairly small motion to allow automatic matching Significant parallax and large baseline to assure of a well- condition for 3d reconstruction Significant parallax and large baseline to assure of a well- condition for 3d reconstruction Uncertainty boundary Small base line Large base line

5. Motivation Problem #2 Problem #2 Unsharp frames by Bad focus Unsharp frames by Bad focus Giving us ill-condition of corner detection. Giving us ill-condition of corner detection. Eliminating unsharp frame to save corner info. Eliminating unsharp frame to save corner info.

6. Motivation Solution → Frame Selection Solution → Frame Selection Deleting redundant and unsharp frames Deleting redundant and unsharp frames Isotropic motion Isotropic motion

7. Motivation Frame Selection Frame Selection Frame decimation for structure and motion Frame decimation for structure and motion David Nister, SMILE2000 David Nister, SMILE2000 An assessment of information criteria for motion model selection An assessment of information criteria for motion model selection Torr, CVPR97 Torr, CVPR97 Key frame Selection for Camera Motion and Structure Estimation from Multiple Views Key frame Selection for Camera Motion and Structure Estimation from Multiple Views Thormahlen, ECCV2004 Thormahlen, ECCV2004

8. Frame Selection Sharpness measure Sharpness measure For removing unsharp frames For removing unsharp frames Shot boundary detection Shot boundary detection For selecting subset of frames For selecting subset of frames Selection of a subsequence of frames Selection of a subsequence of frames For eliminating redundant frames For eliminating redundant frames

9. Frame Selection Sharpness Measure Sharpness Measure for saving corner information for saving corner information The measure of image sharpness The measure of image sharpness The mean square of the horizontal and vertical derivatives. The mean square of the horizontal and vertical derivatives.

10. Frame Selection Sharpness Measure Sharpness Measure Only to consider the relative sharpness of similar images Only to consider the relative sharpness of similar images Sudden Changes

11. Frame Selection Shot Boundary Detection Shot Boundary Detection Shot boundary Shot boundary These occur when the camera has been turned off and then turned on again at a new place. These occur when the camera has been turned off and then turned on again at a new place. Shot boundary

12. Frame Selection Shot Boundary Detection Shot Boundary Detection Shot boundaries are detected by evaluating the correlation between adjacent frames after global motion compensation. Shot boundaries are detected by evaluating the correlation between adjacent frames after global motion compensation. Mean of Normalized correlation value (T sb = 0.75) Mean of Normalized correlation value (T sb = 0.75) Shot boundary detection 0.90.50.9

13. Frame Selection Selection of Subsequence Selection of Subsequence For deleting redundant frames For deleting redundant frames Considering two properties Considering two properties Normalized Correlation (NC) constraint Normalized Correlation (NC) constraint Distant constraint Distant constraint

14. Frame Selection Normalized Correlation (NC) constraint Normalized Correlation (NC) constraint Check the mean of NC value between near frames. Check the mean of NC value between near frames. Delete F i when the mean of NC value between F i-1 and F i+1 is bigger than T(=0.95). Delete F i when the mean of NC value between F i-1 and F i+1 is bigger than T(=0.95).

15. Frame Selection Distance constraint Distance constraint Check maximum distance in correspondences. Check maximum distance in correspondences. Delete F i when maximum distance is smaller than Delete F i when maximum distance is smaller than T d (= image size/10). T d (= image size/10). Maximum distance of correspondence

16. Frame Selection Selection of Subsequence Selection of Subsequence Deleting process is repeated until no additional deletions occur. Deleting process is repeated until no additional deletions occur.

17. Conclusion and Future works Conclusion Conclusion Method of the frame selection in video sequence Method of the frame selection in video sequence Future works Future works Usage of video sequence Usage of video sequence Implementation Implementation