1. Vision-based Registration for AR Presented by Diem Vu Nov 20, 2003

2. Markerless Tracking using Planar Structure in the Scene. G. Simon, A.W. Fitzgibbon and A. Zisserman, 2000. Calibration-Free Augmented Reality. K.N Kutulakos and J.R. Vallino, 1998.

3. Planar-surface tracking. Camera position can be recovered from planar homography. Planar structure is common in almost all scenarios.

4. y x z HwHw World to image homography Image to image homography

5. World to image homography Consider our tracking plane is the plane Z=0 y x z HwHw

6. Projection matrix

7. y x z P

8. y x z P

9. If K and H w are known, then r 1, r 2 and t can be recovered, thus P. Question: How to compute H w ? Direct. Indirect.

10. Direct measurement of H w Select 4 points {x k } on a rectangle in the scene. Compute H which maps the unit square to {x k }. (0,0) (0,1)(1,1) () (1,0)

11. Direct measurement of H w Select 4 points {x k } on a rectangle in the scene. Compute H which maps the unit square to {x k }. Compute H w =Hdiag(1, 1 / s,1) (0,0) (0,s)(1,s) () (1,0)

12. Indirect measurement of H w y x z

13. y x z

14. Algorithm summary Compute (direct measure). For each frame i, compute frame to frame homography (RANSAC) Compute by:

15. Other … Using only 2 points in direct method ?? Matching the frame i with frame 0 in order to reduce error. Estimate intrinsic parameters K Hand-off mechanism.

20. Possible problems? Homography is only up-to-scale? Plain surface (no texture) or moving objects in the foreground ? Depth order, occlusion ? Speed ?

21. Affine virtual object representation Represent virtual objects so that their projection can be computed as a linear combination of the projection of the fiducial points.

23. Project a point from its affine coordinates

24. Compute affine coordinates from projection along two viewing direction

25. Algorithm Setup the affine basis

26. Algorithm Setup the affine basis Locate the object in 2 frames.

27. Algorithm Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates for each point.

28. Algorithm Setup the affine basis Locate the object in 2 frames. Compute the affine coordinates for each point. Compute projection of the object and render the object in each frame.

30. Camera viewing direction  and  are the first and second row of  2x3. The camera viewing direction expressed in the coordinate frame of the affine basis points:  =   

31. Depth order w is the z-value of point p (x,y,z).

33. Advantages No need any metric information. Able to use with the existing hardware to accelerate graphics operations. Can be used to improve tracking.

34. Limitation Affine constraints. Lost of metric information.