1. iMAGIS-GRAVIR / IMAG Augmented Reality Collaborative Environment: Calibration and Interactive Scene Editing Raphaël Grasset Xavier Decoret Jean-Dominique Gascuel iMAGIS-GRAVIR/IMAG iMAGIS is a joint project between CNRS,INRIA,INPG and UJF

2. iMAGIS-GRAVIR / IMAG Motivations Focus on: –tracking and registration of virtual and real objects. –commercial age: off the shelf and personal hardware. Purpose: propose a collaborative environment for multi-user in augmented reality.

3. iMAGIS-GRAVIR / IMAG Related Works Some Augmented Reality Environments GRASP[1995] Studierstube[1998] EMMIE[1999]

4. iMAGIS-GRAVIR / IMAG Overview Motivation and Related Works Setup: Small Group and Computing Environment Calibration Procedure –HMD Calibration –Pen Calibration –World Calibration Dynamic Objects –Mixing Real and Virtual objects –Moving Objects Conclusion

5. iMAGIS-GRAVIR / IMAG Setup: Small Group Centralized Configuration: –user around a table: face to face collaboration. –personal and shared area. Shared area Personal area (virtual menu, private data)

6. iMAGIS-GRAVIR / IMAG Setup: Computing Environment Visualization: optical see-through HMD direct view, private information, low weight. Interaction: stylus easy use. Tracking: magnetic tracker fast, no line of sight.

7. iMAGIS-GRAVIR / IMAG Overview Motivation and Related Works Setup: Small Group and Computing Environment Calibration Procedure –HMD Calibration –Pen Calibration –World Calibration Dynamic Objects –Mixing Real and Virtual objects –Moving Objects Conclusion

8. iMAGIS-GRAVIR / IMAG Calibration Procedure World CS Image CS Transmitter CS Receptor CS Emitter CS Eye CS HMD CS Tip CS Purpose: obtain the relations between different coordinate systems (CS). HMD CS = middle between the 2 optic centers of the HMD. Style: fast and accurate interactive procedure.

9. iMAGIS-GRAVIR / IMAG Calibration Procedure World CS Image CS Transmitter CS Receptor CS Emitter CS Eye CS HMD CS Tip CS HMD CS = middle between the 2 optic centers of the HMD. Style: fast and accurate interactive procedure. HMD Calibration Purpose: obtain the relations between different coordinate systems (CS).

10. iMAGIS-GRAVIR / IMAG Calibration Procedure World CS Image CS Transmitter CS Receptor CS Emitter CS Eye CS HMD CS Tip CS HMD CS = middle between the 2 optic centers of the HMD. Style: fast and accurate interactive procedure. Pen Calibration Purpose: obtain the relations between different coordinate systems (CS).

11. iMAGIS-GRAVIR / IMAG Calibration Procedure World CS Image CS Transmitter CS Receptor CS Emitter CS Eye CS HMD CS Tip CS HMD CS = middle between the 2 optic centers of the HMD. Style: fast and accurate interactive procedure. World Calibration Purpose: obtain the relations between different coordinate systems (CS).

12. iMAGIS-GRAVIR / IMAG Related Works Some Calibration Methods: Azuma[1994] & Oishi[1996]: lot of hardware. Tuceryan[1995]: video see-through method

13. iMAGIS-GRAVIR / IMAG Related Works Fuhrmann[1999] (Studierstube): calibration with a tracked pen. Genc[2000] (GRASP): vision approach.

14. iMAGIS-GRAVIR / IMAG HMD Calibration Purpose: transformation between the Receptor CS and the Image CS. Style: reduce user effort. Constraints: personnal hardware => removable tracker on the HMD. D epends on the position of the tracker in the HMD (HMD CS Receptor CS), Morphology..etc..

15. iMAGIS-GRAVIR / IMAG HMD Calibration: Procedure Receptor CS HMD CS Eye CS Emitter CS Misalignment Method: freeze the view and align a virtual object with real object. Virtual Emitter 1.Freeze the view

16. iMAGIS-GRAVIR / IMAG HMD Calibration: Procedure Receptor CS HMD CS Eye CS Emitter CS Virtual Emitter Freeze View 2. Move head Method: freeze the view and align a virtual object with real object.

17. iMAGIS-GRAVIR / IMAG HMD Calibration: Procedure Receptor CS HMD CS Eye CS Emitter CS Virtual Emitter Freeze View 3. Unfreeze the view Method: freeze the view and align a virtual object with real object.

18. iMAGIS-GRAVIR / IMAG HMD Calibration: Procedure Receptor CS HMD CS Eye CS Emitter CS Virtual Emitter Incremental realignment. 4. Compute transformation Method: freeze the view and align a virtual object with real object.

19. iMAGIS-GRAVIR / IMAG Pen Calibration: Principles and Procedures Purpose: transformation between the Transmitter CS and the Tip CS. Method: orient the stylus in n representative directions. Emitter CS Fixed tip Compute by SVD (nx3 equations, 6 unknowns).

20. iMAGIS-GRAVIR / IMAG Pen Calibration: Results SVD on 100x3 equationsSVD on 4x3 equations Tracker positions Reconstruction of the tip We propose to keep just 4 measurements.

21. iMAGIS-GRAVIR / IMAG World Calibration Purpose: transformation between the Emitter CS and the World CS. Method: selection of 4 corners of shared area. P1 P2 P3 P4 Emitter CS World CS

22. iMAGIS-GRAVIR / IMAG Calibration Results Error largely acceptable for most practical applications. 2 cm Different views of the source square with a virtual figure added: no disturbing drift.

23. iMAGIS-GRAVIR / IMAG Overview Motivation and Related Works Setup: Small Group and Computing Environment Calibration Procedure –HMD Calibration –Pen Calibration –World Calibration Dynamic Objects –Mixing Real and Virtual objects –Moving Objects Conclusion

24. iMAGIS-GRAVIR / IMAG Dynamics objects: Mixing Real and Virtual Environment Purpose: –Describe real objects to the system. –Tangible User Interface. –Poor man haptic feedback. Requirements: –Occlusion compatibility (« black render »). –Respect of laws of physics (e.g gravity). –Manipulation awareness. Our approach: interactive system easy use based on visual feedback. Digitalized objects. Generic Geometric models. TUI objects.

25. iMAGIS-GRAVIR / IMAG Related Works Some interactions with real objects Whitaker[1995]: image-based and pointer- based object calibration. Mixed Reality[1999]: features on objects. Build-IT[1997]

26. iMAGIS-GRAVIR / IMAG Mixing with Already Digitalized Models (1/2) Purpose: obtain transformation between the object CS and the world CS (known geometry + appearance). Constraints: laying on top of something else (3DOF). Method: superpose virtual representation with real representation of the model. Real objectVirtual object Object CS

27. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone,..etc..) Our approach: Description of simple shapes with few points. Example: add a rectangle. 8 DOF

28. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models P1 5 DOF Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone,..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle.

29. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models P1 5 DOF Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone,..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle.

30. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models P1 2 DOF P2 Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone,..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle.

31. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models P1 0 DOF P2 Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone,..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle.

32. iMAGIS-GRAVIR / IMAG Mixing with New Simple Models: Results

33. iMAGIS-GRAVIR / IMAG Tangible User Interface: Moving Objects Purpose: use real object for manipulation of virtual models. Constraints: one tracker + several real objects to manipulate. Our approach: TEMPORARILY stick the magnetic tracker to previously added objects. Real object

34. iMAGIS-GRAVIR / IMAG Video

35. iMAGIS-GRAVIR / IMAG Conclusion Our contributions Multi-user environment setup. Fast user calibration method with sufficient accuracy for a large range of non-critical applications. Dynamic placement and digitalization of objects. Intuitive metaphor to manipulate objects. Future works Develop interaction techniques. Reduce latency.

36. iMAGIS-GRAVIR / IMAG

37. Mixing with already Digitalized Models (2/2) Real object