1. Visual Cues For The Instructed Arrangement of Physical Objects Using Spatial Augmented Reality (SAR)
Jessica Tsimeris
Supervisor: Bruce Thomas
Wearable Computer Lab
University of South Australia

2. What is Augmented Reality?
Properties of Augmented Reality:
Blends real and virtual, in a real environment
Real time interactive
Registered in 3D [thus, the real and virtual objects are aligned]
Azuma et al (2001)

3. What is Augmented Reality?
Typically requires Head Mounted Devices (HMDs).
Differs from Virtual Reality because:
Virtual Reality suppresses the real world
Augmented Reality enhances the real world
Raskar et al (2005)

4. What is Spatial Augmented Reality?
Branch of AR
The paradigm was introduced in 1998 by Raskar et al
Enhances the real world with virtual objects which are integrated into the real world.
Projectors
Flat panel displays
etc

5. Advantages of SAR
Users don’t have to wear a head mounted device or hold a device
Allows the object to be rendered at the real world location
Better for the human eye

6. Advantages of SAR
Multiple user interaction can be achieved without multiple HMDs or mobile devices
The visual fidelity of a physical object is unchanged, only the augmentations are rendered with lesser quality.

7. Advantages of SAR
Allows for larger, brighter, higher resolution virtual objects
Therefore allowing:
Increased integration into the real world
Increased immersion
Improved user interaction
Not restricted to a low resolution
Mobile devices

8. Previous Work
Shaderlamps, projection of textures onto scaled buildings and other real objects
Raskar et al (1999)

9. Previous Work
The BUILD-IT system, a virtual object is manipulated via physical object manipulation
Fjeld (1999)

10. Previous Work Virtual Object Manipulation in Virtual Worlds
Multimodal – speech and gestures, Irawati et al (2006)
Kato et al (2000) addressed a similar problem, but not multimodal
Virtual Object Manipulation in Real Worlds
Arrange virtual objects, surround user in a sphere, Webster et al (1996)
Wang et al (2007) addressed a similar problem, but not limited to a sphere

11. Research Question
What are the appropriate Spatial Augmented Reality visual cues to instruct a user on how to arrange physical objects?

12. Example Use
A stagehand has to arrange sets in a theatre. They would like to rearrange the current sets and include new sets
The system could tell him what objects to move and the order in which to move them

13. Methodology
To arrange the physical objects, the following instructions will be performed:
Rotate
Translate (move)

14. Methodology – Translation Instruction
A circle is projected at the centre of the current location and another is displayed at the centre of the destination location
A line between the circles shows a path from the current position to the destination

15. Methodology – Translation Instruction
The projection updates as the user moves the physical object
The object is successfully translated when the circles overlap

16. Methodology – Translation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

17. Methodology – Translation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

18. Methodology – Translation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

19. Methodology – Rotation Instruction
A line from the centre of the object is displayed to indicate the current rotation, and another line is displayed to indicate the destination rotation
A circle is projected at the end of each line

20. Methodology – Rotation Instruction
The projection updates as the user rotates the physical object
The object is successfully rotated when the circles overlap

21. Methodology – Rotation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

22. Methodology – Rotation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

23. Methodology – Rotation Instruction
Dotted shape is the destination position
Solid shape is the current position of the object

24. Methodology – Visual Cues
Most used the translation and rotation instructions as described

25. Methodology – Visual Cues
Most used the translation and rotation instructions as described

26. Methodology – Visual Cues
Most used the translation and rotation instructions as described

27. Methodology – Visual Cues
Most used the translation and rotation instructions as described

28. Implementation Camera Projector Tracking SAR Module ARToolKitPlus
OpenGL/C++

29. Implementation Two Dimensional Environment
To arrange 2D representations of the physical objects from a top-down view.
Arrangement can then be performed on the corresponding physical objects using SAR cues.
Can be used for remote object arrangement

30. Implementation Object Arrangement Process
Revisit incorrect instructions.

31. Implementation

32. Implementation

33. Implementation

34. Implementation

35. Implementation

36. User Study Nine visual cues to arrange objects One without using SAR
Eight visual cue variations

37. User Study
Task Completion Time
Accuracy
User Opinion (survey)

38. Results
The cue that didn’t use SAR was the second slowest

39. Results
The cue that didn’t use SAR had the least accurate x axis arrangements

40. Results
The cue that didn’t use SAR had the least accurate y axis arrangements

41. Results
The cue that didn’t use SAR had the least accurate rotation

42. Results
95% of participants preferred Task 2 (Translation First) over the manual arrangement

43. Results
75% of participants preferred visual cues that utilised a square projection over visual cues that utilised a wireframe projection

44. Results
Visual cues that displayed both rotation and translation at the same time were most preferable.

45. Conclusion SAR visual cues have been developed that are:
Faster than a manual arrangement technique
More accurate for x axis translation
More accurate for y axis translation
More accurate for rotation
Preferred over a manual arrangement technique

46. Improved Visual Cue
Development of a new visual cue that took the user study results into account, particularly:
Square Cues were quicker
One of the Square Cues was the most accurate cue
Participants preferred seeing both translation and rotation at the same time
More intuitive
Participants ranked the square cues highly
Legible, less cluttered

47. Improved Visual Cue
Developed from results of the user study

48. Improved Visual Cue
Developed from results of the user study

49. Improved Visual Cue
Developed from results of the user study

50. Improved Visual Cue
Developed from results of the user study

51. Improved Visual Cue
Developed from results of the user study

52. Improved Visual Cue
Developed from results of the user study

53. Improved Visual Cue
Developed from results of the user study

54. Improved Visual Cue
Developed from results of the user study

55. Improved Visual Cue
Developed from results of the user study

56. Improved Visual Cue
Developed from results of the user study

57. Future Work The order in which the objects will be arranged
Better tracking
Formal evaluation of the new visual cue
Multiple cameras and projectors
Testing scalability

58. References
Azuma, R 1997, ‘A Survey of Augmented Reality’, Presence: Teleoperators & Virtual Environments, vol. 6, no. 4, pp
Bimber, O & Raskar, R 2005, Spatial Augmented Reality: Merging Real and Virtual Worlds, AK Peters, Ltd, Wellesley, MA.
Fjeld, M, Voorhorst, F, Bichsel, M, Lauche, M, Rauterberg, M & Krueger, H 1999, 'Exploring Brick-Based Navigation and Composition in an Augmented Reality', Handheld and Ubiquitous Computing, vol. 1707, pp
Irawati, S, Green, S, Billinghurst, M, Duenser, A & Ko, H 2006, '"Move the Couch Where?": Developing an Augmented Reality Multimodal Interface', Symposium on Mixed and Augmented Reality, Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp
Kato, H, Billinghurst, M, Poupyrev, K, Imamoto, K & Tachibana, K 2000, 'Virtual Object Manipulation on a Table-Top AR Environment', Human Interface, pp
Raskar, R & Low, K-L 2001, Interacting With Spatially Augmented Reality, ACM, Camps Bay, Cape Town, South Africa, pp
Wang, X & Gong, Y 2007, Augmented Virtuality Space: Enriching Virtual Design Environments with Reality, Brisbane, Australia.
Webster, A, Feiner, S, MacIntyre, B, Massie, W & Krueger, T 1996, Augmented Reality in Architectural Construction, Inspection, and Renovation.

59. Questions?