1. Hrvoje Benko, Andrew D. Wilson, and Ravin Balakrishnan Microsoft Research – University of TorontoSphere Multi-touch Interactions on a Spherical Display

2. 2UIST '08

3. Spherical Displays 3 UIST '08 Pufferfish – PufferSphere ARC Science – OmniGlobe Kettner et al. ‘04 – ViBall Global Imagination - MagicPlanet Companje et al. ‘07 – Globe4D NOAA – Science on a Sphere Not directly interactive Physical rotation only

4. Interaction Metaphor Shift 4UIST '08

5. Sphere  Multi-touch sensitive  Built on Global Imagination’s MagicPlanet  Internal projection and vision-based sensing 5 UIST '08 GOAL: Explore interactive possibilities of spherical displays.

6. Video: Sphere UIST '086

7. Sphere Video UIST '087

8. Sphere Hardware UIST '088 16” 24”

9. 9

10. 10

11. Reusing the Optical Path 11UIST '08

12. Video: Touch-Sensing UIST '0812

13. Touch-Sensing Software Pipeline UIST '0813

14. Sensing and Projection Distortions Vertex shader:  Pre-distorts objects into 2D disk image  Objects appear undistorted on the spherical surface UIST '0814 2D 3D

15. Video: Projection Distortions UIST '0815

16. Projection Distortions UIST '0816

17. Choosing a Coordinate System 17 2D (Disk) Flat (Cylindrical) 3D (Spherical) Hybrid (Tangential Plane) UIST '08

18. Application Prototypes  Photo & Video Browser  Omni-Directional Data Visualization Globe Omni-camera Panoramic Walkthrough  Sphere Painting  Sphere Pong UIST '0818

19. Design Space of Spherical Displays  Non-visible hemisphere  Visibility changes with position “Pseudo-private” and “public” areas  No master user position / orientation  Smooth transitions in depth and orientations Near – far Horizontal – vertical  Borderless, but finite display  Few natural landmarks: poles 19UIST '08

20. Multi-Touch Interaction Techniques  Multi-touch basics Drag, Rotation, Scale  Extend user’s reach Flicking, Send-to-Dark-Side  Minimize orientation difficulties Auto-rotation, Tether  Universal circular menu Orb-like invocation gesture UIST '0820

21. Dragging UIST '0821 V Flat Surface 1 2 Q A θ 1 2 Spherical Surface

22. Send-to-Dark-Side  Open-palm gesture  Control the location where the object will appear 22UIST '08

23. Auto-rotation  Similar to Shen et al. DiamondSpin 23 Default behavior Auto-rotation UIST '08

24. Tether  Allow unconstrained manipulation  Preserve “natural” orientation of omni- directional data upon release 24UIST '08

25. Universal Circular Menu  Orb-like Invocation Easy to perform – Hard to do by accident 25UIST '08

26. Video: Sphere Interactions UIST '0826

27. Sphere Multi-Touch Interactions UIST '0827

28. Video: Contact Area Interactions UIST '0828

29. Sphere Paint & Pong UIST '0829

30. User Observations  3 high-traffic demos >1000 people  Feedback: “Magical”, “Like a crystal ball” “What is it good for?” “Can I zoom?”  Interaction zones: Top – shared Middle – personal/working Bottom – storage  Logs show no position preference 30UIST '08

31. Future Work  Zooming  Sphere as a large controller  Sphere as a room projector (instant CAVE)  Handheld Sphere UIST '0831

32. UIST '0832

33. UIST '0833 benko@microsoft.comhttp://research.microsoft.com/~benko/projects/sphere Try our demo at the reception tonight!

34. 34UIST '08

35. Applications  Visualizing spherical data Planets, stars  Public information booths  Remote omni-directional visualization Sphere + RoundTable 360 deg. video-conferencing Robot operation  Ambient Displays  Large displayable input device Trackball Space Mouse / Joystick Planetarium Control  Games 35UIST '08

36. Conclusions  The first multi-touch sensitive spherical display Self-enclosed solution without occlusion and shadowing problems  A set of multi-touch interactions that facilitate data sharing around the display  Omni-directional room projection possibility Instant CAVE UIST '0836

37. Uses 37UIST '08

38. 38 benko@microsoft.comhttp://research.microsoft.com/~benko/projects/sphere Come to our demo at the reception tonight!

39. UIST '0839 M.C. Escher benko@microsoft.comhttp://research.microsoft.com/~benko

40. Putting it all together… 40UIST '08

41. Two Spheres (of Interest) 41UIST '08

42. Sphere Pong UIST '0842

43. Sphere Painting UIST '0843

44. Extending User’s Reach UIST '0844

45. Auto-rotation & Menu UIST '0845

46. Tether Behavior UIST '0846