Hrvoje Benko, Andrew D. Wilson, and Ravin Balakrishnan Microsoft Research – University of TorontoSphere Multi-touch Interactions on a Spherical Display

2UIST '08

Spherical Displays 3 UIST '08 Pufferfish – PufferSphere ARC Science – OmniGlobe Kettner et al. ‘04 – ViBall Global Imagination - MagicPlanet Companje et al. ‘07 – Globe4D NOAA – Science on a Sphere Not directly interactive Physical rotation only

Interaction Metaphor Shift 4UIST '08

Sphere  Multi-touch sensitive  Built on Global Imagination’s MagicPlanet  Internal projection and vision-based sensing 5 UIST '08 GOAL: Explore interactive possibilities of spherical displays.

Video: Sphere UIST '086

Sphere Video UIST '087

Sphere Hardware UIST '088 16” 24”

9

10

Reusing the Optical Path 11UIST '08

Video: Touch-Sensing UIST '0812

Touch-Sensing Software Pipeline UIST '0813

Sensing and Projection Distortions Vertex shader:  Pre-distorts objects into 2D disk image  Objects appear undistorted on the spherical surface UIST '0814 2D 3D

Video: Projection Distortions UIST '0815

Projection Distortions UIST '0816

Choosing a Coordinate System 17 2D (Disk) Flat (Cylindrical) 3D (Spherical) Hybrid (Tangential Plane) UIST '08

Application Prototypes  Photo & Video Browser  Omni-Directional Data Visualization Globe Omni-camera Panoramic Walkthrough  Sphere Painting  Sphere Pong UIST '0818

Design Space of Spherical Displays  Non-visible hemisphere  Visibility changes with position “Pseudo-private” and “public” areas  No master user position / orientation  Smooth transitions in depth and orientations Near – far Horizontal – vertical  Borderless, but finite display  Few natural landmarks: poles 19UIST '08

Multi-Touch Interaction Techniques  Multi-touch basics Drag, Rotation, Scale  Extend user’s reach Flicking, Send-to-Dark-Side  Minimize orientation difficulties Auto-rotation, Tether  Universal circular menu Orb-like invocation gesture UIST '0820

Dragging UIST '0821 V Flat Surface 1 2 Q A θ 1 2 Spherical Surface

Send-to-Dark-Side  Open-palm gesture  Control the location where the object will appear 22UIST '08

Auto-rotation  Similar to Shen et al. DiamondSpin 23 Default behavior Auto-rotation UIST '08

Tether  Allow unconstrained manipulation  Preserve “natural” orientation of omni- directional data upon release 24UIST '08

Universal Circular Menu  Orb-like Invocation Easy to perform – Hard to do by accident 25UIST '08

Video: Sphere Interactions UIST '0826

Sphere Multi-Touch Interactions UIST '0827

Video: Contact Area Interactions UIST '0828

Sphere Paint & Pong UIST '0829

User Observations  3 high-traffic demos >1000 people  Feedback: “Magical”, “Like a crystal ball” “What is it good for?” “Can I zoom?”  Interaction zones: Top – shared Middle – personal/working Bottom – storage  Logs show no position preference 30UIST '08

Future Work  Zooming  Sphere as a large controller  Sphere as a room projector (instant CAVE)  Handheld Sphere UIST '0831

UIST '0832

UIST '0833 Try our demo at the reception tonight!

34UIST '08

Applications  Visualizing spherical data Planets, stars  Public information booths  Remote omni-directional visualization Sphere + RoundTable 360 deg. video-conferencing Robot operation  Ambient Displays  Large displayable input device Trackball Space Mouse / Joystick Planetarium Control  Games 35UIST '08

Conclusions  The first multi-touch sensitive spherical display Self-enclosed solution without occlusion and shadowing problems  A set of multi-touch interactions that facilitate data sharing around the display  Omni-directional room projection possibility Instant CAVE UIST '0836

Uses 37UIST '08

38 Come to our demo at the reception tonight!

UIST '0839 M.C. Escher

Putting it all together… 40UIST '08

Two Spheres (of Interest) 41UIST '08

Sphere Pong UIST '0842

Sphere Painting UIST '0843

Extending User’s Reach UIST '0844

Auto-rotation & Menu UIST '0845

Tether Behavior UIST '0846