1 CrystalDome: A Projected Hemispherical Display with a Gestural Interface Francis T. Marchese, Jonas Borjesson, Josh Rose Pace University New York, NY

2 ARC Science Simulations Globe4D R. Companje, N. van Dijk, H. Hogenbirk, and D. Mast, “Globe4D: time-traveling with an interactive four-dimensional globe,” In Proceedings of the 14th Annual ACM international Conference on Multimedia (October, 2006), pp

3 Actuality Systems Volumetric Display

4 S. Webb and C. Jaynes, “The DOME: A portable multi-projector visualization system for digital artifacts”, IEEE Workshop on Emerging Display Technologies, Bonn, Germany, The DOME

5 CrystalDome Components 1.Microcontroller-sensor system 2.Graphics-projection system 3.Interface software

6 Display System Takes flat image and projects it onto hemisphere. 2D spherical projection of Earth correctly maps to the hemisphere with no intermediate processing. Single point projection of 3D scene will not project properly.

7 Display System Every point on the dome’s surface is a unique viewpoint in which the observer looks into the hemisphere towards its center.

8 Algorithm Variation on ray casting Every point on the dome’s surface a viewpoint. Objects warped along a radial path from the sphere’s center to its surface. Hidden surface methods are applied. Application is written in OpenGL.

9 Interaction Schemes (1) Gestural interaction T. Grossman, D. Wigdor, and R. Balakrishnan, “Multi-finger gestural interaction with 3D volumetric displays.” In Proceedings of UIST ’04, (October 24–27, 2004), Santa Fe, New Mexico, pp

10 Interaction Schemes (2) Wireless Controller Y. Yasuhara, N. Sakamoto, N. Kukimoto, Y. Ebara, and K. Koyamada, “Interactive controller for 3D contents with omni- directional display,” In Proceedings of ICPADS, 2005, pp

11 Sensor System Sensors associated with input control: x-axis, y-axis, and zoom. Microcontroller programmed in Basic to send sensor id and data to the serial port. Motion to or from sensor causes images to move predictably. 3 sonar sensors range -> 2cm to 3m Basic Stamp 2 microcontroller, Parallax, Inc.

12 Interface System Interface controller Acts as interface between sensors and any program Keeps track of the current system state Reads sensor data from the serial port, interprets it, and issues commands that are recognized by the application programs. Can issue either keyboard or mouse commands. Written in Processing.org Applications adapted to sensors Google Earth for mouse input 3D OpenGL program for keyboard commands.

13 Conclusions Hemispherical display works well for geographical charts because they map as flat surfaces. Projection algorithm for three-dimensional objects gives a sense of depth within the display. System is less successful at producing a sense of transparency and depth, characteristics intrinsic to true volumetric displays but … Objects rendered in the hemisphere maintain a better sense of solidity and light and shadow Number of sensors and their placement remain to be optimized for multiple users.

14 Opportunities for Future Work Spatial warping algorithm must –be expanded to support additional geometric primitives such as polygon meshes and nurbs. –render spatially warped objects with a sufficient degree of tessellation to map properly to the hemisphere, but not overly tessellated to waste resources. User Interface must be optimized for –the Number of sensors and their placement –the types of gestural interactions employed to control CrystalDome