The Steerable Projector and Camera Unit in an Instrumented Environment Lübomira Spassova Saarland University, Saarbrücken, Germany

Agenda Overview about instrumented environments Description of the hardware Image correction Current applications Open research questions

Saarland University Pervasive Instrumented Environment (SUPIE)

Hardware Projector (3.000 ANSI lumen, XGA resolution) Digital camera (4 mega pixels) –High quality pictures –Live video stream Moving yoke –Used for spotlights on stages –Steerable via DMX –Pan / tilt range: 340° / 270°

General Ideas Different aspects of everywhere projection: Display continuum: e.g. user interface can be adapted to the user’s position Virtual layer covering the instrumented room: e.g. virtual objects placed on physical surfaces / objects (images, text, highlighting, …)

Problem How to correct the distortion of the image caused by oblique projection?

Solution Exploit the fact that cameras and projectors are geometrically equal

Implementation Create a virtual model of the room with Java3D Calibrate optical parameters and position of virtual camera and projector Project an image taken by a virtual camera synchronized with the projector Steerable projector beam makes virtual objects visible (virtual torch light)

Possible Applications Augmenting physical objects: –Annotating –Highlighting Placing virtual objects Camera-based interaction Combination with spatial audio Moving virtual character

Example: Search Light Scan the room for optical markers (AR Toolkit) Project a spot around the searched object

Example: Wipe Gesture Control Simple wipe gesture recognition by analyzing video stream Objects (images) can be moved from one screen to another

Example: Virtual Room Inhabitant Movable video and audio

Open Questions 3D scene acquisition / self-calibration Integration of a more elaborated gesture recognition Integration of speech / multimodal interaction Interaction between physical and virtual displays Combination of several steerable projectors

Discussion