A Laser Pointer by Uri Gordon & Mark Shovman for PostPC, fall 2002

Concept layer User layer Implementation layer Presentation roadmap

Problem Definition Consider a computer-based presentation with a projector as a display device and a laser pen as a pointing-out device. To affect the computer, the presenter has to use an additional input device - a mouse or a keyboard. This results in the interruptions in the presentation flow, breaking contact with the audience and loosing the thread of thought.

Existing Solutions Holding wireless mouse in other hand Reading position of non-laser pointer by external sensors (bluetooth, UW, IR) Adding accelerometers on a laser pen and reading its movements into the computer (Previous year project)Previous year project

Problems with Existing Solutions Special devices are uncommon, and, hence, expensive and hard to support. Virtual design environments – clear overkill for a simple presentation.

Our Aim Developing a uniform, intuitive and cheap interface to control a computer during presentation, without breaking contact with audience and with no interruptions in the presentation flow.

Our Solution Add a common webcam to identify a laser spot on the projection surface. Use this datum to perform common presentation control tasks.

Information Flow Chart Our Solution Add a common webcam to identify a laser spot on the projection surface. Use this datum to perform common presentation control tasks. Update image Register change Point out Show

Benefits Using cheap, over-tested COTS hardware. Any laser pen, any projector, any webcam Thus, independent of specific hardware. Easy to upgrade (software only). Extensive hardware support (not us).

Marketing Strategy Via projector manufacturers, as a part of projector software bundle Main hardware requirement Most expensive component Exactly the same target group

Marketing Problem Recent research reveals that there is an almost identical product – Lightmouse by IC Tech. Copyrights may exist. That project probably failed, may suggest possible faults – technical, conceptual, marketing etc.

Future Developments Merging with products besides slideshow Mainly mouse-driven Home multimedia centers Internet browsers Wholesale windows GUI engine Creating dedicated applications Games - Harry Potter emulation Adding more and more complex gesture analysis

Concept layer User layer Implementation layer Presentation roadmap

Laser/Pointer interface Spot detected Spot disappeared Draw a cursor around spot Turn cursor off (after a delay) Spot lingers on hot point Turn hot point on Spot leaves hot point Turn hot point off (after a delay)

Gesture Interface Spot disappears (hot point is on) Activate hot point Spot disappears across left/right borders Toggle menu Spot disappears across Up/Down borders Move to next/previous slide

Floating menu commands Previous Slide Next Slide Navigation Menu Recalibration Help Hide Menu Kill Application

Setup and Calibration Ensuring that the webcam captures all the projected image with minimal distortions. The webcam output (image) is shown in a window. The user is requested to set the webcam and the projector, so that the all image projected is seen in the window, as straight as possible, and as fully as possible. Ensuring that the laser spot will always be detected. The user is requested to put the spot into a designated area on screen. Spot color calibration. Screen reflection values.

Please Bring Laser Spot inside the red square on the top left.

Concept layer User layer Implementation layer Presentation roadmap

Algorithm description Laser Spot Detection- Calibrating-  Either: 3x3 matrix of affine transform between Webcam coordinates and display coordinates,  or: full transform function – for each webcam pixel, get a mapping to display pixels; store reflection coefficient for each pixel. Detection-  Subtract the current image from the previous one.  Search for a range of RGB values suspected to be the laser pointer.  Use the calibration parameters to detect its location on the presentation.

Algorithm description – Cont. Gestures Detection and identification Motion vectors creation. Matching of motion vector to predefined patterns. Acting on the detected gesture.

Technical Issues - Hardware base Camera – Refresh Rate Focus Depth Angle of view Color/Brightness sensitivity Projector – Refresh Rate Image Geometry Distortion Screen – Reflection constants. Intrinsic color Computer- Speed and Memory Laser Pointer- Spot light dispersion Physical size Color and brightness

Technical Issues - Software base OS Currently windows-XP. Coding platform VC++ with MFC and/or DirectShow Possible Application Environments Currently PowerPoint

Expected problems and possible solutions Laser spot detection - solved by specific calibration High reflection variability – calibrate at several points Low luminance – measure exact spectrum High projector curvature – use full matrix Gestures detection and identification Technical Problems High processor load Refresh rates ’ insufficiency or incompatibility

Current State of Development Offline experimentations of laser- pointer detection. Data collection of variability of hardware specifications. Studying the software tools for realtime implementation.

Conclusions The application answers the needs we defined. The application seems to be technically realizable. It is necessary to analyze the failure of “ LightMouse ”.