Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa Ishikawa-Namiki-Laboratory Parallel Processing for Sensory information University of Tokyo

remove the need for any physical I/O space at all remove the need of any additional input device …Next step ? UI : “WIMP” + alphanumeric input I/O Device: touch sensitive screen+stylus Today’s approach nice: input space is merged with viewing space I. The problem: physical interaction Input and interaction with “ever-shrinking” portable electronic devices (*) windows, icons, menus and pointer  ”ergonomics-independent” miniaturization

How to remove the input space? In a word: passive-vision input versatile but not mature yet… …in any case overkill for the problem at hand! (“inputspace-less” WIMP) May require intensive computation Not robust enough Sensitive to external illumination conditions/backgrounds... drawbacks: (non-invasive techniques!)...By providing the PDA with intelligent sight: - handwritten character recognition - gestures recognition, etc Low, medium or high- level vision - hand/finger position

Closer look at the problem... -Hand/finger closer than anything else -fingertip always visible (no occlusion) -fingertip “shape” simple and stable animal antenna proximity sensor angular information should provide sufficient data for tracking! cane for the blind what a short-sighted PDA would actually see…  cost- efficient, ROBUST, FAST! …but cumbersome, annoying?

- laser can be used as output device (on any external surface, including the retina: VRD) And... The proposed “antenna”: a directive beam of light advantages: - active lighting: provides and control illumination - real-time, precise 3D measures (telemetry) - measurement of surface roughness, speed, even biometric data. Inconvenients: no haptic feedback (just like passive vision, but visual cues possible).

... but how realistic is the integration of a LASER RADAR on a PDA? VERY! Thanks to today’s “MOEMS” technology... The realm of “MOEMS” (micro-opto-electro-mechanical systems): Electronic circuit, photodetectors, laser diodes sources and mirrors integrated in the SAME chip.

II. Smart Laser Scanner Prototype laser diode, a pair of steering micro-mirrors, single non-imaging photodetector. Hardware (discrete components):

circular laser “saccade” Tracking Principle smart laser scanner: laser excursion is locked around the area of interest Tracking sequence repeated every millisecond. kHz refreshing rate explains algorithmic extreme simplicity

Prototype Smart Scanner in Action

2 DOF (using fingertip) (angular precision: better than tenth of a degree )

3 DOF (using fingertip) Depth precision: (intensity based) around 1cm up to 2 m

a little farther away …

Simultaneous tracking (up to four targets without hardware duplication)

Several users…

… unique user, higher DOF

Achieving 6 DOF (using three tracking points)

Fast 3d tracking of a ping-pong ball

Interactive laser annotation

UIs based on this 3D Input Device: Proof-of-principle examples

X/Y/Z translation RIGTH CLICK LEFT CLICK Gestural command: 3D locator

X/Y translation ZOOM (-) Interface for visualizing 2D data LEFT CLICK RIGHT CLICK Gestural command: Positional command: ZOOM (+)

X/Y translation Frame position (-) Zoom x2 Zoom /2 Gestural command: Positional command: Frame position (+) Visualization interface for video

Conclusion The problem: design a 3D input device that does not interfere with technological miniaturization Proposed solution: use laser “light antennas” to track one (or more) fingers. Advantages: fast (no image processing), precise 3D measurements, insensitive to external light conditions, no markers necessary, can be used as output device.

Further Work Study of adaptive saccade shapes (ex: follows the hand contour) Prototype without mobile parts, using multiple laser beams... MOEMS integration: PDA on a keychain, PDA on a chest-pin: touching the pin or activates tracking, a gesture can be used to ask for s, and these are read on the hand...

Precision (cm) Speed (3m/s) multi-tracking (up to four fingers) Interactive laser annotation see more at both #102 ACR