1. Jefferson Y. Han, New York University
Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection
Jefferson Y. Han, New York University
Presented by: Cody Boisclair

2. Introduction Touch sensitivity is fairly common in electronics today…
…but only for a single point of contact at any given time.
Touch-sensitive devices that allow for multiple points of contact have thus far been expensive or difficult to produce.

3. Why multi-touch?
Allows a user to interact with a system with more than one finger at once
Chording
Operations using both hands
Also allows multiple users to interact on the same touch-sensitive platform
Interactive walls
Interactive tabletops

4. Prior approaches to the problem
Matrix of smaller sensors
Requires many connections, which severely limits resolution
Visually opaque - no touch-screen
Video cameras
Measuring of brightness
Stereo cameras
Identifying tracking markers in gel

5. A new approach… F T I R
rustrated
otal
nternal
eflection

6. Total Internal Reflection
When light encounters a medium with a lower index of refraction (e.g., going from glass to air), its refraction depends on the angle at which it hits the border.
Beyond a certain critical angle, light is not refracted, but instead reflects entirely within the material.
This is the basis for fiber optics and other optical wave guides.

7. Frustrated Total Internal Reflection
If another material touches that within which the light is reflecting, the reflection is frustrated, causing the light to escape.
This has been used in the past:
fingerprint imaging
early touch sensors (1970s!)
tactile sensors for robotic grips

8. A Schematic of FTIR
(Kasday, 1984)

9. Using FTIR for touch sensitivity
A clear acrylic sheet is used as the touch surface.
16 inches x 12 inches in prototype
Edges of surface lit by infra-red LEDs to produce total internal reflection.
A video camera is mounted under the surface and facing it.
When the surface is touched, the light escapes and registers on the camera.

10. Using FTIR for touch sensitivity
Basic image-processing techniques are performed on the camera output to identify the points of contact.
Computer-vision techniques are used to interpret the motion of contact points as discrete touches or strokes.
Processing easily handled in real-time by a 2 GHz Pentium IV processor

11. Advantages to this approach
High capture rate and resolution
30 frames per second
640x480
True zero-force touch sensitivity
Inexpensive to construct
Scalable to much larger (even wall-sized!) surfaces
Transparent: can be combined with rear-projection display
“True zero-force”: pressure not necessary, but can identify touch from hover

12. How is projection achieved?

13. Disadvantages
Requires significant space behind touch surface for camera
Gloves, certain types of styluses, and even dry skin may not register
a function of refractive index
Residues on surface (e.g., sweat) also produce FTIR effect that may build up

14. Video Clip at NYU (Local Copy)
Some applications…
Video Clip at NYU (Local Copy)
Uses a 36x27-inch rear-projection screen.
Touch information is sent to programs using OSC (Open Sound Control) protocol

15. Any Questions? ?