LEAP MOTION: GESTURAL BASED 3D INTERACTIONS Thanh Nguyen and Tammy Dinh CS320: User Interface

What is Leap Motion? - Is a computer hardware sensor device - Enables users to interact with the computer using hand motion - Developed and marketed by Leap Motion Co. in May, 2012

What is Leap Motion? Computer hardware sensor device Enables users to interact with the computer using hand motion Developed and marketed by Leap Motion Co. in May, 2012

Demo Playground

How does Leap Motion work? Size of a matchbox is placed horizontally on the desktop in front of the computer screen It enables continuous tracking of multiple hands with high accuracy Leap Motion controller and the coordinate system used to describe positions in sensory spaces

How does Leap Motion work? Controller has two cameras and three infrared LEDs Hardware: Scans infrared light to capture the surrounding Generates greyscale stereo image of the near-infrared light spectrum Software: Analyzes and reconstructs 3D representation of the greyscale image Extracts tracking information (hands, fingers, & gestures) Diagnostic visualizer: Shows real-time image input and virtual hand output These track infrared light with a wavelength of 850 nanometers, which is outside the visible light spectrum Distinguish foreground object from background Thanks to its wide angle lenses, the device has a large interaction space of eight cubic feet, which takes the shape of an inverted pyramid – the intersection of the binocular cameras’ fields of view

Applications Medical (e.g. physical therapy for rehab) Education (e.g. medical training) Gaming Music, art, & design UI/UX design VR/AR

Demo Cyber Science Motion Zoology

Benefits Portable Affordable Provides more intuitive ways to interact with 3D user interfaces Well-documented API

Drawbacks Limited sensory space Inconsistent across different LM software Premature community for developers

Using Leap Motion with VR

Setting up in Unity Requires either V2 SDK for Desktop applications or Orion SDK for VR Essential asset is Core – available on the website Drag and drop prefabs & configure attributes

Questions?