1. 3D User Interface Input Hardware
Presented by Ronn Quijada

2. Some quick Terminology
Active vs. Passive Sensors - Do you have to interact with it to work?
Inside-Out vs. Outside-In tracking - Do we track relative from the user or from the world?
Isotonic vs. Isometric joystick- Can we move the joystick?
Degrees of Freedom (DOF) - How many ‘sliders’ can we control?

3. Input Device Categorization
DOF - How many Degrees of Freedom the device can be manipulated
Input Type / Input Frequency
Discrete input (boolean) vs Continuous (range)
Input once vs constant stream
Intended Use
Pick - Which one in a set
Locator - Where it is
Valuator - What value it is

4. Traditional Input

5. Traditional Input Keyboards 2D mice and Trackballs
Pen and Touch-based Tablets
Joysticks
Desktop 6-DOF input devices
Modelling, computer games, spreadsheets; traditional input based on 2D user interfaces, but can be used in 3D interfaces as well (albeit not as well as other solutions)
Pen and touch based tablets (pictured wacom digitizer)
Joystick: Isotonic - PS4 LR sticks, Isometric - 3D connexion, (pictured atari 2600 controller, technically isotonic)
6DOF: Pan horizontal/vertical, tilt, spin, roll, zoom in/out

6. 3D Interface Adapted Traditional Input
Chord Keyboards
Virtual Keyboards
Handheld 2D mice
One handed, press keys in a ‘chord’ to represent a traditional keystroke
Pictured Twiddler2

7. 3D Spatial Input

8. 3D Spatial Input Primer: Sensing Types
Magnetic Sensing - Magnetic field disturbances to determine location
Mechanical Sensing - Hardware such as potentiometers and shaft encoders to track movement
Acoustic Sensing - Ultrasonic pulses, with reflection to determine position
Inertial Sensing - Gyroscopes, Accelerometers, detects changes in velocity and orientation for tracking
Optical Sensing - cameras and 3D vision to provide tracking
Radar Sensing - Radar reflections for tracking
Bioelectric Sensing - senses electrical activity in the body to infer pose or gesture
Hybrid Sensing - combination of techniques

9. Head, Shoulders, Knees, and Toes (Knees and Toes) Tracking
Limb/Head Tracking
Tracks large movements, usually done with optical tracking with inertial sensors
Finger Tracking
bend-sensing technology
Pinch-sensing technology - checks for contact between fingers
Eye Tracking
Optical sensing, using corneal reflections
Pictured, Plexus finger tracking

10. In-Depth: HTC Vive Markerless Inside-Out Tracking System
Lighthouses project infrared light for optical sensors on headset and wands
Optical sensors for ground truth, Inertial sensors for small movements

11. In Depth: Optical Sensing Techniques
Marker Based Outside-in tracking
Optical sensors tracking markers on subjects
Tracking based on known objects
Broken into primitives beforehand
Simultaneous Localisation and Mapping (SLAM)
Building models of the environment using the input data
Pictured: slam visualisation

12. Complementary Inputs

13. Complementary Inputs Speech Input Brain Input Speech recognition
Microphone placement
Implicit push-to-talk (‘alexa’, ‘ok google’)
Brain Input
Brainwave activity as input
Electroencephalogram (EEG)
fMRI, PET, fNIRS can also be used, but impractical for HMDs
Pictured Emotiv EEG (14 channels)

14. Other Weird Bits of Input
ShapeTape
For creating 3D curves
Armature Controllers
Input based on skeletons
Used primarily for animation
iSphere
24 DOF input for 3D modelling
CavePainting Table
Tracked brush to simulate painting
Pictured The Dinosaur input device, used in animating the dinosaurs in Jurassic Park

15. Conclusion Input Categories of DOF, Input Type, and Intended Use
Traditional 2D, Spatial 3D and Complimentary Input types
Traditional discrete and continuous inputs
Different sensing types
Combinations of sensing mechanisms to build Spatial 3D inputs

16. In Case We Have Time… (10:53)