1. Introduction to Tracking
Bowman, et al., Section 4.3
Welch, Greg and Eric Foxlin (2002). “Motion Tracking: No Silver Bullet, but a Respectable Arsenal,” IEEE Computer Graphics and Applications, special issue on “Tracking,” November/December 2002, 22(6): 24–38.. (
Oct, 2010
C, Lok, Babu 2010

2. Motivation We want to use the human body as an input device
More natural
Higher level of immersion
Task performance
Control navigation
head
hand
Control interaction
body
This requires:
Signaling (button presses, etc.)
Location. <- this is tracking!
Oct, 2010
C, Lok, Babu 2010

3. Tracking Pose (how many variables?) What do we want to track?
Pose (how many variables?)
Position
Orientation
What do we want to track?
Head pose
Hand pose
Other body part
Other objects (e.g. spider)
So what does it mean if a tracking system reports your head at 2.5,3.3, 1.9?
Oct, 2010
C, Lok, Babu 2010

4. Receiver coordinate system Origin for tracker coordinate system
Basic Idea
Trackers provide location and/or position information relative to some coordinate system.
(x,y,z) (rx,ry,rz) X Y Z
(0,0,0)
Receiver coordinate system
(0,0,0)
Origin for tracker coordinate system
Oct, 2010
C, Lok, Babu 2010

5. Degrees of freedom
The amount of pose information returned by the tracker
Position (3 degrees)
Orientation (3 degrees)
There are trackers that can do:
only position
only orientation
both position and orientation
Oct, 2010
C, Lok, Babu 2010

6. Question
Okay, given that I want to track your head, I attach a new tracker from NewTracker Corp. it returns 6 degrees of freedom (6 floats). What questions should you have?
In other words, what are some evaluation points for a tracking system?
5 minutes to discuss
Oct, 2010
C, Lok, Babu 2010

7. Evaluation Criteria Data returned Spatial distortion (accuracy)
3, 6, 6+ DOF
Spatial distortion (accuracy) mm
Resolution
Jitter (precision)
Drift
Lag
Update Rate (2000 Hz)
Range (40’x40’ – GPS)
Interference and noise
Mass, Inertia and Encumbrance
Number of Tracked Points (1-4, 128)
Durability
Wireless
Price
$30,000+ wide area
$180k+ motion capture
Which of these are most important?
Oct, 2010
C, Lok, Babu 2010

8. Performance Measures Registration (Accuracy) – Resolution Jitter Drift
Difference between an object’s pose and the reported pose
Location
Orientation
What are determining factors?
Resolution
Granularity that the tracking system can distinguish individual points or orientations
Jitter
Change in reported position of a stationary object
Drift
Steady increase in error with time
Oct, 2010
C, Lok, Babu 2010

9. Accuracy vs. Precision
Accuracy – refers to how close the measured point comes to measuring a “true value”.
Precision – refers to how closely repeated measurements of a point come to duplicating measured values.
Oct, 2010
C, Lok, Babu 2010

10. Performance Measures t0 – time when sensor is at point p
t1 – time when sensor reports p
Lag or Latency – t1 - t0
What makes up latency?
Acquisition
Transmission
Filtering
Oct, 2010
C, Lok, Babu 2010

11. Performance Measures t0 – time when sensor is at point p
t1 – time when sensor reports p
Lag or Latency – t1 - t0
What makes up latency?
Acquisition
Transmission
Filtering
What is a minimum?
Oct, 2010
C, Lok, Babu 2010

12. Update Rate Number of tracker position/orientation samples per second
High update rate != accuracy
Poor use of update information may result in more inaccuracy
Communication pathways and data packet size are important
Oct, 2010
C, Lok, Babu 2010

13. Range Working volume Position and orientation range could be different
What is the shape?
Accuracy decreases with distance
Range is inversely related to accuracy
Position and orientation range could be different
Sensitivity not uniform across all axis
Oct, 2010
C, Lok, Babu 2010

14. Interference and Noise
Interference - external phenomenon that degrades system’s performance
Each type of tracker has different causes of interference/noise
Occlusion
Metal
Noise
Environmental (e.g. door slamming, air conditioner)
Oct, 2010
C, Lok, Babu 2010

15. Mass, Inertia and Encumbrance
Do you really want to wear this?
Inertia
Tethered
Oct, 2010
C, Lok, Babu 2010

16. Multiple Tracked Points
Number of potentially tracked points
Unique
Simultaneous
Difficulties
Interference between the sensors
Multiplexing
Time Multiplexing – Update rate of S samples per second and N sensors results in S/N samples per sensor per second
Frequency Multiplexing – Each sensor broadcasts on a different frequency. More $$
Oct, 2010
C, Lok, Babu 2010

17. Price You get what you pay for. ($30-$100k+)
Rich people are a small market.
Oct, 2010
C, Lok, Babu 2010

18. Tracking Technologies
Different Tracking Technologies
Goals:
Understand how they work
Understand tradeoffs
Know when to use which
Future directions
Oct, 2010
C, Lok, Babu 2010

19. Mechanical Linkage Rigid jointed structure One end (base) is fixed
The other (distal) is free
Distal is user controlled to an arbitrary position and orientation.
Sensors at the joints detect the angle
Concatenate translates and rotates
Determine the position and orientation of the distal relative to the base.
Oct, 2010
C, Lok, Babu 2010

20. Mechanical Tracking Pros: Cons: Accurate Fast Low lag
Data returned: 6 DOF
Spatial distortion – mm
Resolution – very high
Jitter (precision) – very low
Drift - none
Lag – >5ms
Update Rate Hz
Range - 8 ft
Number of Tracked Points – 1
Wireless - no
Interference and noise – metal, earth
Mass, Inertia and Encumbrance – substantial
Durability – low
Price – high
Pros:
Accurate
Fast
Low lag
Minimal environmental interference
No calibration
Can incorporate force feedback
Cons:
Low range (effectively 5’ – does not scale well)
Cost
1 tracked point (body/others are hard to track)
Oct, 2010
C, Lok, Babu 2010

21. Mechanical Tracking Products
Fake Space Labs BOOM Display (discontinued)
Sensible Phantom
Oct, 2010
C, Lok, Babu 2010

22. Electromagnetic Trackers
Emitter
Apply current through coil
Magnetic field formed
3 orthonormal coils to generate fields
Sensor
Strength attenuated by distance
3 orthonormal magnetic-field-strength sensors
Determine the absolute position and orientation of a tracker relative to a source.
Polhemus (a.c.)
Ascension (d.c.)
Oct, 2010
C, Lok, Babu 2010

23. Basic Principles of EM Trackers
Pulse the emitter coils in succession
Sensor contains 3 orthogonal coils
For each pulse, sensor measures the strength of the signal its 3 coils (9 total measurements)
Known:
Pulse strength at the source
Attenuation rate of field strength with distance
Calculate position and orientation of the sensor coils
Oct, 2010
C, Lok, Babu 2010

24. EM Trackers Data returned: 6 DOF Spatial distortion – 0.6 mm, 0.025°
Resolution – mm, 0.025° / inch from receiver
Jitter (precision) – mm to cm
Drift - none
Lag – reported 4 ms
Update Rate Hz
Range - 5 ft
Number of Tracked Points – 16 (divides update rate)
Wireless - yes
Interference and noise – metal, earth
Mass, Inertia and Encumbrance - minimal
Durability - high
Price - $4000+
Oct, 2010
C, Lok, Babu 2010

25. EM Trackers Data returned: 6 DOF Spatial distortion – 0.6 mm, 0.025°
Resolution – mm, 0.025° / inch from receiver
Jitter (precision) – mm to cm
Drift - none
Lag – reported 4 ms
Update Rate Hz
Range - 5 ft
Number of Tracked Points – 16 (divides update rate)
Wireless - yes
Interference and noise – metal, earth
Mass, Inertia and Encumbrance - minimal
Durability - high
Price - $4000+
Pros:
Measure position and orientation in 3D space
Does not require direct line of sight
Low encumbrance
Cost
Good performance close to emitter
Lag
Can be built ‘into’ devices
Earth magnetic field good for 3DOF
Cons:
Accuracy affected by
DC: Ferrous metal and electromagnetic fields.
AC: Metal and electromagnetic fields
Operate on only one side of the source (the working hemisphere)
Low range (effectively 5’ – does not scale well)
Calibration
Oct, 2010
C, Lok, Babu 2010

26. EM Tracking Ascension Flock of Birds Polhemus Fastrak
Extremely popular
Good for many applications
CAVEs (remove metal)
HMDs
Projection displays
Fishtank
Oct, 2010
C, Lok, Babu 2010

27. Acoustic/Ultrasonic Tracking
Time of Flight Tracking
Emitters
Multiple emitters
In succession, emit sound (record time)
Receiver
Report time of receiving sound
Frequency tuned
Calculate time-of-flight (1000 feet/sec)
Use ultrasonic (high) frequencies
Similar:
EM tracking
Radar/sonar
Phase Coherence tracking
Orientation only
Check phase of received signal
Oct, 2010
C, Lok, Babu 2010

28. Ultrasonic Tracking System Setup
How much data does 1 transmitter provide?
How much data do 2 transmitters provide?
How much data do 3 transmitters provide?
Stationary Origin
(receivers)
Tracker
(transmitters)
distance1
distance2
distance3
Oct, 2010
C, Lok, Babu 2010

29. Acoustic/Ultrasonic Tracking Characteristics
Data returned: 3 or 6 DOF
Spatial distortion – low (good accuracy)
Resolution – good
Jitter (precision) – mm to cm
Drift - none
Lag – very slow
Update Rate Hz
Range – 40’+ (scaling issues)
Number of Tracked Points – numerous (spread-spectrum)
Wireless - yes
Interference and noise – medium, noise, environment
Mass, Inertia and Encumbrance - minimal
Durability - high
Price – cheap to $12000+
Pros:
Inexpensive
Wide area
Encumbrance
Cons
Inaccurate
Interference
Requires line-of-sight
Oct, 2010
C, Lok, Babu 2010

30. Ultrasonic Tracking Devices
Logitech
Mattel Power Glove
Intersense
Used as part of hybrid systems
Oct, 2010
C, Lok, Babu 2010

31. Inertial Tracking Electromechanical devices
Detect the relative motion of sensors
Measuring change:
Acceleration (accelerometers)
Gyroscopic forces (electronic gyroscopes piezo electric)
Inclination (inclinometer)
Frameless tracking
Known start
Each reading updates current position
Oct, 2010
C, Lok, Babu 2010

32. Accelerometers Mounted on to body parts Detects acceleration
Acceleration is integrated to find the velocity
Velocity is integrated to find position
Unencumbered and large area tracking possible
Difficult to ‘factor’ out gravity
Oct, 2010
C, Lok, Babu 2010

33. Accelerometer Tracking Errors
Suppose a constant error i, so that measured acceleration is ai(t)+ i
vi(t) = (ai(t)+ i)dt =  ai(t)dt + it
xi(t) =  vi(t)dt = ( ai(t)dt + t)dt
xi(t) =  ai(t)dtdt + 1/2 it2
Errors accumulate since each position is measured relative to the last position
Estimated 10 degrees per minute. How is this related to drift?
Oct, 2010
C, Lok, Babu 2010

34. Inertial Tracking Inclinometer Gyroscopes Measures inclination
Relative to some “level” position
Gyroscopes
Resist rotation
Measure resistance
Oct, 2010
C, Lok, Babu 2010

35. Inertial Tracking Systems Characteristics
Data returned: 3 or 6 DOF
Spatial distortion – low (good accuracy)
Resolution – good
Jitter (precision) – low
Drift - high
Lag – very low
Update Rate - high
Range – very large
Number of Tracked Points – 1
Wireless - yes
Interference and noise – gravity
Mass, Inertia and Encumbrance - minimal
Durability - high
Price – cheap
Pros:
Inexpensive
Wide area
Orientation very accurate
Minimal interference
Encumbrance
Cons
Position poor
Need to recenter
Calibration
Inaccurate over time
Drift
Oct, 2010
C, Lok, Babu 2010

36. Optical Trackers Use vision based systems to track sensors
Outside-Looking In:
Cameras (typically fixed) in the environment
Track a marked point
PPT tracker from WorldViz (
Older optical trackers
Inside-Looking Out:
Cameras carried by participant
Track makers (typically fixed) in the environment
Intersense Optical Tracker
3rdTech HiBall Tracker
Image from: High-Performance Wide-
Area Optical Tracking The HiBall
Tracking System, Welch, et. al
Oct, 2010
C, Lok, Babu 2010

37. Outside Looking In Optical Tracking
Precision Point Tracking by WorldViz
IR Filtered Cameras are calibrated
Intrinsics
Focal length, Center of projection, aspect ratio
Extrinicis
Position and orientation in world space
Each frame:
Get latest images of point
Generate a ray (in world coordinates) through the point on the image plane
Triangulate to get position
Oct, 2010
C, Lok, Babu 2010

38. Outside Looking In Optical Tracking
What factors play a role in O-L-I tracking?
Camera resolution
Frame rate
Camera calibration
Occlusion
CCD Quality
How does it do for:
Position
stable, very good
Orientation
Unstable, poor
Latency
Cameras are 60Hz
Oct, 2010
C, Lok, Babu 2010

39. Orientation How to compensate for poor orientation? Also known as:
Combine with orientation only sensor (ex. Intersense’s InertiaCube)
Also known as:
‘Hybrid tracker’
‘Multi-modal tracker’
Position: vision
Orientation: inertial
Oct, 2010
C, Lok, Babu 2010

40. Inside-Looking-Out Optical Tracking
Tracking device carries the camera
Tracks markers in the environment
Intersense Tracker
3rdTech HiBall Tracker
Images from: High-Performance Wide-
Area Optical Tracking The HiBall
Tracking System, Welch, et. al
Oct, 2010
C, Lok, Babu 2010

41. HiBall Tracker Position Orientation Latency Pretty good Very good
LEPDs can operate at 1500 Hz
Six Lateral Effect Photo
Dioides (LEPDs) in
HiBall. Think 6 cameras.
Oct, 2010
C, Lok, Babu 2010

42. LED Optical Trackers Sensors Track Why LEDs? Super cheap P5 Glove
Webcameras
Photodiodes
Track
LEDs
Reflected LED light
Why LEDs?
Easy to track
Grab your webcam and point a remote at it
Super cheap
P5 Glove
Nintendo Wii
WorldViz PPT
Virtual Patients
Oct, 2010
C, Lok, Babu 2010

43. Optical Tracking Review
Data returned: 6 DOF
Spatial distortion – very low (very good accuracy)
Resolution – moderate to good
Jitter (precision) – decent
Drift - none
Lag – moderate
Update Rate – low - high
Range – very large (40’ x 40’ +)
Number of Tracked Points – 4
Wireless - yes
Interference and noise – occlusion
Mass, Inertia and Encumbrance - moderate
Durability – low - high
Price – cheap to very expensive
Pros:
Inexpensive
Wide area
Very accurate
Cons
High quality is very expensive
Occlusion
Calibration
Oct, 2010
C, Lok, Babu 2010

44. Angle Measurement
Measurement of the bend of various joints in the user’s body
Used for:
Reconstruction of the position of various body parts (hand, torso).
Measurement of the motion of the human body (medical)
Gestural Interfaces
Sign language
Oct, 2010
C, Lok, Babu 2010

45. Angle Measurement Technology
Optical Sensors
Emitter and receiver on ends of sensor
As sensor is bent, the amount of light from emitter to receiver is attenuated
Attenuation is determined by bend angle
Examples: Flexible hollow tubes, optical fibers
VPL Data Glove
Oct, 2010
C, Lok, Babu 2010

46. Angle Measurement Technology (cont.)
Strain Sensors
Measure the mechanical strain as the sensor is bent.
May be mechanical or electrical in nature.
P5 Glove $25 (!)
Cyberglove (Virtual Technologies)
Oct, 2010
C, Lok, Babu 2010

47. Joints and Cyberglove Sensors
Proximal Inter-
phalangeal Joint (PIP)
Interphalangeal
Joint (IP)
Metacarpophalangeal
Joint (MCP)
Metacarpophalangeal
Joint (MCP)
Abduction Sensors
Thumb Rotation
Sensor
Oct, 2010
C, Lok, Babu 2010

48. Angle Measurement Technology (cont.)
Exoskeletal Structures
Sensors mimic joint structure
Potentiometers or optical encoders in joints report bend
Exos Dexterous Hand Master
Oct, 2010
C, Lok, Babu 2010

49. Other Techniques Pinch Gloves
Have sensor contacts on the ends of each finger
Oct, 2010
C, Lok, Babu 2010

50. Technology Dataglove Monkey Mechanical motion capture High accuracy
Low accuracy
Focused resolution
Monkey
High accuracy
High data rate
Not realistic motion
No paid actor
Oct, 2010
C, Lok, Babu 2010

51. Technology Exoskeleton + angle sensors Tethered
No identification problem
Real-time
No range limit
Rigid body approximation
Oct, 2010
C, Lok, Babu 2010

52. Body Tracking Technology
Position/Orientation Tracking
Orthogonal Electromagnetic Fields
Measurement of Mechanical Linkages
Ultrasonic Signals
Inertial Tracking
Optical Tracking
Inside Looking Out
Outside Looking In
Angle Measurement
Optical Sensors
Strain Sensors
Exoskeletal Structures
Oct, 2010
C, Lok, Babu 2010

53. Recap Tracking Table Focusing on Head and Hand Tracking Data returned:
Magnetic: 6 DOF
Acoustic: 3 DOF per sensor (need 3 to get 6 DOF)
Inertial: 3 DOF
Optical: 6 DOF
Spatial distortion (accuracy)
Magnetic: good close to emitter, degrades quickly
Acoustic: okay close to emitter
Inertial: short time very good, poor due to drift
Optical: okay (webcam) to very good accuracy
Resolution
Inertial: very good
Jitter (precision)
Inertial: low
Optical: outside-looking-in vs inside-looking-out (different types of jitter). Overall pretty good
Oct, 2010
C, Lok, Babu 2010

54. Recap Tracking Table Drift Lag Update Rate Range Magnetic: none
Acoustic: none
Inertial: substantial
Optical: none
Lag
Magnetic: low
Acoustic: moderate
Inertial: low
Optical: low to moderate
Update Rate
Magnetic: good
Acoustic: poor
Inertial: good
Optical: poor to very good
Range
Magnetic: 5’
Acoustic: 15’
Inertial: excellent
Optical: 40’+
Oct, 2010
C, Lok, Babu 2010

55. Recap Tracking Table Number of Tracked Points Wireless
Magnetic: 16
Acoustic: 16
Inertial: 1
Optical: <4
Wireless
Magnetic: yes
Acoustic: yes
Inertial: yes
Optical: yes
Interference and noise
Magnetic: metal, Earth
Acoustic: environment, occlusion
Inertial: none
Optical: occlusion
Oct, 2010
C, Lok, Babu 2010

56. Recap Tracking Table Mass, Inertia and Encumbrance Durability Price
Magnetic: low
Acoustic: low
Inertial: low
Optical: low to high
Durability
Magnetic: high
Acoustic: high
Inertial: high
Optical: low
Price
Magnetic: $4000+
Acoustic: $4000++
Inertial: very cheap
Optical: cheap (wecams) - $180k (motion capture systems)
Oct, 2010
C, Lok, Babu 2010