1. 1 Ceng 701 - Tracking Gökhan Tekkaya Gürkan Vural Can Eroğul METU, 2008

2. 2 Outline –Overview –Tracking Devices ElectromagneticMechanicalAcousticOpticalInertialCombinations –Eye Tracking –Finger/Hand Tracking

3. 3 Tracking - Overview Purpose: –The primary purpose of tracking is to update the visual display based on the viewers head position and orientation. –Instead of tracking the viewer's eyes directly, we track the position and orientation of the user's head. From this we determine the position and orientation of the two eyes.

4. 4 Tracking - Overview We may also be tracking the user's hand(s), fingers, legs or other interface devices. Want tracking to be as 'invisible' as possible to the user. Want the user to be able to move freely with few encumbrances

5. 5 Tracking - Overview Want to be able to have multiple 'guests' nearby Want to track as many objects as necessary Want to have minimal delay between movement of an object and the detection of the objects new position / orientation (< 50 msec total)‏ Want tracking to be accurate (1mm and 1 degree)‏

6. 6 Outline –Overview –Tracking Devices ElectromagneticMechanicalAcousticOpticalInertialCombinations –Eye Tracking –Finger/Hand Tracking

7. 7 Tracking Devices - Electromagnetic large transmitter and one or more small sensors. transmitter emits an electromagnetic field. sensors report the strength of that field at their location to a computer sensors can be polled specifically by the computer or transmit continuously.

8. 8 Tracking Devices - Electromagnetic uses: –projection based systems, HMDs advantages are: –large tracking volume (approx 10 feet / 3 meters) –no line-of-sight restriction –sensors are small and light –technology has been around for a while disadvantages are: –affected by metal in the nearby area –latency can be high (0.1 seconds) –accuracy is low in large volumes –somewhat expensive

9. 9 Tracking Devices - Electromagnetic Polhemus: –Motion Tracking: LIBERTY LATUS LIBERTY LATUSLIBERTYPATRIOTFASTRAKMINUTEMAN –Scanning: FastSCAN –Eye Tracking: VisionTrak

10. 10 Tracking Devices - Electromagnetic

11. 11 Tracking Devices - Mechanical rigid structures with multiple joints one end is fixed, the other is the object being tracked could be tracking users head, or their hand

12. 12 Tracking Devices - Mechanical physically measure the rotation about joints in the armature to compute position and orientation structure is counter-weighted - movements are slow and smooth Knowing the length of each joint and the rotation at each joint, location and orientation of the end point is easy to compute.

13. 13 Tracking Devices - Mechanical uses: –BOOMs, Phantom advantages are: –low latency –high accuracy –sensors are small and light –technology has been around for a while disadvantages are: –small volume –only track one object at a time

14. 14 Tracking Devices - Acoustic small transmitter and one medium sized sensor each transmitter emits ultrasonic pulses which are received by microphones on the sensor (usually arranged in a triangle)‏ as the pulses will reach the different microphones at slightly different times, the position and orientation of the transmitter can be determined

15. 15 Tracking Devices - Acoustic Transmit acoustic pulse Listen for arrival (or echo)‏ Inside-out or outside-in –Outside-in Measure position Sound wave field, maybe Multipath interference –Inside-out Measure position and orientation Must wait for receipt of signal

16. 16 Tracking Devices - Acoustic advantages are: –Small and lightweight –Not magnetic –Relatively little infrastructure disadvantages are: –Line-of-sight requirement –Acoustic interference (e.g. jingle)‏ –Ambient temperature concerns –PC wave ambiguity –PC error accumulation

17. 17 Tracking Devices - Acoustic Logitech: –6 degrees of freedom –3D and 2D modes –Ultrasonic Technology –High level of ruggedness –PC and Unix compatibility –250 dpi resolution/ 3D mode –400 dpi resolution/ 2D mode

18. 18 Tracking Devices - Optical LEDs or reflective materials are placed on the object to be tracked video cameras at fixed locations capture the scene image processing techniques are used to locate the object

19. 19 Tracking Devices - Optical Inside looking out –beacons/landmarks are fixed in the environment –user-mounted sensors –accurate orientation Outside looking in –sensors are fixed in the environment –user-mounted beacons/landmarks –orientation can be a problem as distance to target grows Hybrid or cascaded –head tracked inside-out from room –hands tracked outside in from head

20. 20 Tracking Devices - Optical A. R. T. system from Germany:

21. 21 Tracking Devices - Inertial Sense rate, integrate once Sense acceleration, integrate twice

22. 22 Tracking Devices - Inertial self-contained gyroscopes / accelerometers used knowing where the object was and its change in position / orientation the device and 'know' where it now is tend to work for limited periods of time then drift.

23. 23 Tracking Devices - Inertial advantages are: –Truly sourceless –Very very fast –Robust disadvantages are: –Bias, scale, and alignment errors –Bias integration results in drift –gravity vector kills you

24. 24 Tracking Devices - Combinations Intersense uses a combination on Acoustic and Inertial. Inertial can deal with fast movements and acoustic keeps the inertial from drifting

25. 25 Outline –Overview –Tracking Devices ElectromagneticMechanicalAcousticOpticalInertialCombinations –Eye Tracking –Finger/Hand Tracking

26. 26 Eye Tracking Eye tracking is the process of measuring either the point of gaze "where we are looking" or the motion of an eye relative to the head.

27. 27 Eye Tracker Eye tracker is a device for measuring eye positions and eye movements Eye movements are typically divided into two episodes: –fixation: the eye gaze pauses in a certain position –saccades: the eye moves to another position The resulting series of fixations and saccades is called a scanpath

28. 28 Scanpath of Eye An example of fixations and saccades over text The typical pattern of eye movements during reading Never move smoothly over still text Most information is made available during a fixation

29. 29 Application Areas of Eye Trackers Psychology (notably psycholinguistics, the visual world paradigm)‏ Cognitive science Marketing research Medical research (neurological diagnosis)‏ Human-computer interaction Vehicle Simulators Communication systems for disabled...

30. 30 Types of Eye Trackers - I Optical - Using video images –the eye position is extracted from video images –a camera focuses on one or both eyes and records their movement as the viewer looks at some kind of stimulus

31. 31 Optical Eye Trackers - I modern eye- trackers use –contrast to locate the center of the pupil –infrared and near- infrared non- collimated light to create a corneal reflection (CR).

32. 32 Optical Eye Trackers - II two general techniques based on the location of the illumination source –Bright Pupil creates greater iris/pupil contrast allowing for more robust eye tracking with all iris pigmentation allows for tracking in lighting conditions ranging from total darkness to very bright –Dark Pupil not effective for tracking outdoors as extraneous IR sources interfere with monitoring

33. 33 Optical Eye Trackers - III use a sampling rate of at least 30 Hz although 50/60 Hz is most common modern video-based eye trackers run at 240, 350 or even 1000/1250 Hz –needed in order to capture the detail of the very rapid eye movements during reading, or during studies of neurology

34. 34 BEWARE The photograph in the next slide can be disturbing !

35. 35 Types of Eye Trackers - II Search coils –known as magnetic eye trackers –uses coils that are embedded into a tightly-fitting contact lens or a rubber ring that adheres to eye

36. 36 Magnetic Eye Trackers Alternating magnetic field with horizontal and vertical components is used to induce electric currents in these coils These currents depend on the eye position –on the angle by which the eye is rotated –thus provide a measure of the eye position

37. 37 Types of Eye Trackers - III Electrooculogram –pairs of electrodes are placed either above and below the eye or to the left and right of the eye –a potential difference occurs between the electrodes

38. 38 Electrooculogram If the eye is moved from the center position towards one electrode, this electrode "sees" the positive side of the retina and the opposite electrode "sees" the negative side of the retina A potential difference occurs between the electrodes Assuming that the resting potential is constant, the recorded potential is a measure for the eye position

39. 39 Setups of Eye Trackers Setups vary greatly –some are head- mounted –some require the head to be stable (for example, with a chin rest)‏ –some function remotely and automatically track the head during motion

40. 40 Sample Devices - eyebox2 attached to store displays or billboards to count how many times viewed by passersby uses video images infrared leds create red eyes from up to 10 meters price $1500

41. 41 Sample Devices - Lexus LS460 the vehicle with the capacity to assess in real-time the visual behavior of the driver automobiles with the ability to monitor drowsiness, inattention, and cognitive engagement

42. 42 Outline –Overview –Tracking Devices ElectromagneticMechanicalAcousticOpticalInertialCombinations –Eye Tracking –Finger/Hand Tracking

43. 43 Finger/Hand Tracking Easily done by mechanical tracking devices like: joystick, haptic devices, touch screen, stylus, gloves :) We do not cover these, subjects because it will be covered in the other presentations. Let's watch a touchscreen you do not have to touch to interact with and it works through glass at 15 cm distance.

44. 44 Finger Tracking Optic Tracking –Background has to be smooth, for better finger recognition. Infra-red Tracking (with Wii, or any infra red camera)‏ –Reflective tape is needed, but works in any environment, easier with better success rate.

45. 45 Finger Tracking Several successful gadgets are used for hand & finger tracking. Optic tracking is a difficult job of pattern recognition. Gets easier with a GREEN GLOVE

46. 46 Minority Report Effect Tiring way for interaction with computers! Mechanical tracking devices (gloves) perform better than optic tracking methods, but optic methods are cheaper. Optic tracking with stereo camera performs far more better than one camera.

47. 47 Face/Head Tracking Popular topic. Face is modelled better than the fingers, but still in the tests, generally solid backgrounds are used. Infra-red tracking needs less CPU, which is critical for the other applications. Optical face tracking software is sold with webcams & now with Playstation 3. Face recognition is still a problem, but face tracking is good enough, even in real-time.

48. 48 Haptic Products Remote surgery using teleoperators Training of minimally invasive procedures Better subject for other presentations. (Force feedback & Interaction device)‏

49. 49 Motion Tracking Beyond the scope of this presentation. Will be presented on May 14 th in Motion Capture subject.

50. 50 Questions? Thank you for listening