Advanced Programming for 3D Applications CE Bob Hobbs Staffordshire university Application of Motion Capture Lecture 10

Sensing position Tracking

3 xwxw zwzw ywyw xmxm zmzm ymym xmxm zmzm ymym Taking a view of scene (head looking at bird) Camera yvyv zvzv xvxv

4 BCS Lecture 19 th October 2004 Tracking the system Various tracking systems Various tracking systems Polhemus fastrak Polhemus fastrak –Transmitter –Receiver aka sensor One sensor attached to viewpoint One sensor attached to viewpoint Up to 15 separate sensors to track components of the environment Up to 15 separate sensors to track components of the environment Some systems track up to 64 points Some systems track up to 64 points

5 BCS Lecture 19 th October 2004 Sensing Head Position x y transmitter Position sensor HMD

6 BCS Lecture 19 th October 2004 Sensing Head Position x y HMD x,y,z

7 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z

8 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z

9 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, roll

10 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, roll

11 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z

12 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, pitch

13 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, pitch

14 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, pitch

15 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z

16 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, yaw

17 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, yaw

18 BCS Lecture 19 th October 2004 Sensing Head Position x y x,y,z, yaw

19 BCS Lecture 19 th October 2004 Sensing Head Position x y transmitter Position sensor Attached to item ‘virtually’ in scene

20 Position/orientation trackers 3 main ways of recording positions and orientations: magnetic, ultrasonic and optical Magnetic tracking devices most successful. Polhemus 3Space Isotrack and Ascension Birds (Flock of Birds), not perfect but most common. Polhemus 3Space Isotrack and Ascension Birds (Flock of Birds), not perfect but most common. Source generates low frequency magnetic field detected by sensor. Source generates low frequency magnetic field detected by sensor. Second approach generally based on tripod consisting of 3 ultrasonic speakers set in triangular position that emits ultrasonic sound signals from each of 3 transmitters. Second approach generally based on tripod consisting of 3 ultrasonic speakers set in triangular position that emits ultrasonic sound signals from each of 3 transmitters. Optical uses light sources in similar way (InterSense) Optical uses light sources in similar way (InterSense) Eddy effect used to detect orientation, position by grid reference Eddy effect used to detect orientation, position by grid reference

21 Electromagnetic Position Tracking transmitterreceiver driving electronics SP electronics computer position, orientation

22 Altering current (AC) Altering current (AC) (Direct current DC) (Direct current DC) transmitter X antenna transmitter Y antenna transmitter Z antenna receiver X antenna receiver Y antenna receiver Z antenna time T0T0 T1T1 T2T2 T3T3 T0T0 transmitter X antenna transmitter Y antenna transmitter Z antenna receiver X antenna receiver Y antenna receiver Z antenna time T0T0 T1T1 T2T2 T0T0 Electromagnetic Position Tracking

23 Position Tracking Systems Polhemus Inc. ( Polhemus Inc. ( –3Space ISOTRAK (1 sensor) –3Space FASTRAK (many sensors) Ascension Technology Corp. ( Ascension Technology Corp. ( –Flock of Birds –pcBIRD –SpacePad

24 Trackers Calibration Dynamic errors Dynamic errors –caused by external electromagnetic fields –can be corrected by increasing measurements frequency, synchronizing the measurements with the external field source, and filtering Static errors Static errors –caused by the field distortions due to the surrounding metal and external fields –can be corrected via trackers calibration

25 Calibration Table Z X true tracked

26 Calibration Example CAVE, FoB CAVE, FoB 4 feet from the floor 4 feet from the floor 1 foot grid 1 foot grid 4 th order polynomial fit 4 th order polynomial fit

27 Interpolation True space True space Tracked space Tracked space 1 d 8 d V. Kindratenko, A. Bennett, “Evaluation of Rotation Correction Techniques for Electromagnetic Position Tracking Systems”, in Proc. VE 2000, pp

28 Regular Grid in the True Space

29 Interaction with virtual Body Limitations mean reliance on metaphors for Limitations mean reliance on metaphors for –object manipulation (grasping and moving) –locomotion (movement) Limitations in haptics mean that restraint on the virtual environment exists Limitations in haptics mean that restraint on the virtual environment exists

30 Object Manuipulation World Body BObject O Hand HObject P World Body BObject O Hand H Object P Grasping Releasing

31 Object Manipulation Hand posture may not be tracked - makes grasping difficult Hand posture may not be tracked - makes grasping difficult Must establish a point at which union is deemed to have taken place Must establish a point at which union is deemed to have taken place Moved by repositioning in the scene graph Moved by repositioning in the scene graph Robinett and Holloway 1992 Robinett and Holloway 1992

32

33 Sensors in joints detect position Sensors in joints detect position 3D viewer updates 3D viewer updates Robot applies force to joints Robot applies force to joints Force is felt on hand Force is felt on hand

34

35 Phantom

36 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

37 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

38 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

39 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil Apply force

40 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil Apply force Motors lock

41 Phantom working θ1θ1 θ2θ2 θ3θ3 Virtual pencil

42 Motion Database In computer games In computer games –Many short, carefully planned, labeled motion clips –Manual processing

43 Walk CycleStopStart Left Turn Right Turn

44 Jehee Lee, Jinxiang Chai, Paul Reitsma, Jessica Hodgins, and Nancy Pollard, Interactive Control of Avatars Animated with Human Motion Data, submitted. Sketch Interface

45 Motion Data for Rough Terrain

46

47 Comparison to Real Motion

48

49 User Interfaces

50 Choice-based Interface What is available in database ? What is available in database ? –Provided with several options –Select from among available behaviors

51 Jehee Lee, Jinxiang Chai, Paul Reitsma, Jessica Hodgins, and Nancy Pollard, Interactive Control of Avatars Animated with Human Motion Data, submitted.

52

53 Most Probable Paths

54 Silhouette extraction and matching implemented by Jinxiang Chai

55 Database Search 3 sec

56 The Art of Animation Animators need good tools Animators need good tools –Modify, vary, blend, transition, filter, … Motion Database Motion Editing Toolbox Convincing Animation The Art of Animation

57 Challenges in Motion Editing Reusability and flexibility Reusability and flexibility –Motion data is acquired For a specific performer For a specific performer Within a specific environment Within a specific environment In a specific style/mood In a specific style/mood High dimensionality High dimensionality Inherent non-linearity of orientation data Inherent non-linearity of orientation data

58 Walk Limp Turn ? Turn with a Limp

59 Walk Limp Turn Turn with a Limp

60 Analogy Low frequency (Content) Low frequency (Content) Result = Limp + (Turn – Walk) High frequency (Style) High frequency (Style) Result = Turn + (Limp – Walk) WalkTurn Limp Turn with A limp

61 Walk Strut Run

62 Stub toesLimp Stitched

63 Re-sequence

64 Motion Editing through Optimization Constraints Constraints [Witkin & Kass 88] [Cohen 92] [Gleicher 98] –Features to be retained –New features to be accomplished Find a new motion Find a new motion –Satisfy given constraints –Preserve original characteristics

65 Jehee Lee and Sung Yong Shin, A Hierarchical Approach to Interactive Motion Editing for Human-Like Figures, Siggraph 99

66 Motion Representation Motion of articulated characters Motion of articulated characters –Bundle of motion signals –Each signal describe positions / orientations / joint angles

67 Basic Idea Inter-frame relationship Inter-frame relationship –Enforce constraints –By inverse kinematics Inter-frame relationship Inter-frame relationship –Avoid jerkiness –By curve fitting

68 Adaptation to Rough Terrain Jehee Lee and Sung Yong Shin, A Hierarchical Approach to Interactive Motion Editing for Human-Like Figures, Siggraph 99

69 Adaptation to New Characters

70 Character Morphing