1. Advanced Programming for 3D Applications CE00383-3 Bob Hobbs Staffordshire university Introduction to Human Motion Lecture 2

2. 2 General Outline Human Skeleton Human Skeleton Muscle Groups Muscle Groups How Robots simulate humans How Robots simulate humans Kinematics Kinematics Gait Gait Locomotion Locomotion

3. 3 Human Dynamics Users described as participants Users described as participants basic interaction involves control of camera (viewpoint) basic interaction involves control of camera (viewpoint) –exploratory navigation / locomotion –Walk through systems More advanced environment allow interaction More advanced environment allow interaction –Touch, selection, manipulation –referred to as direct manipulation

4. 4 Simulation of Body Body model is the description of the interface Body model is the description of the interface eyes are viual interface, ears are audio interface eyes are viual interface, ears are audio interface geometric description drawn from egocentric point of view geometric description drawn from egocentric point of view description of hand and fingers forms basis of grasping simulation for picking up objects (Boulic 1996) description of hand and fingers forms basis of grasping simulation for picking up objects (Boulic 1996)

5. 5 Simulation of Body - Building the body The more points represneting the body the more realistic the movement The more points represneting the body the more realistic the movement Up to 90 points for motion-capture in animation Up to 90 points for motion-capture in animation Standard for human skeleton (H-Anim 1999) Standard for human skeleton (H-Anim 1999) More typically head, Torso, Both hands More typically head, Torso, Both hands Inferred movement from limited points Inferred movement from limited points Inverse kinematics problem - infinite possibilities of movement in virtual environment, consistent restraint Inverse kinematics problem - infinite possibilities of movement in virtual environment, consistent restraint Elbow position in 4- Tracker system (Badler, 1993) Elbow position in 4- Tracker system (Badler, 1993)

6. 6 H-Anim Humanoid Sacroiliac L MidtarsalL AnkleL KneeL Hip R MidtarsalR AnkleR KneeR Hip L WristL ElbowL Shoulder R WristR ElbowR Shoulder vl5 Skullbase

7. 7 Simulation Of body - tracking the participant Choice of system depends on 5 factors Choice of system depends on 5 factors –accuracy, resolution, range, lag, update rate Many different tracking technologies Many different tracking technologies –Meyer 1992 –frequency and time ultrasonic time-of-flight measurement ultrasonic time-of-flight measurement Pulsed Infra-red Pulsed Infra-red GPS GPS Optical Gyroscopes Optical Gyroscopes Phase difference Phase difference

8. 8 Simulation Of body - tracking the participant Spatial Scan Spatial Scan Outside-in Outside-in Inside-out Inside-out Inertial sensing Inertial sensing –mechanical gyroscope –Accelerometer Mechanical Linkages Mechanical Linkages Direct - Field Sensing Direct - Field Sensing

9. 9 Interaction with virtual Body Limitations mean reliance on metaphors for Limitations mean reliance on metaphors for –object manipulation (grasping and moving) –locomotion (movement) Limitations in haptics mean that restraint on the virtual environment exists Limitations in haptics mean that restraint on the virtual environment exists

10. 10 Muscles http://www.youtube.com/watch?v=T- ozRNVhGVg&feature=PlayList&p=37A3DC 6AF2D7C881&index=5 http://www.youtube.com/watch?v=T- ozRNVhGVg&feature=PlayList&p=37A3DC 6AF2D7C881&index=5 http://www.youtube.com/watch?v=T- ozRNVhGVg&feature=PlayList&p=37A3DC 6AF2D7C881&index=5 http://www.youtube.com/watch?v=T- ozRNVhGVg&feature=PlayList&p=37A3DC 6AF2D7C881&index=5 http://www.youtube.com/watch?v=pbTah 5NVOtU&NR=1 http://www.youtube.com/watch?v=pbTah 5NVOtU&NR=1 http://www.youtube.com/watch?v=pbTah 5NVOtU&NR=1 http://www.youtube.com/watch?v=pbTah 5NVOtU&NR=1

11. 11 11 The Musculotendinous Unit Tendon- spring-like elastic component in series with contractile component (proteins) Tendon- spring-like elastic component in series with contractile component (proteins) Parallel elastic component (epimysium, perimysium, endomysium, sarcolemma) Parallel elastic component (epimysium, perimysium, endomysium, sarcolemma) F x PEC: parallel elastic component CC: contractile component SEC: series elastic component

12. 12 12 II. Mechanics of Muscle Contraction Neural stimulation – impulse Neural stimulation – impulse Mechanical response of a motor unit - twitch Mechanical response of a motor unit - twitch T: twitch or contraction time, time for tension to reach maximum F 0 : constant of a given motor unit Averaged T values Tricep brachii 44.5 ms Soleus 74.0 ms Biceps brachii 52.0 ms Medial Gastrocnemius 79.0 ms Tibialis anterior 58.0 ms

13. 13 13 Summation and tetanic contraction (ms)

14. 14 14 Generation of muscle tetanus 10 Hz 100Hz Note: muscle is controlled by frequency modulation from neural input very important in functional electrical stimulation

15. 15 15 Wave summation & tetanization Critical frequency

16. 16 16 Motor unit recruitment All-or-nothing event 2 ways to increase tension: - Stimulation rate - Recruitment of more motor unit Size principle Smallest m.u. recruited first Largest m.u. last

17. 17 Models Springs Springs Joints Joints Segments Segments Muscles Muscles Robots Springs Screws Metal parts Servos Rubber simple, fast, easy to understand

18. 18 Robotic Basics Have moveable segments Have moveable segments Connected with joints Connected with joints Robots spin wheels and pivot jointed segments with some sort of actuator Robots spin wheels and pivot jointed segments with some sort of actuator Some robots use electric motors and solenoids as actuators some use a hydraulic system and some use a pneumatic system (a system driven by compressed gases). Some robots use electric motors and solenoids as actuators some use a hydraulic system and some use a pneumatic system (a system driven by compressed gases). Robots may use all these actuator types. Robots may use all these actuator types. Robots usually have some sort of sensor Robots usually have some sort of sensor

19. 19 Actuators Electrical current drives actuators controlling individual joints Electrical current drives actuators controlling individual joints Directly to motors or solenoids Directly to motors or solenoids To valves controlling flow of fluids to hydraulic or pneumatic systems To valves controlling flow of fluids to hydraulic or pneumatic systemsvalves

20. 20 Robot arm Simplest sort of robot Simplest sort of robot Typical arm has 7 segments, 6 joints Typical arm has 7 segments, 6 joints 6DOF 6DOF Human arm 7DOF Human arm 7DOF Usually driven by Step Motors Usually driven by Step Motors Main use is in manufacturing Main use is in manufacturing

21. 21 Robot Arm Fitted with end effector Fitted with end effector Usually interchangeable Usually interchangeable Artificial Hand, paint gun, welding rod Artificial Hand, paint gun, welding rod Pressure sensor needed to prevent crushing Pressure sensor needed to prevent crushing Programmed by incremental steps which are then replicated ad infinitum Programmed by incremental steps which are then replicated ad infinitum

22. 22 Step Motor electromagnetic, rotary actuator, that mechanically converts digital pulse inputs to incremental shaft rotation. electromagnetic, rotary actuator, that mechanically converts digital pulse inputs to incremental shaft rotation. The rotation not only has a direct relation to the number of input pulses, but its speed is related to the frequency of the pulses. The rotation not only has a direct relation to the number of input pulses, but its speed is related to the frequency of the pulses.

23. 23 Step Motor Each pulse corresponds to an angular rotation

24. 24 Step Motors Between steps holds position w/o brake or clutch Between steps holds position w/o brake or clutch Can be programmed to move a precise number of steps and then hold position Can be programmed to move a precise number of steps and then hold position Possible to be bi-directional Possible to be bi-directional Rapid acceleration, deceleration and reversal Rapid acceleration, deceleration and reversal cf DC Servo motors cf DC Servo motors

25. 25 Choosing the right motor Basic Types: Basic Types: – Variable Reluctance, – Permanent Magnet, – Hybrid Parameters to be considered Parameters to be considered – Distance to be traversed. – Maximum time allowed for a traverse. – Desired detent (static) accuracy. – Desired dynamic accuracy (overshoot).

26. 26 More parameters Settling time Settling time Required step resolutiong Required step resolutiong System friction System friction System inertia. System inertia. Speed/Torque characteristics of the motor: When selecting a motor/drive, the capacity of the motor must exceed the overall requirements of the load. Speed/Torque characteristics of the motor: When selecting a motor/drive, the capacity of the motor must exceed the overall requirements of the load. Torque-to-inertia Ratio Torque-to-inertia Ratio Torque Margin: Selecting a motor drive that provides at least 50% margin above the minimum required torque is ideal. Torque Margin: Selecting a motor drive that provides at least 50% margin above the minimum required torque is ideal.

27. 27 Frameworks, Chains (or Skeletons) A lot of mechanical objects in the real world consist of solid sections connected by joints A lot of mechanical objects in the real world consist of solid sections connected by joints Obviously robot arm but also Obviously robot arm but also –Creatures such as humans and animals. –Car Suspension –Ropes, string and Chains

28. 28 Frameworks, Chains (or Skeletons) Sections and joints of robot arm are known as a 'chain ‘ Sections and joints of robot arm are known as a 'chain ‘ In creatures could be referred to as a skeleton In creatures could be referred to as a skeleton Moveable sections correspond to bones Moveable sections correspond to bones Attachments between bones are joints. Attachments between bones are joints.

29. 29 Frameworks, Chains (or Skeletons) Motions of chains can be specified in terms of translations and rotations. Motions of chains can be specified in terms of translations and rotations. Forward Kinematics - From the amounts of rotation and bending of each joint in an arm, for example, the position of the hand can be calculated. Forward Kinematics - From the amounts of rotation and bending of each joint in an arm, for example, the position of the hand can be calculated. Inverse Kinematics - If the hand is moved, the rotation and bending of the arm is calculated, in accordance with the length and joint properties of each section of the arm. Inverse Kinematics - If the hand is moved, the rotation and bending of the arm is calculated, in accordance with the length and joint properties of each section of the arm.

30. 30 Joint Translation-Rotation We can use a transform (T) to transform each point relative to the body to a position in world coordinates. We can use a transform (T) to transform each point relative to the body to a position in world coordinates. If we want to model both linear and angular (rotational) motion then we need to use a 4x4 matrix to represent the transform If we want to model both linear and angular (rotational) motion then we need to use a 4x4 matrix to represent the transform

31. 31 What is Inverse Kinematics? Forward Kinematics Forward Kinematics Base End Effector ?

32. 32 What is Inverse Kinematics? Inverse Kinematics Inverse Kinematics Base End Effector

33. 33 What does looks like? ? Base End Effector

34. 34 Solution to Our example Our example Number of equation : 2 Unknown variables : 3 Infinite number of solutions !

35. 35 Redundancy  System DOF > End Effector DOF Our example System DOF = 3 End Effector DOF = 2

36. 36 Redundancy A redundant system has infinite number of solutions A redundant system has infinite number of solutions Human skeleton has 70 DOF Human skeleton has 70 DOF –Ultra-super redundant How to solve highly redundant system? How to solve highly redundant system?

37. 37 Iterative solution Start at end effector Start at end effector Move each joint so that end gets closer to target Move each joint so that end gets closer to target The angle of rotation for each joint is found by taking the dot product of the vectors from the joint to the current point and from the joint to the desired end point. Then taking the arcsin of this dot product. The angle of rotation for each joint is found by taking the dot product of the vectors from the joint to the current point and from the joint to the desired end point. Then taking the arcsin of this dot product. To find the sign of this angle (ie which direction to turn), take the cross product of these vectors and checking the sign of the Z element of the vector. To find the sign of this angle (ie which direction to turn), take the cross product of these vectors and checking the sign of the Z element of the vector.

38. 38 Goal Potential Function “ Distance ” from the end effector to the goal “ Distance ” from the end effector to the goal Function of joint angles : G(  ) Function of joint angles : G(  )

39. 39 Our Example Base End Effector Goal distance

40. 40 Seyfarth et al. (1999) J. Biomech. Dynamics of the long jump k m1m1 m2m2 Nonlinear spring-damper element time (ms) Ground reaction force (N) Energetic losses may increase performance!

41. 41 Joint Structures This allows two nodes to be attached to each other in a flexible way so that forces in the plane of the joint will be transmitted through it, but forces perpendicular to the joint will cause it to bend. This will provide IK like capabilities This allows two nodes to be attached to each other in a flexible way so that forces in the plane of the joint will be transmitted through it, but forces perpendicular to the joint will cause it to bend. This will provide IK like capabilities

42. 42 Types of Joint NameSymbolDOF Revolute jointsR1 Prismatic jointsP1 Helical joints-1 Cylindrical jointsRP2 Spherical joints3R3 Planar jointsRRP3

43. 43 Joint Structures In character animation, only 2 types of joint need to be considered. These are the "revolute" and "prismatic" joints. All other types can be based on these two. In character animation, only 2 types of joint need to be considered. These are the "revolute" and "prismatic" joints. All other types can be based on these two. 1 degree of freedom: 1 degree of freedom: –rotational joint - wheel. –hinge - similar to rotational joint above but with limits to motion (end stops) 2 degrees of freedom 2 degrees of freedom –ball & socket joint

44. 44 Dynamics Forward Dynamics - The movements are calculated from the forces, such as, force = mass * acceleration. Forward Dynamics - The movements are calculated from the forces, such as, force = mass * acceleration. Inverse Dynamics - Constraints are applied which specifies how objects interact, for example, they may be linked by a hinge joint or a ball joint, and from this the forces can be calculated Inverse Dynamics - Constraints are applied which specifies how objects interact, for example, they may be linked by a hinge joint or a ball joint, and from this the forces can be calculated

45. 45 Forward Dynamics 1. If no forces act on a particle, the particle retains its linear momentum. 2. The rate of change of the linear momentum of a particle is equal to the sum of all forces acting on it. 3. When two particles exert forces upon each other, these forces are equal in magnitude and opposite in direction.

46. 46 Forward Dynamics These laws can also be applied to rigid bodies by assuming that the forces are acting on the centre of mass of the object. These laws can also be applied to rigid bodies by assuming that the forces are acting on the centre of mass of the object. Assuming that the mass is constant then the second law becomes: Assuming that the mass is constant then the second law becomes: –force = mass * acceleration Euler extended these laws to include rotation. So there are equivalent laws for rotation such as: Euler extended these laws to include rotation. So there are equivalent laws for rotation such as: –torque = inertia * angular acceleration.

47. 47 What is a robot? Joseph Engelberger, a pioneer in industrial robotics, once remarked "I can't define a robot, but I know one when I see one." Joseph Engelberger, a pioneer in industrial robotics, once remarked "I can't define a robot, but I know one when I see one." Many different machines called robots Many different machines called robots Everybody has a different idea of what constitutes a robot Everybody has a different idea of what constitutes a robot Name from robota – forced labour Name from robota – forced labour

48. 48 What relevance to us? VR models use robotic principles VR models use robotic principles Avatars behave like robots Avatars behave like robots Simulations of robots used to test real robots Simulations of robots used to test real robots May be used to control remote robotics May be used to control remote robotics

49. 49 Virtual Actors: Autonomous or Guided Guided Actors are Slaved to the Motions of a Human Participant Using Body Tracking – Optical, mechanical,... – A.K.A. Avatar Autonomous Actors Are Controlled by Behavior Modeling Programs, and Can Autonomous Actors Are Controlled by Behavior Modeling Programs, and Can - Augment or replace human participants - Serve as surrogate instructors - Act as guides in complex synthetic worlds Hybrid Control Desirable Hybrid Control Desirable - VRLOCO uses interaction to invoke and control locomotion behaviors

50. 50 The Weiss 6-Level Motor Organization Hierarchy Organism Level 6. Motor Behavior 5. Motor Organ System 4. Motor Organ 3. Muscle Group 2. Muscle 1. Motor Unit Neuron Level 3. Muscle Group - Coordinated action of several muscles - Motion at one joint 2. Muscle - Muscle contraction 1. Motor Unit - Neuron + muscle fibers - Twitching, shivering

51. 51 The Weiss 6-Level Motor Organization Hierarchy Organism Level 6. Motor Behavior 5. Motor Organ System 4. Motor Organ 3. Muscle Group 2. Muscle 1. Motor Unit Neuron Level 6. Motor Behavior - Movement of the whole organism - E.G., Goal-directed locomotion - Task manager 5. Motor Organ System - Coordinated action of several limbs - E.g., Walking - Motor programs, skills 4. Motor Organ - Coordinated action of several joints - E.G., Stepping motion of a limb - Local motor programs

52. 52 Motion and Reaction Sensorymotor level Sensorymotor level - Levels 1 - 5 - Peripheral and proprioceptive feedback associated with reflex arcs - Motor programs and reflexes coordinate and control motion - Executes behaviors Reactive level Reactive level – Level 6 and higher - Perception triggers and modulates behavior - Organism responds to environmental stimuli to select and compose behaviors - Selects behaviors

53. 53 Organization of a Virtual Actor Organism Level 6. Motor Behavior 5. Motor Organ System 4. Motor Organ 3. Muscle Group 2. Muscle 1. Motor Unit Neuron Level Level 6 and above Reactive level Levels 1-5 Sensorymotor level

54. 54 Virtual Actor

55. 55

56. 56 Abstraction and Interaction

57. 57 Representation and Abstraction

58. 58

59. 59 Finite State Machines for Walking

60. 60 Control and Abstraction

61. 61 Avatars

62. 62 Kinematic Chains Solid links connected at movable joints Solid links connected at movable joints Fixed end: base Fixed end: base Movable end: tip or end effector Movable end: tip or end effector One degree of freedom (DOF) per joint One degree of freedom (DOF) per joint Open chain: one fixed end, one movable end Open chain: one fixed end, one movable end Closed chain: both ends fixed Closed chain: both ends fixed

63. 63 Forward and Inverse Kinematics

64. 64 Kinematic Redundancy End-effector has 6 DoFs End-effector has 6 DoFs - (x, y, z) position - (,, ) orientation Non-redundant linkage has < = 6 joints (DoFs) Non-redundant linkage has < = 6 joints (DoFs) Redundant linkage has > 6 joints (DoFs) Redundant linkage has > 6 joints (DoFs) - Human arm has 7 DoFs » Shoulder 3 » Elbow 1 » Forearm 1 » Wrist 2 - Redundancy enables multiple solutions

65. 65 Inverse Kinematics (IK) Non-redundant Linkages Non-redundant Linkages - Analytical solutions Redundant Linkages Redundant Linkages - Many techniques » Pseudo-inverse (Jacobian) » Gradient » Others IK Commonly Found in Animation Packages IK Commonly Found in Animation Packages - 3D Studio Max

66. 66 Static Balance

67. 67 Weight Bend: Bend: – Non-weight-bearing motion – Traverse subtree rooted at rotating joint Pivot Pivot – Weight-bearing motion – Traverse entire tree starting at root EXCEPT for subtree rooted at rotating joint Critical Element of Realism Critical Element of Realism – Is the character supported by its legs, or are the legs dangling in space as the character is translated along ?

68. 68 Non-weight-bearing motion – traverse subtree rooted at rotating joint Bend

69. 69 Weight-bearing motion – traverse entire tree starting at root EXCEPT for subtree rooted at rotating joint Pivot

70. 70 Gait Parameters Gait Pattern Gait Pattern – Sequence of lifting and placing feet Gait Cycle Gait Cycle – One repetition Of the gait pattern Period Period – Duration of one gait cycle Relative Phase of Leg I Relative Phase of Leg I – Fraction of gait cycle before leg I is lifted Duty Factor Duty Factor – Fraction of gait cycle period a given leg spends on ground Swing Time Swing Time – Time a leg spends In the air Stance Time Stance Time – Time a leg spends On the ground Stroke Stroke – Distance body travels during a leg's stance time

71. 71 Finite State Machines for Walking

72. 72 Locomotion Tracker has a limited range Tracker has a limited range Must use locomotion metaphor to move greater distances Must use locomotion metaphor to move greater distances Locomotion is on an even plane, virtual terrain may not be even Locomotion is on an even plane, virtual terrain may not be even Collision detection can be employed to raise or lower the participant accordingly Collision detection can be employed to raise or lower the participant accordingly

73. 73 Fly in direction of aim Fly in direction of pointing Fly in direction of gaze Fly in direction of torso Directions of locomotion