1 STEP: a Scripting Language for Embodied Agents Zhisheng Huang, Anton Eliens, and Cees Visser 연세대학교 컴퓨터과학과 로봇 공학 특강 학기 유 지 오

2 Agenda Introduction Principles The Scripting Language STEP –Reference Systems –Primitive Actions and Composite Operators –Dynamic Logic –High-Level Interaction Operators Examples –Walk –Run –Look at ball –Touch Related Work Conclusions

3 Introduction Interaction  multi-modal communicative acts –Verbal –Non-verbal Gestures, postures, facial expressions –Specifying communicative acts is not easy Scripting language STEP (Scripting Technology for Embodied Persona) –Focus on specification and modeling of gestures and postures for 3D web agents –Specification of communicative acts can be separated from the programs  Advantage!! –For H-anim-based humanoids in DLP (based on WWW) –Prolog-like syntax

4 Principles Design principles For External-oriented communicative acts Principle 1: Convenience –Hide geometric difficulties –The authors who have limited knowledge of C.G. can use it in a natural way –Example Turn a left arm forward slowly turn(Agent, left_arm, front, slow) More specifty… Turn(Agent, left_arm,rotation(1,0,0,1.57), 3) –In STEP Uses some natural-language-like terms for references

5 Principle cont’d Principle 2: Compositional Semantics –Specification of composite actions based on existing components –Sequence action seq, parallel action par, repeat action repeat –Examples Turn arms forward slowly par([turn(Agent, left_arm, front, slow), turn(Agent, right_arm, front, slow)]) –In STEP A set of built-in action operators Principle 3: Redefinability –Scripting actions can be defined in terms of other action explicitly –Example Run then kick script(run_then_kick(Agent), Action) :- Action = seq([run(Agent), kick(Agent)]) –In STEP Rule-based specification system Principles

6 Principle cont’d 2 Principle 4: Parameterization –Adapted to be other actions –Can be specified in terms of how they cause changes over time to each individual degree of freedom –Example When define the action “run” with respect to a degree of freedom “tempo” –In STEP Prolog-like syntax Principle 5: Interaction –Should be able to interact with the world, including objects and other agents –In STEP Powerful meta-language Principles

7 Reference Systems Direction reference –Natural-language-like direction reference scheme The Scripting Language STEP Direction reference for humanoid Combination of the directions for left arm x y z

8 Reference Systems cont’d Body reference –According to the H-anim standard –All body joints are contained in a hierarchical structure The Scripting Language STEP Typical joints for humanoid Elbow joint in different situations

9 Reference Systems cont’d 2 Time reference –Same time reference system as VRML –Flexible time reference system based on.. Beat  interval for body movement Tempo  number of beats per minute The Scripting Language STEP

10 Primitive Actions and Composite Operators turn(Agent, BodyPart, Direction, Duration) –Direction : natural-language-like direction or rotation value –Duration : speed name or explicit time specification move(Agent, BodyPart, Direction, Duration) –Direction : natural-language-like direction or increment value turn_body(Agent, Direction, Duration) move_body(Agent, Direction, Duration) Sequence operator: seq([Action 1, …, Action n ]) Parallel operator: par([Action 1, …, Action n ]) Non-deterministic choice operator: choice([Action 1, …, Action n ]) Repeat operator: repeat(Action, T) The Scripting Language STEP

11 STEP and Dynamic Logic STEP is based on dynamic logic –Allows for arbitrary abstractions using the primitive and composition operators A state  the properties at a paticular moment An action  a relation between two states Two sub-languages in dynamic logic –Sub-language for actions: a –Sub-language for states: Ψ, Φ The Scripting Language STEP

12 High-Level Interaction Operators Scripting actions can directly interact with –Internal states of embodied agents –External states of worlds Lower case greek letters Φ, Ψ, Χ denote formulas in the meta language –test(Φ) : if Φ holds then skip, otherwise fail –do(Φ) : make the state Φ true –If_then_else(Φ,action 1,action 2 ) –until(action, Φ) : perform action until Φ holds All temporal relation can be defined in STEP –13 possible temporal relations between two actions Before, meets, overlaps, starts, during, finishes, equals and their inverse relations –Example overlap(A1,A2)=par([A1,seq([duration(A1,T1),do(random(R)), do(N is T1*R),wait(N),A2])]) The Scripting Language STEP

13 Walk Examples script(walk_pose(Agent), Action):- Action=seq( [par( [ turn(Agent,r_shoulder,back_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_shoulder,front_down2,fast), turn(Agent,l_hip,back_down2,fast)] ), par( [turn(Agent,l_shoulder,back_down2,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_shoulder,front_down2,fast), turn(Agent,r_hip,back_down2,fast)] )] ). script(walk_forward_step(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,front,fast)] ). script(walk_forward_step07(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0,0.7),fast)] ). script(walk_forward_step0(Agent,StepLength), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0, StepLength),fast)] ). script(walk_forward(Agent,StepLength,N), Action):- Action=repeat(walk_forward_step0(Agent,StepLength),N). The walk action Walking posture + moving action Walking forward N steps

14 Run Based on the action “walk” Examples script(basic_run_pose(Agent), Action):- Action=par( [turn(Agent,r_elbow,front,fast), turn(Agent,l_elbow,front,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ). Start pose (b) script(run_pose(Agent,N), Action):- Action=seq([basic_run_pose(Agent), repeat(walk_pose(Agent),N)] ). Run action (a) script(basic_run_pose_elbow(Agent,Elbow_Angle), Action):- Action=par( [turn(Agent,r_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ). script(run(Agent,StepLength,N), Action):- Action=seq([basic_run_pose(Agent), walk_forward(Agent,StepLength,N)] ). Run action with specifying Steps

15 Look at Ball Interaction with Virtual Worlds Calculate angle Examples script(turn_to_direction(Object,ScrVector,DestVector), Action):- Action=seq( do(vector_cross_product(SrcVector,DestVector,vector(X,Y,Z),R)), do(setRotation(Object,X,Y,Z,R))] ). script(look_at_position(Agent,X1,_Y1,Z1), Action):- Action=seq( [do(getPosition(Agent,X,_Y,Z)), do(Xdif is X1-X), do(Zdif is Z1-Z), turn_to_direction(Agent,vector(0.0,0.0,1.0), vector(Xdif,0.0,Zdif))] ). script(look_at_ball(Agent,Ball), Action):- Action=seq( [do(getPosition(Ball,X1,Y1,Z1)), look_at_position(Agent,X1,Y1,Z1)] ).

16 Touch An inverse kinematics problem Examples script(getABvalue(Agent,position(X0,Y0,Z0),Hand,A,B), Action):- Action=seq( [ getDvalue(Agent,position(X0,Y0,Z0),Hand,D), get_upperarm_length(Agent,L1), get_forearm_length(Agent,L2), do(D1 is L1+L2), if_then_else(sign(D1-D)>sign(0.001-D), seq( [do(cosine_law(L1,L2,D,A)), do(cosine_law(L1,D,L2,B))] ), seq( [do(A is 1.57*(1+sign(D-0.001))), do(B is 0.0)]))]). α=π and β=0 script(getVvalue(Agent,position(X0,Y0,Z0),Hand,V), Action):- Action=seq( [ get_shoulder_center(Agent,Hand,position(X2,Y2,Z2)), do(direction_vector(position(X2,Y2,Z2),position(X0,Y0,Z0),V))] ).

17 Touch cont’d Examples script(touch(Agent,position(X0,Y0,Z0),l), Action):- Action=seq( [ getABvalue(Agent,position(X0,Y0,Z0),l,A,B), do(R1 is 3.14-A), getVvalue(Agent,position(X0,Y0,Z0),l,D), get_arm_vector(Agent,l,V0), do(vector_cross_product(V0,V,vector(X3,Y3,Z3),c)), do(R2 is C-B), par( [turn (Agent,l_shoulder,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_elbow,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_wrist,rotation(X3,Y3,Z3,-0.5),fast)])]). script(touch_absolutePosition(Agent, postion(X1,Y1,Z1),Hand), Action):- Action=seq( [do(getPosition(Agent,X,Y,Z)), do(getRotation(Agent,X2,Y2,Z2,R)), do(X3 is X1-X), do(Y3 is Y1-Y), do(Z3 is Z1-Z), do(R1 is –R), do(position_rotation(position(X3,Y3,Z3), rotation(X2,Y2,Z2,R1), position(X4,Y4,Z4))), touch(Agent,position(X4,Y4,Z4), Hand)]).

18 Related Work Perlin and Goldberg’s Improv system –A system for scripting interactive actors in virtual worlds –Compare with STEP… STEP is based on the H-anim specification STEP is design to separate the scripting language from the agent architecture Badler et al ‘s PAR –Similar to Supports the specification of complex actions for agents –Different.. PAR uses a special syntax and has many built-in notions

19 Related Work con’d Funge’s CML (Cognitive Modeling Language) –CML is not VRML/X3D and H-anim based –CML is based on first-order predicate logic vs. STEP is based on dynamic logic Prendinger’s MPML, SCREAM –Higher level concepts such as affect and social context

20 Conclusions Proposed the scripting language STEP for embodied agents, in particular for their communicative acts, like gestures and postures Can be extended for other communicative acts like facial expressions or speech