1. “Is It Within My Reach?” -An Agents Perspective Zhisheng Huang, Anton Eliens, and Cees Visser Vrije University Amsterdam The Netherlands http://wasp.cs.vu.nl/step

2. Outline of This Talk Virtual Agent Platform: DLP+Web3D Scripting Language STEP Reach by Hand: Inverse Kinematics Reach by Body: Decision Making Reach by Moving: Reasoning about Environments Conclusions and Future Work

3. Why DLP Object-oriented parallel Prolog Web3D support TCP/IP support for network communication Java-based. All DLP programs are compiled into Java classes Efficiency for programming Reasonably good performance

4. Implementation STEP has been implemented by using Distributed Logic Programming Language (DLP) STEP (version 1.0) and its authoring tools are available from the STEP website: http://wasp.cs.vu.nl/step

5. FACT : STEP Source Code (1.0)

6. FACT: Web_oriented Web deliverable animation. No need to install any software or tool to run and test STEP scripts. No DLLs are needed to run STEP facial animation engine. Animation and presentation can be created on-the-fly, well suitable for web agents which need interaction with users in real time with reasonably good performance.

7. Reach by Hand Inverse Kinematics

8. Interaction Operators in STEP for Computation High-level interaction operators are used to access the computational capabilities of the meta language in order to find the computation solutions, which involve inverse kinematics.

9. STEP script: Touch script(getABvalue(Agent,position(X0,Y0,Z0),Hand,A,B),Action):- Action = seq([getDvalue(Agent,position(X0,Y0,Z0),Hand, D), get_upperarm_length(Agent,L1), get_forearm_length(Agent,L2), do(D1 is L1 + L2), if_then_else(sign(D1-D)>sign(0.001-D), seq([do(cosine_law(L1,L2,D,A)), do(cosine_law(L1, D, L2, B))]), seq([do(A is 1.57*(1+sign(D-0.001))), do(B is 0.0)]))]).

10. script(getVvalue(Agent,position(X0,Y0,Z0),Hand,V),Action):- Action = seq([get_shoulder_center(Agent,Hand, position(X2,Y2,Z2)), do(direction_vector(position(X2,Y2,Z2),position(X0,Y0,Z0),V))]). script(touch(Agent, position(X0,Y0,Z0),l),Action):- Action = seq([getABvalue(Agent,position(X0,Y0,Z0),l,A,B), do(R1 is 3.14-A), getVvalue(Agent,position(X0,Y0,Z0),l,V), get_arm_vector(Agent,l,V0), do(vector_cross_product(V0,V,vector(X3,Y3,Z3),C)), do(R2 is C-B), par([turn(Agent,l_shoulder,rotation(X3,Y3,Z3,R2),fast), turn(Agent,l_elbow,rotation(X3,Y3,Z3,R1),fast), turn(Agent,l_wrist,rotation(X3,Y3,Z3,-0.5),fast)])]).

11. script(touch_absolutePosition(Agent,position(X1,Y1,Z1),Hand),Acti on):- Action = seq([do(getPosition(Agent,X,Y,Z)), do(getRotation(Agent, X2,Y2,Z2,R)), do(X3 is X1-X), do(Y3 is Y1-Y), do(Z3 is Z1-Z), do(R1 is -R), do(position_rotation(position(X3,Y3,Z3), rotation(X2,Y2,Z2,R1),position(X4,Y4,Z4))), touch(Agent,position(X4,Y4,Z4), Hand)]).

12. Reach by Body Reasoning about Situations to find a proper list of involved joints Use a rule-based decision making model

13. A Simplified Reaching Model If the object is lower than the agent’s knee, then the agent should squat to reach it; If the object is higher than the hip and lower than the head, the agent should turn the torso to reach it; If the object is higher than the head and is not reachable by hand, the agent should jump to reach it; ……

14. Torso Bending At least 4 joints

15. Squat to Reach At least 5 joints

16. Reach by Moving reach with knowldge approach A set of rational postulates for reaching with knowledge Define a preference relation on positions. Agents would move to the most-preferred position to find a path to reach the object

17. Reach with Knowledge

18. Rational Postulates (A)Reachability The agent prefers a reachable position to unreachable position d(p1, object) dmax => pref(p1,p2) (B0) Nearness The agent prefers a position which is closer to the object d(p1, object) pref(p1,p2)

19. (C0) Shortness The agent prefers a position for which the move path is shorter. d(p1,agent) pref(p1, p2) Rational Postulates (continued)

20. (B) (Conditional Nearness) If the object is unreachable, then the agent prefers a position which is closer to the object.\\ d(p1,object)>dmax and d(p2,object)>dmax and d(p1,object) pref( p1, p2). (C) (Conditional Shortness) If the object is reachable, then the agent prefers a position for which the move path is shorter.\\ d(p1,object) pref( p1, p2)

21. Accepted Reachable Distance (ARD) ARD denotes the distance from which the agent can reach the object with an acceptable bending of the torso.

22. Rational Postulates (continued) (D) (Acceptable Nearness) If the object is not acceptably reachable, then the agent prefers a position which is closer to the object.\\ d(p1,object)>ARD and d(p2,object)>ARD and d(p1,object) pref(p1, p2). (E) (Acceptable Shortness) If the object is acceptably reachable, then the agent prefers a position for which the move path is shorter. d(p1,object) pref(p1, p2)

23. Future Work Reach by Hand: More complicated hand shapes for grasp and hold; Reach by Body: more fine-grained models for decision making models and inverse kinematics Reach by Moving: More comprehensive scenarios for planning and reasoning