Using Collaborative Interactive Objects and Animation to Enable Dynamic Interactions in Collaborative Virtual Environments Pieter Jorissen* Maarten Wijnants, Wim Lamotte

Overview  Interactions in CVEs  Interactive Objects approach  Avatars in CVEs  Creating more flexible avatar interaction  User input  Network setup  Network traffic  Results  Future Work

Collaborative Virtual Environments (CVEs)  Immersive 3D World  Multiple participants (distributed)  Users can interact through an avatar  Navigate  Interact with objects/other users  Avatar represents the user and their interactions  Position/orientation (3D model)  Interactions (animations)  (other: personality, emotion,…)

Interactions in CVEs  Traditional  Navigation  Direct interaction techniques for actor - object interactions  Advanced behavior = case specific  Hard-coded in application  Result:  Not very flexible  Not runtime extensible  Little reusability

Interactions in CVEs

Goal Make CVE worlds more interactive  Generalize CVE interactions  Make interactions independant of the application  Allow new objects to be introduced at any time  Make no distinction between different kinds of object interactions (avatars, objects, AI agents)  Allow every object/avatar to interact with every other object/avatar in the world  Single scheme for all CVE interactions  Keep netwerk traffic as low as possible

Our Approach  Put interaction info and behaviors in the object description  Describe behaviors in parameterized scripts  Put only the general interaction scheme in the application (script handling, communication)  Interaction layer communicates with the objects  Objects can communicate with other objects through links and messages

Our Approach

Interactive Object Description  Object Properties  Parts, transformations, models, id, constraints,…  Interaction Properties  Commands, triggers, interaction zones  Object Behaviors  Scripts  Trigger – script coupling  XML  Easy to read, easy to understand

Interaction Layer

Interactive Worlds  Advantage  Objects/parts are easy to modify (runtime)  Objects/parts are easy to reuse  New objects can be introduced at any time  Interaction information could be used for planning  Information can be used for network optimizations  Disadvantage  More work in the modeling stage  Less programming (more scripting)  High reusability of object description

Avatars in CVEs  Evolution:  Avatar is the users means of interaction  Proper avatar animation increases feeling of immersion

Avatar Animation in CVEs  Skeletal animation + Keyframed actions  Advantages  Low memory use  Computationally inexpensive  Disadvantages  Lack of flexibility (fixed set of animations)  More flexibility => Use inverse kinematics  Very flexible  Computationally more expensive

Adding IK to the CVE avatar  “Arm” movement (grasping, pointing, pushing,...)  Store IK info separate from animated model  IK joint chain  End effector  Joint constraints  Joint reach (bounding box)  Not human avatar specific !!!  Direct control

User Input User Input  Direct 3D input  Keyboard  Slow  Unintuitive  Difficult context switch  No reference  Microscribe-3D  Easy context switch  Intuitive => fast  No force feedback

Network setup

Network Traffic  Startup: get world from server TCP  Updates: UDP + multicast  Keep it as low as possible!!!  Use interactive object information (at server)  Object/part constraints  determine possible moves  Animation  Send only high level description (animID, start, stop)  Animation synchronization not perfect, but unnecessary  Inverse Kinematics  Send only position of end effector  Calculate joints locally (no perfect synch for all joints)

Results  Collaborative World 1 Collaborative World 1  Hand interaction 1 Hand interaction 1  Hand Interaction 2 Hand Interaction 2

Future Work  Adding AI agents  Large scale testing  Coupling to physical simulation engine  Head tracking + Stereo vision  Add force feedback  Usability study

QUESTIONS ?