Continuous control of avatar gesture (with dirty details) Some of you have already seen talks by me where I discussed some of the high level motivation for my research. I know I left a lot of questions about what it is that I actually do when I go into the lab. This talk is a closeup look at what it is I have been doing. I’ll discuss the technologies I am using and what it’s like to use them and some of the problems I have run into. Perhaps instructive to some of you who haven’t chosen your thesis topics yet. Francesca Barrientos GUIR Meeting  22 October 1999

Goals To develop an interface for continuous control of avatar gesture To gain some understanding of the range of gesture necessary for communication and self expression

Gesticulation Gesticulation is gesture that co-occurs with speech Meaning of the utterance is divided up between the words and gestures Gestures derive meaning from timing with respect to speech

Issues Building the continuous control system Design kinematic mappings between the user’s movements and the avatars movements Measuring user’s satisfaction with expressive capabilities

System components User input Output Virtual world and user management Glove Tracker Voice Output Graphics Audio Virtual world and user management

Dirty details Graphics format renderer Humanoid avatar format Input devices Multi-user virtual world manager Audio conferencing technologies Hassles of using 3rd party software and emerging/submerging technologies

Graphics VRML format H-anim VRML browser Web browser to support the VMRL browser H-anim Definition for humanoid model in VRML Joints, joint hierarchies, etc. Nancy is an h-anim model from Y3D

Picture of Nancy

Animation and control Java EAI Java libraries to allows a java applet to manipulate the VRML model within a VRML browser

Input devices 5th Glove Flock of Birds Bending of five fingers Roll and pitch angles Flock of Birds 6 DOF tethered tracker

Multi-user virtual world manager DeepMatrix by Geometrek Pure Java VRML world server Java client runs as browser plugin Uses standard VRML browsers which support EAI

DeepMatrix screen shot

Audio Would like to add audio conferencing capability to DeepMatrix Java Media Framework? MS NetMeeting? Maybe better just to simulate audio for user studies

Quick and dirty survey Asked avatar users their thoughts on gestures and avatars Sent to WEAVE list ----women+ interested in avatar communities One response

Response highlights [a] In which situations do you find [avatar gestures] most useful? Occasionally in general conversation for emphasis, but, as I am [a role playing game] mage as well as an empath in [real life], to enhance the effect of that.

Response highlights [b] Do you find the…gestures enjoyable to use? 1=not at all; 5=extremely 4 What do you like or dislike about using avatar gestures? Don’t like time involved in accessing each one…and in some cases having to send them to other person…

Response highlights [c] Do you think gestures are necessary for communicating in avatar worlds? 1=not at all; 5=extremely 5 Why or why not? Since no other body language so to speak is available, is an important way to tap into that aspect of communication

Response highlights [d] What, if anything, is missing from gestures in avatar worlds? Well, just being able to actually make your av do it…but I do realize how hard that is Needs to be more fluid if you know what I mean…

Measuring effectiveness Idea: borrow from studies such as Berger and Popelka Present twenty sentences to subjects who were to write down what they heard In one condition, sentences accompanied by appropriate gesture In second condition, no gesture

Installation Set up a station at, say, the Exploratorium and have kids communicate using avatars Follows in tradition of anecdotal and observational study of gesture (e.g. Efron 1972, Birdwhistell1970)

The end Of this talk