1. Continuous control of avatar gesture (with dirty details)
Some of you have already seen talks by me where I discussed some of the high level motivation for my research.
I know I left a lot of questions about what it is that I actually do when I go into the lab.
This talk is a closeup look at what it is I have been doing. I’ll discuss the technologies I am using and what it’s like to use them and some of the problems I have run into. Perhaps instructive to some of you who haven’t chosen your thesis topics yet.
Francesca Barrientos
GUIR Meeting  22 October 1999

2. Goals To develop an interface for continuous control of avatar gesture
To gain some understanding of the range of gesture necessary for communication and self expression

3. Gesticulation Gesticulation is gesture that co-occurs with speech
Meaning of the utterance is divided up between the words and gestures
Gestures derive meaning from timing with respect to speech

4. Issues Building the continuous control system
Design kinematic mappings between the user’s movements and the avatars movements
Measuring user’s satisfaction with expressive capabilities

5. System components User input Output Virtual world and user management
Glove
Tracker
Voice
Output
Graphics
Audio
Virtual world and user management

6. Dirty details Graphics format renderer Humanoid avatar format
Input devices
Multi-user virtual world manager
Audio conferencing technologies
Hassles of using 3rd party software and emerging/submerging technologies

7. Graphics VRML format H-anim VRML browser
Web browser to support the VMRL browser
H-anim
Definition for humanoid model in VRML
Joints, joint hierarchies, etc.
Nancy is an h-anim model from Y3D

8. Picture of Nancy

9. Animation and control Java EAI
Java libraries to allows a java applet to manipulate the VRML model within a VRML browser

10. Input devices 5th Glove Flock of Birds Bending of five fingers
Roll and pitch angles
Flock of Birds
6 DOF tethered tracker

11. Multi-user virtual world manager
DeepMatrix by Geometrek
Pure Java VRML world server
Java client runs as browser plugin
Uses standard VRML browsers which support EAI

12. DeepMatrix screen shot

13. Audio Would like to add audio conferencing capability to DeepMatrix
Java Media Framework?
MS NetMeeting?
Maybe better just to simulate audio for user studies

14. Quick and dirty survey
Asked avatar users their thoughts on gestures and avatars
Sent to WEAVE list ----women+ interested in avatar communities
One response

15. Response highlights [a]
In which situations do you find [avatar gestures] most useful?
Occasionally in general conversation for emphasis, but, as I am [a role playing game] mage as well as an empath in [real life], to enhance the effect of that.

16. Response highlights [b]
Do you find the…gestures enjoyable to use? 1=not at all; 5=extremely 4
What do you like or dislike about using avatar gestures?
Don’t like time involved in accessing each one…and in some cases having to send them to other person…

17. Response highlights [c]
Do you think gestures are necessary for communicating in avatar worlds? 1=not at all; 5=extremely 5
Why or why not?
Since no other body language so to speak is available, is an important way to tap into that aspect of communication

18. Response highlights [d]
What, if anything, is missing from gestures in avatar worlds?
Well, just being able to actually make your av do it…but I do realize how hard that is Needs to be more fluid if you know what I mean…

19. Measuring effectiveness
Idea: borrow from studies such as Berger and Popelka
Present twenty sentences to subjects who were to write down what they heard
In one condition, sentences accompanied by appropriate gesture
In second condition, no gesture

20. Installation
Set up a station at, say, the Exploratorium and have kids communicate using avatars
Follows in tradition of anecdotal and observational study of gesture (e.g. Efron 1972, Birdwhistell1970)

21. The end
Of this talk