1. 4 November 2000Bridging the Gap Workshop 1 Control of avatar gestures Francesca Barrientos fbarr@cs.berkeley.edu Computer Science Division UC Berkeley

2. 4 November 2000Bridging the Gap Workshop2 Communicating via desktop VR Communicate verbally using speech Communicate visually using avatars Body pose Body movement

3. 4 November 2000Bridging the Gap Workshop3 Project goals Build system for controlling avatar gestures Simple input device such as a pen and tablet Understand expression using avatar body How do we express ourselves using avatar bodies? How do we evaluate a user’s sense of expressiveness?

4. 4 November 2000Bridging the Gap Workshop4 Control problems Limited input and complex output Control interface divides user’s attention

5. 4 November 2000Bridging the Gap Workshop5 Nonverbal behaviors Displays Gesture Facial expression Gaze Posture Functions Augment/replace speech Regulate conversation Indicate emotion Cognition Awareness Conscious control

6. 4 November 2000Bridging the Gap Workshop6 Other approaches Expression selection Automatic animation Natural language inference Nivi waves to Harry.)

7. 4 November 2000Bridging the Gap Workshop7 Gesticulation Co-occurs with speech Free form and made up in the moment Meaning derived from combination of words and gestures

8. 4 November 2000Bridging the Gap Workshop8 Demo of Doodler

9. 4 November 2000Bridging the Gap Workshop9 Doodler: things that worked Within a small range, tracking is intuitive Can produce free form gestures Movement seems expressive Control is transparent No hunting for controls

10. 4 November 2000Bridging the Gap Workshop10 Doodler: limitations Hard to form specific shapes Need a way to switch between different mappings Hard to do gestures which involve different body parts serially Lack of avatar proprioception

11. 4 November 2000Bridging the Gap Workshop11 Speech independent behaviors Emblematic gesture Replace words Symbolic Arbitrary Affective expression Emotion State of being

12. 4 November 2000Bridging the Gap Workshop12 Demo with writing Using alphabet

13. 4 November 2000Bridging the Gap Workshop13 Writing as control Discrete symbols + Continuously valued variables Intentional and unconscious

14. 4 November 2000Bridging the Gap Workshop14 Writing as control Elements of discrete and continuous control Letter selects a discrete choice Way it is written selects values for Specify multiple variables Explicit control? Implicit control? Don’t have to look

15. 4 November 2000Bridging the Gap Workshop15 Summary Simple Continuous Implicit Flows Abstract Unwitting Transparent Complex Discrete Explicit Conscious Intentional Symbolic

16. 4 November 2000Bridging the Gap Workshop16 Future work Design and record gesture vocabulary Map variations in writing to variations in gesture animation Figure out how to analyze expressive aspects of control

17. 4 November 2000Bridging the Gap Workshop17 Facing the Gap How to think about expression Computer science Psychology Human-computer interaction Puppetry New culture How to study it

18. 4 November 2000Bridging the Gap Workshop18 Explore design space

19. 4 November 2000Bridging the Gap Workshop19 Two control schemes Continuous control with free form motion Discrete control with more complex motion and multiple parameters

20. 4 November 2000Bridging the Gap Workshop20 Other solutions continued Analysis of text ComicChat uses keywords, acronyms, punctuation, etc. Semi-autonomous behaviors BodyChat by Vilhjálmsson Simple kinematic controls Sliders and similar widgets (e.g.. Slater) Full body motion capture

21. 4 November 2000Bridging the Gap Workshop21 Other solutions for nonverbal communication Discrete choices (menus) of expressions Usually affective (happy, sad, angry…) Usually facial Usually used with chat environments Examples: Emotion wheel in ComicChat Palace Gesture/Mimic panel in Vlnet