1. Francesca Barrientos and John Canny
Cursive
A novel interaction technique for controlling expressive avatar gesture
Francesca Barrientos and John Canny
UC Berkeley
pen gestures
> Type of avatar is used in graphics chat worlds
UIST 12 November 2001 , Orlando Florida

2. Avatar worlds Verbal communication Nonverbal communication Chat Spoken
Mechanisms
Change image
Play animation
Gestures have continuous qualities
> How specify continuous qualities
12 November 2001
Cursive

3. Current techniques Specialized widgets Avatar agency
Vlnet expression panel
Select
Modulate
Trigger
3 step process
ComicChat EmotionWheel
Select and modulate
Hunt and click diverts attention
Avatar agency
BodyChat
Driven by chat text
Lacks direct control
Limitations of each method
> Want best of both worlds
12 November 2001
Cursive

4. Expressive and Controllable
Want gesture to be expressive
Range of gestures
Vary quality of gesture
Single step - modeless
Enable co-occuring verbal and nonverbal communication
Solution: pen gesture input
> How apply pen gesture
12 November 2001
Cursive

5. Mapping pen to avatar gesture
Pen gesture carries symbolic and qualitative data
Human/avatar gesture can be symbolic and qualitative
Pen gesture simultaneously selects avatar gesture and modulates multiple expressive qualities
Data
Pen
Avatar
Symbolic
Letter of alphabet
Gesture type
Qualitative
Size
Magnitude
Duration
Speed
Pressure
Effort
Compact
Natural (writing is previously learned skill)
> PROCESS
12 November 2001
Cursive

6. Conceptual model w- s- r- Library Generated offline
Input User writes letter in GUI
Generation Gesture selection and modulation
Animation Gesture performed w- s- r-
See
library
GUI
Gesture generator
Avatar animator
12 November 2001
Cursive

7. Example: modulating size
Writing letter ‘s’ triggers animation
Single gesture
Modulation along single parameter
Writing size mapped to gesture’s size
Closer look
12 November 2001
Cursive

8. Inputs modulate gesture
Extraction
Type library w- s- r- w-
Gesture Type s-
Character
recognizer r-
Inputs modulate gesture
Explain modulate
Feature extractor
Speed .5 Size .3
GUI
Style
parameters
12 November 2001
Cursive

9. Modulation through multilinear interpolation
Joint trajectory I
Rotation angles over time
Gesture type β
Semantic category
Set of 2n prototypes - G
n style parameters
Prototypes represent extremal trajectories
Gesture instance Iβ(u,v )
Vector of joint trajectories
Multilinear interpolation on type produces instance
Speed u
Size v
Iβ(0,1)
Iβ(0,0)
Iβ(1,1)
Iβ(1,0)
Iβ(l,k)
Iβ(0,k)
Iβ(1,k) k l
Actually this is simplified
12 November 2001
Cursive

10. Speed parameter special case
Interpolation along curves of different durations
Time dilation step
Determine duration for interpolated curve
Choose sample rate on interpolated curve
Compress slower curve - sample at proportionately slower rate
Sample faster trajectory at proportionately faster rate
Angle θ
Briefly describe implementation t
12 November 2001
Cursive

11. Architecture - Gesture producer Animator script VRML Gesture commandsw- s- r-
Gesture commands
Animator
Animation
commands
VRML
script
Show mechnisms
12 November 2001
Cursive

12. Communication Cursive communicates with any copies of user’s avatar
visitor host
user host
User logs into Vworld server
Other visitors receive notification
Other visitors request and download avatar copy
Animator opens socket connection to producer
Generator sends gesture commands via socket
Web server
request avatar
Download avatar
open socket
sends commands
Summarize
login
notification
VWorld server
12 November 2001
Cursive

13. Summary Current techniques lack expressive control of avatar gesture
Apply pen gesture to avatar gesture
Modulate recorded gestures using multilinear interpolation
Augmented VRML avatars can drop into multi-user virtual worlds
12 November 2001
Cursive

14. Conclusion Pen gestures control Expressive avatar gesture
Thank Erin.
Expressive avatar gesture
Single action invokes and modulates gesture
Multidimensional modulation
Enables co-occuring verbal and non-verbal communication
12 November 2001
Cursive