Francesca Barrientos and John Canny Cursive A novel interaction technique for controlling expressive avatar gesture Francesca Barrientos and John Canny UC Berkeley pen gestures > Type of avatar is used in graphics chat worlds UIST 12 November 2001 , Orlando Florida

Avatar worlds Verbal communication Nonverbal communication Chat Spoken Mechanisms Change image Play animation Gestures have continuous qualities > How specify continuous qualities 12 November 2001 Cursive

Current techniques Specialized widgets Avatar agency Vlnet expression panel Select Modulate Trigger 3 step process ComicChat EmotionWheel Select and modulate Hunt and click diverts attention Avatar agency BodyChat Driven by chat text Lacks direct control Limitations of each method > Want best of both worlds 12 November 2001 Cursive

Expressive and Controllable Want gesture to be expressive Range of gestures Vary quality of gesture Single step - modeless Enable co-occuring verbal and nonverbal communication Solution: pen gesture input > How apply pen gesture 12 November 2001 Cursive

Mapping pen to avatar gesture Pen gesture carries symbolic and qualitative data Human/avatar gesture can be symbolic and qualitative Pen gesture simultaneously selects avatar gesture and modulates multiple expressive qualities Data Pen Avatar Symbolic Letter of alphabet Gesture type Qualitative Size Magnitude Duration Speed Pressure Effort Compact Natural (writing is previously learned skill) > PROCESS 12 November 2001 Cursive

Conceptual model w- s- r- Library Generated offline Input User writes letter in GUI Generation Gesture selection and modulation Animation Gesture performed w- s- r- See library GUI Gesture generator Avatar animator 12 November 2001 Cursive

Example: modulating size Writing letter ‘s’ triggers animation Single gesture Modulation along single parameter Writing size mapped to gesture’s size Closer look 12 November 2001 Cursive

Inputs modulate gesture Extraction Type library w- s- r- w- Gesture Type s- Character recognizer r- Inputs modulate gesture Explain modulate Feature extractor Speed .5 Size .3 GUI Style parameters 12 November 2001 Cursive

Modulation through multilinear interpolation Joint trajectory I Rotation angles over time Gesture type β Semantic category Set of 2n prototypes - G n style parameters Prototypes represent extremal trajectories Gesture instance Iβ(u,v ) Vector of joint trajectories Multilinear interpolation on type produces instance Speed u Size v Iβ(0,1) Iβ(0,0) Iβ(1,1) Iβ(1,0) Iβ(l,k) Iβ(0,k) Iβ(1,k) k l Actually this is simplified 12 November 2001 Cursive

Speed parameter special case Interpolation along curves of different durations Time dilation step Determine duration for interpolated curve Choose sample rate on interpolated curve Compress slower curve - sample at proportionately slower rate Sample faster trajectory at proportionately faster rate Angle θ Briefly describe implementation t 12 November 2001 Cursive

Architecture - Gesture producer Animator script VRML Gesture commands w- s- r- Gesture commands Animator Animation commands VRML script Show mechnisms 12 November 2001 Cursive

Communication Cursive communicates with any copies of user’s avatar visitor host user host User logs into Vworld server Other visitors receive notification Other visitors request and download avatar copy Animator opens socket connection to producer Generator sends gesture commands via socket Web server request avatar Download avatar open socket sends commands Summarize login notification VWorld server 12 November 2001 Cursive

Summary Current techniques lack expressive control of avatar gesture Apply pen gesture to avatar gesture Modulate recorded gestures using multilinear interpolation Augmented VRML avatars can drop into multi-user virtual worlds 12 November 2001 Cursive

Conclusion Pen gestures control Expressive avatar gesture Thank Erin. Expressive avatar gesture Single action invokes and modulates gesture Multidimensional modulation Enables co-occuring verbal and non-verbal communication 12 November 2001 Cursive