Specifying gestures by example By Dean Rubine Presenting: Julian Ramos

Who is Dean Rubine BSc and MSc from MIT PhD in Computer Science from CMU According to the Mathematics Genealogy Project his PhD advisor was Roger Berry Dannenberg (Still at CMU working at the intersection of CS and Music) Currently the Director of Technology at Spark Investment Management LLC (an employee owned hedge fund sponsor)

Who is Dean Rubine His PhD thesis was titled: The Automatic Recognition of Gestures Thesis committee: Roger B. Dannenberg Dario Giuse Brad Myers William A. S. Buxton

Motivation “The ability to specify objects, an operation, and additional parameters with a single intuitive gesture appeals to both novice and experienced users.”

Motivation *Taken from: CHUNKING AND PHRASING AND THE DESIG OF HUMAN-COMPUTER DIALOGUES

Problem “Gesture-based interfaces have not been extensively researched, partly because they are difficult to create”

Solution

Architecture Follows a model/view/controller which makes it easy to add new gestures

Adding Gestures

Creating a gesture

Gesture features

Gesture detection method This is nice detection rule is obtained after making assumptions, the derivation complete derivation is in page 57 of Rubine’s PhD Thesis The actual method for deriving the weight’s is called Quadratic Discriminant Analysis Many assumptions done like same covariance matrix for all classes, normally distributed features, etc

Gesture detection method The method is super fast to train but … Only if the number of samples is low O(C*N^3) At the time it seems this was way faster than training a neural network Many assumptions done like same covariance matrix for all classes, normally distributed features, etc

Evaluation

Discussion