1 Designing Gestural Interfaces Eduard Martorell A book by Dan Saffer 22/10/2010

2 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

3 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

4 Introducing interactive gestures We’ve entered the era of interactive gestures. Direct vs indirect manipulation. –Direct manipulation: manipulate digital objects without command-line commands. Using the body to control the digital space around us. Gestural interfaces have a much wider range of actions; can take advantage of the hole body. Indirect gesture manipulation.

5 Introducing interactive gestures Mechanics of touchscreens and gestural controllers. –Sensor. –Comparator. –Actuator. Sensors can detect: –Pressure. –Light. –Proximity. –Acoustic. –Motion. Sensor Comparator Actuator Environment measured by alerts drives affects

6 Introducing interactive gestures Reasons to not have a gestural interface: –Heavy data input. –Reliance on the visual. –Reliance on the physical. –Inappropiate for context. Reasons to have a gestural interface: –More natural interactions. –Less visible hardware. –More flexibility. –More fun.

7 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

8 Designing for the human body It’s necessary to understand how the human body moves. Kinesiology. Know the limits of the body. “The more complicated the gesture, the fewer the people who will be able to perform it”. Fingers present a diameter from 16mm to 20mm in adults and are more inaccurate than cursors. “Don’t make similar gestures for different actions”.

9 Designing for the human body “Never put essential information under an interface element that can be touched, it may be hidden by the arm or hand”. Touch targets: Fitt’s Law –Time to reach target is proportional to the distance divided by the size of the object. –Avoid placing targets at edges of screen for gestural interfaces. –Use 1x1cm as minimum target size. Adaptative targets.

10 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

11 Patterns for touchscreens Tap to open/activate. –Simple, natural replacement for clicking with mouse. Tap to select. Drag to move object. Slide to scroll. Spin to scroll.

12 Patterns for touchscreens Pinch to shrink and spred to enlarge. Two fingers to scroll.

13 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

14 Patterns for free-form gestures Proximity activation/deactivation. Move body to activate. Point to select / activate. Wave to activate. Rotate to change state. Shake to change. Tilt to move.

15 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

16 Documenting interactive gestures Documentation is important to get the product efficiently and effectively built. Existing movement notation systems. –Labanotation. –Benesh movement notation. –Eshkol Wachman movement notation. All very complex, seldom used to document gestural interfaces.

17 Documenting interactive gestures Scenarios. –Sketches with words: –Sketches with words: “She touches de blinking area, and the map zooms…” Use cases. Task analysis and flows. Wireframes. –For visual design. Keyframes. –When the gesture is slightly complicated. Gestural modules document. Storyboarding. Animation & movies.

18 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

19 Prototyping interactive gestures Prototypes give you feedback to refine the final product. Initially use low-fidelity prototypes, to focus on the concept, features, gestures, etc. Each prototype should become increasingly refined.

20 Prototyping interactive gestures Low-fidelity prototypes can be made of paper to focus on the concept. For some devices it’s usefull to prototype it’s physical form. It can change the gestures that can be employed to engage with it.

21 Prototyping interactive gestures “The man behind the curtain” can manipulate a low- fidelity prototype into seeming more interactive than it really is. High-fidelity prototypes are expensive and the idea is to design and test as many of the details as possible. A high-fidelity prototype should look like the final product.

22 Prototyping interactive gestures To test the prototype you need a test plan and a moderator script, that make sure your gestures are: –Discoverable (“What would you do if you wanted to do X?”). –Meaningful (“Would you ever do that?”). –Useful (“Are you able to do what you wanted to do?”). –Usable (“Can you do that action?”). 10 test subjects will usually quickly reveal the flaws of your product.

23 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

24 Communicating interactive gestures Communicating interactive gestures means communicating 2 pieces of information: –Presence: there is a gestural system. –Instruction: How to use the system. Three zones of engagement. –Attraction. –Observation. –Interaction. Attraction Observation Interaction

25 Communicating interactive gestures People aren’t accustomed to using interactive gestures. They have fear. An attraction affordance should invite the user to initiate interaction with it.

26 Communicating interactive gestures Methods of communicating interactive gestures. –Written instructions. For simple, unambiguous gestures. –Illustration. More universal than written instructions. –Demonstration. Typically show a loop so the gesture can be imitated. –Use of symbols. Communicate the presence of the system and how to use it. –Simile and metaphor. “Move your hand like you are stirring a pot”.

27 Index Introducing interactive gestures. Designing for the human body. Patterns for touchscreens. Patterns for free-form gestures. Documenting interactive gestures. Prototyping interactive gestures. Communicating interactive gestures. The future of interactive gestures.

28 The future of interactive gestures Widespread adoption in the home and office. Supplanting the desktop metaphore? Specialized applications and products. New input devices. Convergence with other technologies. –Voice recognition, virtual reality… Gesturing with smart objects. Optimized for touch. Better tools for creation.

29 The future of interactive gestures Toward standards. –We’ll need some standards for interactive gestures: turn light on/off, ATM’s and kiosks… –Patents stand in the way of standarization. Gestures cannot be patented. Gestures tied to a specific action can be patented (pinch to shrink). “The best way to predict the future is to invent it” – Alan Kay.

30 The end