1. Untold Stories of Touch, Gesture, & NUI
Joe Fletcher
Design Manager, Microsoft Surface

3. Touch is not good for everything Touch is a computing evolution
It is not an end all solution
Touch is a computing evolution
It is an addition element for users to interact with their content in new and unique ways
Touch is great for specific aspects
New technology has enabled touch systems, which can enhance certain activities

5. Microsoft Surface
Microsoft Surface is a revolutionary computer that responds to natural hand gestures and real-world objects, creating unique new ways for people to collaboratively interact with digital content.

6. Microsoft Surface

8. Vision
Strategy
Design

9. Surface Windows 7 Massive Multi-touch, Multi-user Unique Spot
PC’s, Form Factors, Technology

10. Larger Surface Team

11. Touch is not good for everything Touch is an evolution
Touch is great for specific aspects

12. Natural User Interface
Touch
The act of directly touching your digital content
Gesture
Hand motions to take an action
Manipulation
Alteration or freeform movement of digital content using touch
Natural User Interface
The combination of these and other elements the create a unique way to interact with your digital content

13. Surface Design Principles
History and Ecology
Surface Design Principles
Touch Challenges

14. Surface Design Principles
History and Ecology
Surface Design Principles
Touch Challenges

15. Why Evolve?

16. Command Line Interface (CLI)

17. Graphic User Interface (GUI)

18. Natural User Interface (NUI)

19. Natural User Interface (NUI)

20. Natural User Interface (NUI)

21. Microsoft Surface and NUI
Microsoft Surface computing uses sensing and display technology to infuse everyday surfaces with digital content and is comprised of the following unique attributes:
Direct interaction
Multi-touch
Multi-user
Object recognition

22. NUI
vs.
Web 2.0

23. NUI & Touch should not be a novelty

25. Keyboard Mouse Stylus Single point Multi point Input Ecology Accuracy
Good for
Accuracy
Con
Limited
Mouse
Good for
Moving across the screen efficiently
Con
Indirect
Stylus
Good for
Freeform input
Con
Low accuracy
Need large controls
Single point
Good for
Direct object selection
Con
Not for quick input
Low accuracy
Need of large controls
Mis-hits from fingernails of perceived hit area
Multi point
Good for
Manipulating objects
Social
Con
Not for input
Low accuracy
Need large controls
Support of hands and screen

26. [Partial] Touch Ecology
Multi point
(screen)
Reacts to direct input
Multi Touch
(screen)
Reacts to multiple inputs
Orients UI to the direction of the touch
Spatial recognition of system
NUI
(Gesture & Physical/Digital)
Reacts to multiple inputs
Spatial Recognition - Orients UI to the direction of the touch
Object recognition - Blurring physical and digital

27. Tap is not the new click ≠

28. Context, task, and use define the solution

29. Context, task, and use define the solution
Touch is not a solution in search of a problem

30. Touch is not good for everything Touch is an evolution
Touch is great for specific aspects

31. Surface Design Principles
History and Ecology
Surface Design Principles
Touch Challenges

32. Create an inflection point

33. The start of almost every successful technological innovation is marked by an inflection point, which defines a moment of dramatic change.

34. Companies spend 90% of their energy trying to imitate

35. Microsoft Surface and principle driven design

36. Microsoft Surface and principle driven design
Using history to develop the future

37. CLI
GUI
NUI
Text
Graphics
Objects

38. Recall
Recognition
Intuition
CLI
GUI
NUI
Text
Graphics
Objects

39. CLI GUI NUI Directed Exploratory Contextual Recall Recognition
Intuition
CLI
GUI
NUI
Text
Graphics
Objects

40. CLI GUI NUI High-Low Double Medium Fast Few Directed Exploratory
Contextual
Recall
Recognition
Intuition
CLI
GUI
NUI
Text
Graphics
Objects

41. CLI GUI NUI Disconnected Indirect Unmediated High-Low Double Medium
Fast Few
Directed
Exploratory
Contextual
Recall
Recognition
Intuition
CLI
GUI
NUI
Text
Graphics
Objects

42. CLI GUI NUI Static Responsive Evocative Disconnected Indirect
Unmediated
High-Low
Double Medium
Fast Few
Directed
Exploratory
Contextual
Recall
Recognition
Intuition
CLI
GUI
NUI
Text
Graphics
Objects

43. NUI Principle of Performance Aesthetics
Evocative
Principle of Direct Manipulation
Unmediated
Principle of Scaffolding
Fast Few
Principle of Contextual Environments
Contextual
Principle of the Super Real
Intuition
NUI

44. Microsoft Surface and principle driven design

45. Microsoft Surface and principle driven design
Three Super Principles

46. Social
Multiple simultaneous users

48. Social Seamless Multiple simultaneous users
Digital and Physical combined

49. Seamless

50. Social Seamless Spatial Multiple simultaneous users
Digital and Physical combined
Spatial
Kinesiology

52. Overcome Homeostasis

53. Touch is not good for everything Touch is an evolution
Touch is great for specific aspects

54. Surface Design Principles
History and Ecology
Surface Design Principles
Touch Challenges

55. Let’s become practitioners

56. Unique Spot
PC’s, Form Factors, Technology

57. Challenge
Form Factor & Technology

58. Touch Hardware Ecology

59. Touch Hardware Ecology
Capacitive
Capacitive
Infrared
Vision system
Stylus

60. Touch Hardware Ecology
Capacitive
N-Trig (2-10)
Capacitive
N-trig (2-10)
Infrared
HP (2)
Vision system
Surface (52)
Stylus
Wacom (1)

61. Challenge
Patterns

62. Swipe Left: Go back in history Swipe Up: Go to the top of the page
FireFox
Swipe Left: Go back in history
Swipe Up: Go to the top of the page

63. Swipe Left: Go Forward in History Swipe up: Go to the bottom of a page
Apple iPhone
Swipe Left: Go Forward in History
Swipe up: Go to the bottom of a page

64. Challenge
Orientation

65. Horizontal
Vertical

66. Challenge
Cardio Effect

67. Challenge
Paradigms

68. To button or
not to button

69. Challenge
Getting people to touch

70. Exploration
Instruct

71. Challenge
Visualizing Touch

72. Is it working?

73. Future
Where do we go?

74. Your effort to remain what you are is what limits you Ghost in the Shell

75. Debating the future of touch is useless

76. Creating the future is more interesting

77. Touch is not good for everything Touch is an evolution
Touch is great for specific aspects

78. Take advantage of Microsoft Surface
Available from the Surface Community site
Download SDK Workstation Edition
Learn how to build applications from scratch on Surface by viewing videos on Surface.com
Develop for Surface
Interested in purchasing a Surface? to
Purchase a Surface

79. Future Vision
2019

81. Joe Fletcher joeflet@microsoft.com Design Manager, Microsoft Surface
Las Vegas, Thank you!
Joe Fletcher
Design Manager, Microsoft Surface

82. Please Complete an Evaluation Form Your feedback is important!
MIX 09
11/7/ :00 PM
Please Complete an Evaluation Form Your feedback is important!
Evaluation forms can be found on each chair
Temp Staff at the back of the room have additional evaluation form copies
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.