1. virtual techdays INDIA │ 9-11 February 2011 Developing Windows 7 based Multi-Touch Application Ujjwal Kumar │ Partner technical consultant, Microsoft

2.  Touch Hardware  Key Touch Scenarios  Windows 7 Touch Investments  Touch Application Development Guidance  Gesture  Manipulation and Inertia  Demo  Surface Device  Resources virtual techdays INDIA │ 9-11 February 2011 S E S S I O N A G E N D A

3. …creates new opportunities. – Software that is intuitive for users to walk up & master – Software that is collaborative with multiple users …is not new. – Researchers have been exploring this stuff for decades …is now going mainstream. 1.Hardware: Robust touch-capable hardware from Microsoft and OEMs 1.Dozens of Win7 touch devices now available 2.OS: Windows 7 has great touch support baked in 3.SDKs: Microsoft is making touch easy to leverage 4.You: Developers & designers creating innovative apps that take advantage of the technology virtual techdays INDIA │ 9-11 February 2011 Multi-Touch…

4. Windows Touch is defined as two or more independently traceable touch points. Consumers  Touch interfaces have proven successful  Multi-Touch offers clear differentiation with a high “WOW” factor  Multi-touch enhances on-the-go use and enable new scenarios Hardware  Multi-touch capable machines in market today in a broad set of form factors virtual techdays INDIA │ 9-11 February 2011 Windows Touch is here

5. Navigating and Consuming the Web Playing Casual Games Reading and Sorting Email Viewing Photos 1 2 3 4 5 6 Navigating Files and Using Applications Consuming Music and Video Key Touch Scenarios Are Consumptive

6. Internet Explorer® – Pan and zoom gestures, flick forward/back – “Open link in new tab” gesture – Address bar drag menu Windows® Media Center: – Live gallery panning – On-screen keyboard – Larger transport controls & seek – New Now Playing & details pages Photo Viewer (& Windows Live Photo Gallery) – Multitouch Zoom and rotation XPS Viewer – Stretch zoom centered around gesture – Smart zoom Touch Pack for Windows 7 Windows Investment

7. Users have expectations for how an experience will “Look” and what it should “Do” Direct Manipulation demands more from “Look” and “Do” Touch Pack for Windows 7 – Surface Globe: Immersive, 2-D and 3-D touch world exploration: Zoom in on locations Pinpoint areas of interest Get local information Personalize map views Windows 7 Principle: Be Great at “Look” and “Do”

8. virtual techdays INDIA │ 9-11 February 2011

9. Architectural Overview - 1

10. Architectural Overview - 2

11. Architectural Overview - 3

12. Development Tiers

13. .NET 3.5 Multitouch library Multi-touch for Windows 7 Native Windows SDK.NET 4 WPF Silverlight Silverlight3 Gestures Available via manipulation Raw Touch Manipulation Inertia

14. Windows 7 Touch Gestures

15.  Windows Message and APIs: o WM_GESTURE, WM_GESTURENOTIFY o GESTURECONFIG struct o Set/GetGestureConfig() o GESTUREINFO o GetGestureinfo() virtual techdays INDIA │ 9-11 February 2011 WM_GESTURE

16. case WM_GESTURENOTIFY: { BOOL bResult; GESTURECONFIG gcAll[] = {0,GC_ALLGESTURES,0}; // Enable all gestures bResult = SetGestureConfig(hWnd, 0, 1, gcAll, sizeof(GESTURECONFIG)); } case WM_GESTURE: { GESTUREINFO gi; gi.cbSize = sizeof(gi); bResult = GetGestureInfo((HGESTUREINFO)lParam, &gi); switch (gi.dwID) { case GID_BEGIN: break; case GID_END: break; case GID_ZOOM: //GID_PAN, GID_ROTATE, GID_TWOFINGERTAP, GID_PRESSANDTAP break; } Setting Gestures and WM_GESTURE

17. WM_TOUCH* family of messages – Enables touch-optimized experiences – Provides standard Win32 messages WM_TOUCHDOWN, WM_TOUCHMOVE, WM_TOUCHUP – Semantically similar to mouse messages – Conveys raw touch data to Win32 apps Scenario examples: Finger painting, custom gestures, feeding higher level controls, etc WM_TOUCH

18. Manipulation and Manipulation Container

19. Manipulation Events StartingStartedDelta Inertia Starting DeltaCompleted StartInertia() Touch Down (Initial) Touch Move Touch Up (All) Initialize: Mode Container Pivot Completed()

20. WPF 4 Multi-Touch APIs Overview WPF Controls Styles with Panning Enabled ScrollViewer Panning SupportManipulation EventsTouch EventsExtensible Touch DeviceWin7 Touch Device Surface Touch Device Beta 1Beta 2 Release Candidate Surface V2

21. Silverlight 3 Touch APIs - Listening public partial class MainPage : UserControl { public MainPage() {... // listen to touch events from the system Touch.FrameReported += new TouchFrameEventHandler(OnFrame); } void OnFrame(object sender, TouchFrameEventArgs e) { // enumerate and respond to touch events }

22. Silverlight 3 Touch APIs - Processing void Touch_FrameReported(object sender, TouchFrameEventArgs e) { TouchPointCollection touchPoints = e.GetTouchPoints(LayoutRoot); foreach (TouchPoint tp in touchPoints) { if (tp.Action == TouchAction.Down) { // a new touch has come down } if (tp.Action == TouchAction.Move) { // a previously down touch has moved } if (tp.Action == TouchAction.Up) { // a touch has been removed }

23. Inertia Behaviors ManipulationInertiaStarting EventArgs PropertiesUnits TranslationBehaviorInitialVelocity1/96 th DIP per millisec DesiredDeceleration1/96 th DIP per millisec^2 DesiredDisplacement1/96 th DIP RotationBehaviorInitialVelocityDegrees per millisec DesiredDecelerationDecrees per millisec^2 DesiredRotationDegrees ExpansionBehaviorInitialVelocity1/96 th DIP per millisec InitialRadius1/96 th DIP DesiredDeceleration1/96 th DIP per millisec^2 DesiredExpansion1/96 th DIPS Deceleration and Displacement/Rotation/Expansion mutually exclusive Deceleration – useful to simulate friction Displacement – useful on page flips Default – no inertial movements

24. virtual techdays INDIA │ 9-11 February 2011 DEMO: Raw Touch and Image Manipulation in WPF 4 Ujjwal Kumar │ Partner Technical Consultant, Microsoft

25. Gestures: Touch actions that are one step removed from direct manipulations – Two finger tap – Press and tap – Pinch/stretch (zoom) – Circular motion (rotate) – Single and two-finger panning – Flicks Useful for shortcut-like efficiency – Consistent operations like zoom in, or navigate – Focus on content : to surface extra information or invoke contextual action UI Design Considerations – Consistency Should work in the same way across Windows applications – Keep it simple – users won’t discover or remember too many abstractions Gestures: Consistency is key

26. virtual techdays INDIA │ 9-11 February 2011 VIDEO: Surface Computing

27. Surface is not just multi-touch Windows 7 & WPF 4 are a great baseline for touch Surface creates unique opportunities with specialized HW – Robust top – Horizontal form factor – Infrared camera array

28. Surface: Beyond Multi-Touch Massive multi-touch Multi-user experiences Object recognition Optical engineering

29. Surface: Multi-User Trend: Software continues to reduce the need for face-to-face human interactions Problem: Happiness and creativity depend on social interactions Surface brings people together – Industrial design encourages users to gather – Input capabilities allow simultaneous use – Apps facilitate collaboration / competition

30. Surface: Massive Multi-Touch Surface can efficiently track 50+ simultaneous fingers and objects

31. Surface: Objects Uniquely identify & react to billions of tagged objects Byte Tags 256 unique values Identity Tags >340,282,366,920,938,000,000,000,000,000,000,000,000 unique values Tags include orientation indictors dots

32. Surface: Optics Determine the size and shape of every input See into and project onto physical objects

33. What’s next for Surface hardware? Things we’re exploring: Cheaper Thinner Vertical Mobile Surface

34. virtual techdays INDIA │ 9-11 February 2011 VIDEO: Surface Computing

35. virtual techdays INDIA │ 9-11 February 2011 RESOURCES Software Design – Windows User Experience Interaction Guidelines Windows User Experience Interaction Guidelines – Multi-Touch in Windows 7 Multi-Touch in Windows 7 – Developer Enhancements to Windows Touch and Tablet PC Developer Enhancements to Windows Touch and Tablet PC – (User Perceptions of Response Time) Performance UX Patterns (User Perceptions of Response Time) Performance UX Patterns Developer Resources – Code Magazine: Multi-touch capabilities in Windows 7 Code Magazine: Multi-touch capabilities in Windows 7 – Windows Touch on MSDN Windows Touch on MSDN – Windows Touch Developer Resources Windows Touch Developer Resources – Tablet and Touch SDK Tablet and Touch SDK – Introducing the Microsoft Touch Pack for Windows 7 Introducing the Microsoft Touch Pack for Windows 7

36. virtual techdays THANKS │ 9-11 February 2011 ujjwalk@microsoft.com