Presentation is loading. Please wait.

Presentation is loading. Please wait.

Build advanced touch apps in Windows 8

Similar presentations


Presentation on theme: "Build advanced touch apps in Windows 8"— Presentation transcript:

1 Build advanced touch apps in Windows 8
4/24/2017 9:31 AM APP-186T Build advanced touch apps in Windows 8 Reed Townsend Program Manager Microsoft Corporation © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2 Agenda Windows Runtime Platform for touch, gestures, and devices
Win32 platform for touch, gestures, devices, and more Q&A You’ll leave with examples of how to Add power and flexibility! Go beyond the touch support in the HTML and XAML frameworks. Get down to the metal! Build directly on ICoreWindow if you choose to.

3 Windows 8 is a touch-first OS, and as such, we had to reimagine the touch platform and the input stack that supports it

4 Central Touch Platform Concepts
Unify touch, mouse, and pen into a single pointer input API Express the touch interaction language in the platform “Code for touch, get mouse and pen for free!” Performance, performance, performance!

5 Touch and Gesture Platform:
Windows Runtime

6 Windows Runtime touch APIs let you go beyond what the HTML and XAML frameworks provide by default

7 Touch and Gesture Support in Frameworks
API surface Metro style app using HTML Metro style app using XAML ICoreWindow Windows Runtime Pointer events Pointer events with PointerPoint PointerPoint Gesture events GestureRecognizer Device APIs PointerDevice & Capabilities Targeting Baked into framework and controls TouchHitTesting event * Plus some Win32 touch APIs in the Metro SDK

8 GestureRecognizer Sample
demo GestureRecognizer Sample Built using WinRT and ICoreWindow C++ projection © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9 PointerPoint How to get touch data? Use PointerPoint!
Provides raw touch, mouse, and pen data and is the input for GestureRecognizer Pointer unifies all “pointing” devices into a single, consistent set of interfaces and events Makes it easy to “code for touch and get mouse and pen for free” If you want input-specific UX, there are properties in pointer to support that too

10 Handling Pointer Events
void GestureRecognizerSample::OnPointerPressed( ... ) { // Hit testing: find the object under the pointer ... if (iObject != -1) // Assign the pointer to the object found in hit testing _gestureRecognizers[iObject]->ProcessDownEvent(pointerPoint); } else // No objects found in hit testing. Assign this pointer to background _gestureRecognizers[_objects->Length]->ProcessDownEvent(pointerPoint); © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11 Code for touch, get mouse and pen for free
4/24/2017 9:31 AM demo Code for touch, get mouse and pen for free © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12 Handling Pointer Events
void GestureRecognizerSample::OnPointerPressed( ... ) { // Hit testing: find the object under the pointer ... if (iObject != -1) // Assign the pointer to the object found in hit testing _gestureRecognizers[iObject]->ProcessDownEvent(pointerPoint); } else // No objects found in hit testing. Assign this pointer to background _gestureRecognizers[_objects->Length]->ProcessDownEvent(pointerPoint); © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13 PointerPoint Tiered approach to properties
Basic, common data is on provided as properties on the object directly Extended, input-specific data hangs off the object as a property bag Many properties have smart defaults to make coding against features that may or may not be there easier Can get/set raw HID usages as well

14 PointerPoint – Raw HID Usages
void Sample::OnPointerPressed( _In_ Windows::UI::Core::CoreWindow^, _In_ Windows::UI::Core::PointerEventArgs^ args) { Windows::UI::Input::PointerPoint^ pointerPoint = args->CurrentPoint; bool fHasPressure; INT32 vendorValue; // Does the packet have pressure info? pointerPoint->HasUsage(0x0D, 0x30, &fHasPressure); // Get vendor defined usage from HID packet pointerPoint->GetUsageValue(0x06, 0x01, &vendorValue); } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15 demo Gesture Recognition 4/24/2017 9:31 AM
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16 GestureRecognizer Componentized gesture detection based off PointerPoints Allows opting in/out of gestures as well as configuring gestures themselves Allows multiple simultaneous gestures Going beyond frameworks: Specific gestures and configurations Using multiple GestureRecognizers together Detecting multiple gestures within an element, e.g. <canvas/> Building on top of ICoreWindow directly

17 GestureRecognizer: Setup and Configuration
void DrawingObject::Attach(_In_ Windows::UI::Input::GestureRecognizer^ gestureRecognizer) { // Configure gesture recognizer gestureRecognizer->GestureSettings = Windows::UI::Input::GestureSettings::Tap | Windows::UI::Input::GestureSettings::Hold | Windows::UI::Input::GestureSettings::RightTap | Windows::UI::Input::GestureSettings::ManipulationTranslateX | Windows::UI::Input::GestureSettings::ManipulationTranslateY | Windows::UI::Input::GestureSettings::ManipulationTranslateInertia; ... // Register all the gesture event handlers _tokenTapped = gestureRecognizer->Tapped::add( ref new Windows::Foundation::TypedEventHandler< Windows::UI::Input::GestureRecognizer^, Windows::UI::Input::TappedEventArgs^>( this, &DrawingObject::OnTapped)); } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

18 GestureRecognizer: Feeding Pointer Data
void GestureRecognizerSample::OnPointerMoved( _In_ Windows::UI::Core::CoreWindow^, _In_ Windows::UI::Core::PointerEventArgs^ args) { Windows::Foundation::Collections::IVector<Windows::UI::Input::PointerPoint^>^ pointerPoints = args->GetIntermediatePoints(); for (unsigned int i=0; i<_gestureRecognizers->Length; ++i) _gestureRecognizers[i]->ProcessMoveEvents(pointerPoints); } // Render the latest position OnRender(); © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19 GestureRecognizer: Handling an Event
void DrawingObject::OnTapped( _In_ Windows::UI::Input::GestureRecognizer^, _In_ Windows::UI::Input::TappedEventArgs^) { ++_color; if (_color >= Color::MaxCount) _color = Color::First; } _parent->RequestRedraw(false); © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20 GestureRecognizer “Code for touch, get mouse and pen for free” is supported in gesture recognition too RightTap gesture is the set of all interactions that are intended to be “right taps” Mouse right click, touch press and hold, pen press and hold, and pen barrel button tap

21 GestureRecognizer: RightTap Gesture
void DrawingObject::Attach( ... ) { // Configure gesture recognizer gestureRecognizer->GestureSettings = Windows::UI::Input::GestureSettings::Tap | Windows::UI::Input::GestureSettings::Hold | Windows::UI::Input::GestureSettings::RightTap | ... } void DrawingObject::OnRightTapped( _In_ Windows::UI::Input::GestureRecognizer^, _In_ Windows::UI::Input::RightTappedEventArgs^) Initialize(_initColor, _initX, _initY, _initDX, _initDY); _parent->RequestRedraw(false); © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22 Device Support in the Sample
demo Device Support in the Sample Gesture Sample © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23 PointerDevice Query the set of pointer devices on the system and get their properties: Type, max contacts, device rects, etc Get supported HID usages for the device

24 PointerDevice: Device Properties
void BackgroundObject::GetDeviceDescription() { Windows::Devices::Input::PointerDeviceType type = _pointerDevice->PointerDeviceType; boolean external = !_pointerDevice->IsIntegrated; UINT32 maxContacts = _pointerDevice->MaxContacts; Windows::Foundation::Rect rcPhysical = _pointerDevice->PhysicalDeviceRect; Windows::Foundation::Rect rcScreen = _pointerDevice->ScreenRect; // Build device description string ... } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25 PointerDevice: List of HID usages
void Sample::GetDeviceSupportedUsages(UINT 32 pid) { Windows::Devices::Input::PointerDevice device; IVectorView<PointerDeviceUsage> usages; Windows::Devices::Input::PointerDevice::GetPointerDevice(pid, &device); usages = device->SupportedUsages; ... for (int i = 0; i < usages.length; i++) UINT32 HIDUsagePage = usages[i].UsagePage; UINT32 HIDUsage = usages[i].Usage; } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

26 Other Device Goodies Touch/Pen/MouseCapabilities – Get quick access to the system’s input properties

27 Hardware Capabilities Helpers
bool Sample::IsMultiplayerSupported() { PenCapabilities ^pPenCapabilities = ref new PenCapabilities(); bool PenPresent = pPenCapabilities->PenPresent; TouchCapabilities ^pTouchCapabilities = ref new TouchCapabilities(); UINT32 TotalContacts = pTouchCapabilities->Contacts; return PenPresent && (TotalContacts >= 8); } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

28 Targeting Targeting uses touch contact geometry to make smart decisions about what you intended to tap Using ICoreWindow’s TouchHitTesting event, you can control how targeting behaves

29 Targeting: Handling OnTouchHitTesting
void CProgram::OnTouchHitTesting(_In_ Windows::UI::Core::CoreWindow^ /*sender*/, _In_ Windows::UI::Core::TouchHitTestingEventArgs^ touchHitTestingArgs) { // Current and best touch hit testing results Windows::UI::Core::CoreProximityEvaluation bestResult; Windows::UI::Core::CoreProximityEvaluation currentResult; // Initialize best result and best touch hit testing target from input arguments bestResult = touchHitTestingArgs->ProximityEvaluation; bestResult.AdjustedPoint = touchHitTestingArgs->Point; // Use appropriate logic to find best target – see ShapesPuzzle sample ... // Set best target touchHitTestingArgs->ProximityEvaluation = bestResult; // Event is handled touchHitTestingArgs->Handled = true; } © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

30 Touch and Gesture Platform:
Win32

31 For desktop developers, Win32 support mirrors Windows Runtime Platform support and lets you build great touch apps

32 WM_POINTER, InteractionContexts, and Devices
Win32 representation of the pointer concept Can be used in either a per-pointer or frame-based manner InteractionContext Mirrors GestureRecognizer and provides componentized gesture recognition Pointer device APIs Query the set of pointer devices on the system and get their properties, supported HID usages, etc Register for device arrival/departure notifications* * Available in the Metro SDK as well for Metro style apps

33 Other Win32 Goodies Touch targeting Touch visual feedback
Framework or custom control developers can influence touch targeting Touch visual feedback Configure touch contact and gesture feedback Touch injection Lets you inject touch input similarly to SendInput() Input source identification Helps identify source devices (e.g. touch) when relying on legacy mouse messages

34 Q&A

35 Recap Start building Metro style apps with HTML and XAML!
When you need richer, more flexible control of touch, mix in the Windows Runtime touch and gesture APIs If you want to get down to the metal, use ICoreWindow and the Windows Runtime touch and gesture APIs

36 Related sessions [APP-185T] Make great Metro style apps that are touch-optimized using HTML5 [APP-503T] Make great touch apps using XAML [APP-162T] Building high performance Metro style apps using HTML5 [PLAT-754T] From touch to gamepads: master player input in your Metro style game [PLAT-874T] Lap around the Windows Runtime [APP-391T] Designing Metro style apps that are touch-optimized

37 Further reading and documentation
SDK samples: GestureRecognizer, GestureRecognizerSample, ShapesPuzzle Docs: Contact info –

38 thank you Feedback and questions http://forums.dev.windows.com
Session feedback

39 4/24/2017 9:31 AM © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

40 Windows Core OS Services
Metro style Apps Desktop Apps View XAML HTML / CSS HTML JavaScript Model Controller C C++ C# VB JavaScript (Chakra) C C++ C# VB WinRT APIs Communication & Data Graphics & Media Devices & Printing System Services Application Model Internet Explorer Win32 .NET / SL Core Windows Core OS Services

41 The Windows Runtime The new platform for building Windows Metro style apps Bring forward the full power of Windows A family of integrated APIs with great tools Build upon that a new foundation for the future Clean & simple Flexible markup (HTML or XAML) Flexible languages (JavaScript, C++, C#/VB, …)

42 WM_POINTER vs WM_TOUCH
API behavior WM_TOUCH WM_POINTER Exclusivity of gestures and raw data WM_TOUCH and WM_GESTURE are exclusive WM_POINTER can be used in conjunction with TIE Input delays WM_TOUCH has palm rejection delays by default WM_POINTER has no input delays Multiple window support WM_TOUCH can only be delivered to one window. Contacts in other windows are dropped after the initial target window is determined. WM_POINTER messages can be delivered to multiple windows simultaneously. Capture APIs provide additional functionality here as well. Frame vs non-frame WM_TOUCH is frame-based WM_POINTER is non-frame-based by default, but developers can consume frames if they choose. Rich data WM_TOUCH contains basic contact geometry (height and width only) WM_POINTER supports more detailed contact geometry data as well as other richer information in the associated structs such as TOUCH_INFO Tying touch to mouse data WM_TOUCH always promotes the primary contact to mouse WM_POINTER does no mouse promotion by default. WM_POINTER has a separate method for controlling activation. Capture support WM_TOUCH supports only it’s implicit capture model on a single window WM_POINTER provides a capture API for controlling how capture works for contacts and windows Activation support WM_TOUCH relies on the mouse messages it promotes to control activation WM_POINTER provides an activation API to control how activation works for a window Pen and other pointing device input WM_TOUCH does not include pen input WM_POINTER can carry any pointing device’s input

43 WM_POINTER Win32 representation of the pointer concept
Unifies pointing inputs into a single API Provides raw pointer data and is the basis for the platform Can be used in either a per-pointer or frame-based manner Retrieve pointer data with GetPointerInfo() and related functions Part of the massive touch input stack work for Windows 8

44 InteractionContext Mirrors WinRT GestureRecognizer
Componentized gesture detection based off pointer data Allows opting in/out of gestures as well as configuring gestures themselves Create as many ICs as you want, configure them, feed them data, and handle event callbacks

45 Pointer Device APIs Query the set of pointer devices on the system and get their properties: Max contacts, device rects, etc Get supported HID usages for the device Register for device arrival/departure notifications* * Available in the Metro SDK as well for Metro style apps

46


Download ppt "Build advanced touch apps in Windows 8"

Similar presentations


Ads by Google