1. Tablet PC Platform Advanced Topics Software Design Engineer/Test Lead toddLand@microsoft.com

2. Topics 1. Tablet Hardware API 2. Overview of the Tablet SDK a. Ink, Stroke and related objects b. Renderer 3. Clipboard Support 4. Ink Recognition 5. Gesture Support

3. Tablet PC Platform Hardware API

4. Examining Tablet Hardware Electromagnetic digitizers High resolution, High throughput There are two types of digitizers Integrated and Non-Integrated Multiple Digitizers are supported The mouse is also represented as a digitizer Digitizers support various capabilities # of stylus buttons, pressure, angle or rotation attributes Ink Stroke PacketData varies between digitizers

5. Tablet Hardware API Tablet Object representation of a digitizer Query for digitizer’s supported properties Get property metrics E.g.: How many levels of pressure are supported? Tablets Returns collection of all connected digitizers Including the mouse

6. Tablet Hardware API Cont. Cursor Represents a stylus Allows for access to stylus properties Inverted, Button state, Name Available in all InkCollector/Overlay events CursorInRange / CursorOut0fRange CursorDown / CursorUp Cursors Collection of Cursor objects

7. Ink Overlay & Ink Collector Main function is to collect ink Ink Collector provides core functionality Ink Overlay is an Ink Collector with the addition of easy to use Editing Modes (Ink, Select, Erase) Contains an Ink object Full set of event notifications

8. InkCollector Top level object for collecting ink Attaches to any HWND Handles input from all attached Tablets (digitizers) and Cursors (pens) Intercepts cursor movements CollectionMode property Ink Only Gesture Only Ink and Gesture

9. InkOverlay Superset of InkCollector Adds Editing Modes Ink – Real-time inking mode Select – Selection mode Delete – Eraser mode (both point and stroke)

10. Ink Collection Events CursorInRange/CursorOutOfRangeNewInAirPacketsCursorDownNewPacketsStrokeGestureSystemGesture Mouse Events

11. Ink Object Primary Data Object in the Tablet Platform Container for: Stroke data Metadata Controls all means of persistence Extensible via ExtendedProperties collection

12. Stroke Object Represents a single stroke Single pen down, pen move, pen up sequence ExtendedProperties Allows applications to access and store custom data DrawingAttributes Controls the rendering of the stroke Rich API PacketData, BezierPoints, PolylineCusps, SelfIntersections

13. Strokes Collections What are Stroke collections? Collections of references to strokes Not the stroke data itself Important so you can pass a subset to the Recognizers Stroke Collections can share strokes Both Stroke collections include Stroke 3 Strokes Stroke (1) Strokes Stroke (2)Stroke (3)Stroke (4)Stroke (5)

14. DrawingAttributes Object Controls the appearance of Ink Color, width, transparency Smoothing, anti-aliasing Height, width Pen Tip: Ball or Rectangle DefaultDrawingAttributes property Specifies the DrawingAttributes for all Cursors Can be applied to Stroke Strokes collection Cursor (Pen)

15. Renderer Object Two coordinate systems on a Tablet PC Device coordinates Ink coordinates (HIMETRIC units) Renderer object Map between coordinate systems Draw methods for rendering Manipulation methods Object and View transforms supported Helper methods: Scale, Move, Rotate

16. Clipboard Support

17. Use the Ink API to copy and paste ClipboardCopyClipboardPaste Multiple formats supported ISF – Ink Serialized Format HTML Bitmap, Metafile Applications interacting with the clipboard can choose the format

18. Ink Recognition Interpret pen movements and/or strokes as Text (Handwriting) Gestures Shapes and symbols Language Support Synchronous and Asynchronous recognition supported Provides a full set of recognition results Extensibility architecture

19. Recognizer Context Object for accessing handwriting recognizers Binds ink data to a particular language for recognition Performs synchronous or asynchronous handwriting recognition Recognition results and alternates returned Attributes used to bias recognition results

20. RecognitionResult Recognizers contain a set of structures Each structure called a “lattice” TopString property TopConfidence property TopAlternate property RecognitionAlternates object GetAlternatesFromSelection method Access to alternate paths through the lattice

21. Gesture Support

22. Gestures Collected through InkCollector, InkOverlay, or RealTimeStylus Two Types: System or Application System Gestures Mimic mouse events Cannot be disabled Application Gestures Can be single or multi-stroke Can be turned on/off individually

23. System Gestures Available Gestures TapDoubleTapRightTapDragRightDragHoldEnterHoverEnterHoverLeave Coordinates are Window Coordinates Can’t be cancelled

24. Application Gestures CollectionMode determines support InkOnlyInkAndGestureGestureOnly Single vs. Multi-Stroke Gestures Inform your users about gesture support Good UI feedback Consistent use (i.e. Scratchout)

25. Gestures Focus: Scenario driven: don’t over useScenario driven: don’t over use

26. The Tablet and Mobile PC Developer Center http://msdn.microsoft.com/mobilepc

27. Tablet PC Developer Center The latest technical articles Downloads Developer resources Regular columns.Newsgroups Featured partners and more……

28. Attend a free chat or web cast http://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asp List of newsgroups http://communities2.microsoft.com/communities/newsgroups/en-us/default.aspx MS Community Sites http://www.microsoft.com/communities/default.mspx Locate Local User Groups and Community Sites http://www.microsoft.com/communities/usergroups/default.mspx http://www.microsoft.com/communities/related/default.mspx

29. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

30. Appendix

31. RealTimeStylus Plug-in Interfaces

32. RealTimeStylus 1. Performance De-couples rendering and collection of ink 2. Flexibility You can control exactly how and when the ink is collected and rendered 3. Extensible You can create your own plug-ins to render and collect ink

33. RealTimeStylus 1. Performance Gives you the opportunity to ensure users see no lag when creating ink Collection of stylus input is moved off the UI thread and is collected asynchronously Provides direct access to stylus input even before it is rendered Queued events allow for separation of Real Time Collection thread and UI thread Minimizes thread usage (From 5 per instance to 1 per process)

34. RealTimeStylus Focus: No lag renderingNo lag rendering

35. RealTimeStylus 2. Flexibility Plug-in based architecture You determine how you render and collect ink based on which plug-ins you choose Plug-ins can be dynamically added, removed or re-ordered

36. RealTimeStylus 3. Extensible Allows for the creation of new Real Time components Custom dynamic rendering, gesture recognition, disabled regions, etc. Customize your ink collection and rendering to suit your need

37. Architecture (Overview) Pen Service RealTimePlugIn (IStylusSyncPlugin) RealTimeStylus queue RealTimePlugIn (IStylusSyncPlugin) “Inking” Thread UI Thread Ink Collecting Object (IStylusAsyncPlugin) storage InkCollector

38. RealTimeStylus Plug-ins Object that implements a synchronous or asynchronous interface SyncPlugin: fast custom rendering AsyncPlugin: ink data collection Can manipulate packet data Read, Modify, Add, or Ignore Tablet OS includes two plug-ins Dynamic Rendering Gesture Recognition SDK includes two sample plug-ins Custom Rendering Constrained Region

39. RealTimeStylus Plugin Focus: Flexibility in rendering inkFlexibility in rendering ink Flexibility in collecting stroke dataFlexibility in collecting stroke data

40. Journal Note Reader

41. Provides Read-only access to Journal Notes Exposes all attributes in Journal files Ink, Images, Title, Stationary, etc. Enables your application to consume common note files JNT file is presented as an XML document Native and Managed versions DownloadDownload as an SDK update Download

42. Journal Reader with Ink Divider Focus: Easy to consume Journal filesEasy to consume Journal files