1. Overview of the Tablet PC Platform Tablet PC Platform Group Microsoft Corporation

2. Agenda Tablet PC Platform Overview Description of the Object Model Platform Controls and Components New features in the 1.7 platform Development Overview Deployment Overview

3. The Tablet PC Platform Overview Ink Collection (Input) Ink Data and Management Ink Recognition

4. The Tablet PC Platform History V1.0/1.1 API:Ink Collection Ink Data Ink Reco Controls:InkEdit InkPicture SDK:Docs Samples Components:n/a V1.5 Same As V1 Updated Docs Updated Samples PenInputPanelInkDividerV1.7 Same As V1 StylusInput APIs Web Support Same As V1 Web Support Updated Docs Updated Samples Same As 1.5

5. Ink Collection (Input) Digitizer Basics  Very High Resolution and Throughput  Packet Properties  X, Y  Pressure, Angle, Rotation, etc … Using the API to collect Ink  InkCollector  InkOverlay  RealTimeStylus (new in Lonestar)

6. Ink Collection (Input)-Notes Digitizer Basics  Very High Resolution  Very High Throughput  Packet Properties  X, Y  Pressure, Angle, Rotation, etc … Using the API to collect ink  InkCollector  InkOverlay  RealTimeStylus (new in Lonestar)

7. Ink Data and Management Ink is a data type Multiple persistence formats Ink looks good  Vector-based  Bezier-smoothed  Anti-aliased  Customizable drawing attributes Rich, extensible API Copy to / paste from the Clipboard

8. Ink Recognition Interpret pen movements and/or strokes as:  Text (handwriting)  Gestures  Shapes and symbols Current language support Additional language support with Lonestar Synchronous and asynchronous recognition Data structure of reco result and alternates Extensibility architecture

9. Ink Recognition - Notes Interpret pen movements and/or strokes as:  Text (handwriting)  Gestures  Shapes and symbols Current language support Additional language support with Lonestar Synchronous and asynchronous recognition Data structure of reco result and alternates Extensibility architecture

10. Tablet PC Platform SDK Two APIs  Automation (COM) API  Dual-interfaced  IDispatch-based callbacks for events  Managed (.NET) API  Thin layer over the COM API Ink controls  InkEdit  InkPicture Documentation and Samples

11. Object Model Overview InkCollector InkOverlay Ink Strokes Renderer Tablet Stroke Recognizers Recognizer DrawingAttributes RecognizerContext

12. Focus on Key Objects Ink Collection  InkCollector  InkOverlay  Ink collection events Ink Data Management  Ink  Strokes  Stroke  DrawingAttributes  Renderer  Ink data management events Ink Recognition  RecognizerContext  RecognitionResult

13. Ink Collection Objects

14. InkCollector Object Top-level object for collecting ink Attaches to any HWND Handles input from all attached tablets (digitizers) and cursors Intercepts cursor movements CollectionMode property  InkOnly  GestureOnly  InkAndGesture

15. InkOverlay Object Superset of InkCollector EditingMode property  Ink  Delete  Select EraserMode property  StrokeErase  PointErase

16. Ink Collection Events CursorInRange/CursorOutOfRangeNewInAirPacketsCursorDownNewPacketsStrokeGesture SystemGesture events Mouse events

17. Ink Collection

18. Ink Data Objects

19. Ink Object Container for:  Stroke data  Metadata  Contextual awareness info Controls all means of persistence Extensible via ExtendedProperties property

20. Stroke Object Represents a single stroke  Single pen down, pen move, pen up sequence ExtendedProperties – allows applications to access and store custom data DrawingAttributes – controls the rendering of the stroke Rich API  PacketSize, PacketCount, PacketDescription  BezierPoints, PolylineCusps, SelfIntersections

21. Strokes Collection Collection of references to Stroke objects Implements ICollection, IEnumerable Group related strokes together  For filtering – by metadata  For recognition – by setting the RecognizerContext.Strokes property

22. DrawingAttributes Object Controls the appearance of ink  Color, width, transparency  Smoothing, anti-aliasing  Height, width  PenTip: Ball or Rectangle DefaultDrawingAttributes property Can be applied to  Stroke  Strokes collection  Cursor (Pen)

23. Renderer Object Two coordinate systems on a Tablet PC  Device coordinates  Ink coordinates (HIMETRIC) Renderer object  Maps between coordinate systems  Draw methods for rendering  Manipulation methods  Object and View transforms supported  Helper methods: Scale, Move, Rotate

24. Ink Data Events InkAdded fires when:  A Stroke object is added to an Ink object  Erasing in StrokeErase mode InkDeleted fires when:  A Stroke object is deleted from an Ink object  Erasing in StrokeErase mode

25. Clipboard Support Use the Ink API to copy and paste  ClipboardCopy  ClipboardPaste Multiple formats supported  ISF – Ink Serialized Format  HTML  Bitmap, Metafile Applications interacting with the Clipboard can choose the format

26. Ink Data Manipulation

27. Ink Recognition Objects

28. RecognizerContext Object Is the instantiation of a given recognizer Allows recognition of a collection of Strokes  Synchronous (foreground) recognition  Asynchronous (background) recognition  Raises events  Recognition  RecognitionWithAlternates Provides access to the recognizer result and alternates

29. RecognitionResult Object Recognizers contain a set of structures  Each structure called a “lattice”  Copied to RecognitionResult and others TopString property Store on the Strokes collection and add to Ink.CustomStrokes  SetResultOnStrokes method RecognitionAlternates object  GetAlternatesFromSelection method  Access to alternate paths through the lattice

30. Ink Recognition

31. Platform Controls and Components

32. Platform Controls InkEdit  Manipulate ink as text  Intended for text and ink-to-text scenarios  Enables smart forms  Superclass of the RichEdit control  Win32,.NET, ActiveX versions InkPicture  Ink-annotate images (.jpg,.bmp,.png,.gif)  Collect and save ink as ink  Subclass of PictureBox, akin to InkOverlay .NET and ActiveX versions only

33. Platform Components PenInputPanel  Programmatic access to the PIP (v 1.5)  Binds to any hWnd or Control  Basic control of the TIP (Lonestar)  Enable / Disable, etc.  Needed for backward compatibility Divider (InkDivider)  Divides Ink into words, lines, paragraphs and drawings  Transforms angled ink

34. Platform Components-Notes PenInputPanel  Programmatic access to the PIP (v 1.5)  Binds to any hWnd or Control  Basic control of the TIP (Lonestar)  Enable / Disable, etc.  Needed for backward compatibility Divider (InkDivider)  Divides Ink into words, lines, paragraphs and drawings  Transforms angled ink

35. Ink Division and the Divider Component

36. New Features in the 1.7 Platform

37. Tablet PC Input Panel Replaces TIP and PIP No longer docked to top/bottom of screen Character Pad panel OS “just knows” when to show the TIP or a TIP icon Grows as user writes On-the-fly recognition and pre-insertion error correction Context support

38. Tablet PC Input Panel-Notes Replaces TIP and PIP No longer docked to top/bottom of screen Character Pad panel OS “just knows” when to show the TIP or a TIP icon Grows as user writes, if needed On-the-fly recognition and pre-insertion error correction Context support

39. Context What is context?  App-specific info you provide to help the recognizer  Constrains recognition rules/possibilities  Greatly improves recognition accuracy Examples of context information  Factoids: recoContext.Factoid = “(!IS_DATE)”;  Word and phrase lists  Custom input scopes with regular expressions Context-enabled applications in Lonestar  Internet Explorer (Address Bar)  Outlook 2003 (To:, Cc:, Bcc:)

40. Context Features in Lonestar

41. Context for Non-Ink-Enabled Controls SetInputScope API (new WIN32 API)  Low-level programmatic solution  Binaries impacted  White paper available for.NET development Context Tagging Tool  Non-programmatic  Binaries not impacted  Persists input scope / field relationships with XML manifest

42. Context Tagging Tool

43. Web Support Support for Partially Trusted callers  Host ink-enabled Windows Forms controls on Web pages  Ink on a Web page  Send ink to the Web server  Retrieve ink from the Web server No-touch deploy Tablet PC applications  http://www.Fabrikam.com/claims.exe http://www.Fabrikam.com/claims.exe Application examples  Inventory management  Expense reporting  Claims handling

44. Hosting Ink Controls Create the Windows Forms control  Ink-enable with InkOverlay object  Use InkEdit or InkPicture controls Deploy the control library to the Web server Create the Web page  tag to hold the control  Script functions to invoke the control’s exposed members More info: ASP.NET QuickStart Tutorial  “Using Windows Forms Controls in Internet Explorer”

45. Ink in a Web Application

46. StylusInput APIs Direct access to digitizer data stream  Manipulate packet data in real time  Separate Real Time Collection and UI threads  Better performance than InkOverlay, InkEdit, etc. Build custom components  Real Time Ink components  Custom dynamic rendering (e.g., hardware acclerated)  Custom gesture recognition  Disabled regions (WindowInputRectangle property)  Ink Collection components  Custom InkOverlay or InkCollector components

47. RealTimeStylus Plugins

48. Other Improvements Manipulation of ink  Create stroke with full packet description HTTP User-Agent  Now identifies Tablet PC OS and SDK version Enhanced SDK Documentation  Lots of new content, samples, etc…

49. Development Environment Install on non-tablet devices  SDK on Win2K SP3 and above  WinXP Tablet PC Edition OS on non-tablet hardware  Microsoft Virtual PC  Available on MSDN  MSDN: SDK  MSDN Subscriber Downloads: OS Simulate tablet digitizer  External HID digitizer  Mouse (UseMouseForInput property)

50. Deploying your Tablet Application Redistributable merge modules Considerations  Recognition  Ink collection w/o a digitizer  PIP and TIP  No support for the Win9X platforms

51. © 2004 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.