Overview of the Tablet PC Overview of the Tablet PC PlatformPlatform

Tablet PC Platform GroupTablet PC Platform GroupMicrosoft CorporationMicrosoft Corporation

AgendaAgenda

Tablet PC Platform OverviewTablet PC Platform Overview

Description of the Object ModelDescription of the Object Model

Platform Controls and ComponentsPlatform Controls and Components

New features in the 1.7 platformNew features in the 1.7 platform

Development OverviewDevelopment Overview

Deployment OverviewDeployment Overview

The Tablet PC Platform The Tablet PC Platform OverviewOverview

Ink Collection (Input)Ink Collection (Input)

Ink Data and ManagementInk Data and Management

Ink RecognitionInk Recognition

The Tablet PC Platform The Tablet PC Platform HistoryHistory

V1.0/1.1V1.0/1.1

API:API: Ink Ink CollectionCollection

Ink DataInk Data

Ink RecoInk Reco

Controls:Controls: InkEditInkEdit

InkPictureInkPicture

SDK:SDK: DocsDocs

SamplesSamples

Components:Components: n/an/a

V1.5V1.5

Same As V1Same As V1

Same As V1Same As V1

Updated DocsUpdated Docs

Updated SamplesUpdated Samples

PenInputPanelPenInputPanel

InkDividerInkDivider

V1.7V1.7

Same As V1Same As V1

StylusInput APIsStylusInput APIs

Web SupportWeb Support

Same As V1Same As V1

Web SupportWeb Support

Updated DocsUpdated Docs

Updated SamplesUpdated Samples

Same As 1.5Same As 1.5

Ink Collection (Input)Ink Collection (Input)Digitizer BasicsDigitizer Basics Very High Resolution and ThroughputVery High Resolution and Throughput Packet PropertiesPacket Properties

X, YX, Y Pressure, Angle, Rotation, etc …Pressure, Angle, Rotation, etc …

Using the API to collect InkUsing the API to collect Ink InkCollectorInkCollector InkOverlayInkOverlay RealTimeStylus (new in Lonestar)RealTimeStylus (new in Lonestar)

Ink Data and ManagementInk Data and ManagementInk is a data typeInk is a data type

Multiple persistence formatsMultiple persistence formats

Ink looks goodInk looks good Vector-basedVector-based Bezier-smoothedBezier-smoothed Anti-aliasedAnti-aliased Customizable drawing attributesCustomizable drawing attributes

Rich, extensible API Rich, extensible API

Copy to / paste from the ClipboardCopy to / paste from the Clipboard

Ink RecognitionInk RecognitionInterpret pen movements and/or strokes as:Interpret pen movements and/or strokes as: Text (handwriting)Text (handwriting) GesturesGestures Shapes and symbolsShapes and symbols

Current language supportCurrent language support

Additional language support with LonestarAdditional language support with Lonestar

Synchronous and asynchronous recognitionSynchronous and asynchronous recognition

Data structure of reco result and alternatesData structure of reco result and alternates

Extensibility architectureExtensibility architecture

Tablet PC Platform SDKTablet PC Platform SDKTwo APIsTwo APIs Automation (COM) APIAutomation (COM) API

Dual-interfacedDual-interfaced IDispatch-based callbacks for eventsIDispatch-based callbacks for events

Managed (.NET) APIManaged (.NET) API Thin layer over the COM APIThin layer over the COM API

Ink controlsInk controls InkEditInkEdit InkPictureInkPicture

Documentation and SamplesDocumentation and Samples

Object Model OverviewObject Model OverviewInkCollector InkOverlay

Ink

Strokes

Renderer

Tablet

Stroke

Recognizers

Recognizer

DrawingAttributes

RecognizerContext

Focus on Key ObjectsFocus on Key ObjectsInk CollectionInk Collection InkCollectorInkCollector InkOverlayInkOverlay Ink collection eventsInk collection events

Ink Data ManagementInk Data Management InkInk StrokesStrokes StrokeStroke DrawingAttributesDrawingAttributes RendererRenderer Ink data management eventsInk data management events

Ink RecognitionInk Recognition RecognizerContextRecognizerContext RecognitionResultRecognitionResult

Ink Collection ObjectsInk Collection Objects

InkCollector ObjectInkCollector ObjectTop-level object for collecting ink Top-level object for collecting ink

Attaches to any HWNDAttaches to any HWND

Handles input from all attached tablets Handles input from all attached tablets (digitizers) and cursors (digitizers) and cursors

Intercepts cursor movements Intercepts cursor movements

CollectionMode propertyCollectionMode property InkOnlyInkOnly GestureOnlyGestureOnly InkAndGestureInkAndGesture

InkOverlay ObjectInkOverlay Object

Superset of InkCollectorSuperset of InkCollector

EditingMode propertyEditingMode property InkInk DeleteDelete SelectSelect

EraserMode propertyEraserMode property StrokeEraseStrokeErase PointErasePointErase

Ink Collection EventsInk Collection EventsCursorInRange/CursorOutOfRangeCursorInRange/CursorOutOfRange

NewInAirPacketsNewInAirPackets

CursorDownCursorDown

NewPacketsNewPackets

StrokeStroke

GestureGesture

SystemGesture eventsSystemGesture events

Mouse eventsMouse events

Ink CollectionInk Collection

Ink Data ObjectsInk Data Objects

Ink ObjectInk Object

Container for:Container for: Stroke dataStroke data MetadataMetadata Contextual awareness infoContextual awareness info

Controls all means of persistenceControls all means of persistence

Extensible via ExtendedProperties Extensible via ExtendedProperties propertyproperty

Stroke ObjectStroke ObjectRepresents a single strokeRepresents a single stroke Single pen down, pen move, pen up Single pen down, pen move, pen up

sequencesequence

ExtendedProperties – allows applications ExtendedProperties – allows applications to access and store custom datato access and store custom data

DrawingAttributes – controls the rendering DrawingAttributes – controls the rendering of the strokeof the stroke

Rich API Rich API PacketSize, PacketCount, PacketDescriptionPacketSize, PacketCount, PacketDescription BezierPoints, PolylineCusps, BezierPoints, PolylineCusps,

SelfIntersectionsSelfIntersections

Strokes CollectionStrokes Collection

Collection of references to Stroke objectsCollection of references to Stroke objects

Implements ICollection, IEnumerableImplements ICollection, IEnumerable

Group related strokes togetherGroup related strokes together For filtering – by metadataFor filtering – by metadata For recognition – by setting the For recognition – by setting the

RecognizerContext.Strokes propertyRecognizerContext.Strokes property

DrawingAttributes ObjectDrawingAttributes ObjectControls the appearance of inkControls the appearance of ink Color, width, transparencyColor, width, transparency Smoothing, anti-aliasingSmoothing, anti-aliasing Height, widthHeight, width PenTip: Ball or RectanglePenTip: Ball or Rectangle

DefaultDrawingAttributes propertyDefaultDrawingAttributes property

Can be applied toCan be applied to StrokeStroke Strokes collectionStrokes collection Cursor (Pen)Cursor (Pen)

Renderer ObjectRenderer ObjectTwo coordinate systems on a Tablet PCTwo coordinate systems on a Tablet PC Device coordinatesDevice coordinates Ink coordinates (HIMETRIC)Ink coordinates (HIMETRIC)

Renderer objectRenderer object Maps between coordinate systemsMaps between coordinate systems Draw methods for renderingDraw methods for rendering Manipulation methodsManipulation methods

Object and View transforms supportedObject and View transforms supported Helper methods: Scale, Move, RotateHelper methods: Scale, Move, Rotate

Ink Data EventsInk Data Events

InkAdded fires when:InkAdded fires when: A Stroke object is added to an Ink objectA Stroke object is added to an Ink object Erasing in StrokeErase modeErasing in StrokeErase mode

InkDeleted fires when:InkDeleted fires when: A Stroke object is deleted from an Ink objectA Stroke object is deleted from an Ink object Erasing in StrokeErase modeErasing in StrokeErase mode

Clipboard SupportClipboard SupportUse the Ink API to copy and pasteUse the Ink API to copy and paste ClipboardCopyClipboardCopy ClipboardPasteClipboardPaste

Multiple formats supportedMultiple formats supported ISF – Ink Serialized FormatISF – Ink Serialized Format HTMLHTML Bitmap, Metafile Bitmap, Metafile

Applications interacting with the Applications interacting with the Clipboard can choose the formatClipboard can choose the format

Ink Data Ink Data ManipulationManipulation

Ink Recognition ObjectsInk Recognition Objects

RecognizerContext ObjectRecognizerContext ObjectIs the instantiation of a given recognizerIs the instantiation of a given recognizer

Allows recognition of a collection of Allows recognition of a collection of StrokesStrokes Synchronous (foreground) recognition Synchronous (foreground) recognition Asynchronous (background) recognitionAsynchronous (background) recognition

Raises eventsRaises events RecognitionRecognition RecognitionWithAlternatesRecognitionWithAlternates

Provides access to the recognizer result Provides access to the recognizer result and alternatesand alternates

RecognitionResult ObjectRecognitionResult ObjectRecognizers contain a set of structuresRecognizers contain a set of structures Each structure called a “lattice”Each structure called a “lattice” Copied to RecognitionResult and othersCopied to RecognitionResult and others

TopString propertyTopString property

Store on the Strokes collection and add to Store on the Strokes collection and add to Ink.CustomStrokesInk.CustomStrokes SetResultOnStrokes methodSetResultOnStrokes method

RecognitionAlternates objectRecognitionAlternates object GetAlternatesFromSelection methodGetAlternatesFromSelection method Access to alternate paths through the latticeAccess to alternate paths through the lattice

Ink RecognitionInk Recognition

Platform Controls and Platform Controls and ComponentsComponents

Platform ControlsPlatform ControlsInkEditInkEdit Manipulate ink as textManipulate ink as text Intended for text and ink-to-text scenariosIntended for text and ink-to-text scenarios Enables smart formsEnables smart forms Superclass of the RichEdit controlSuperclass of the RichEdit control Win32, .NET, ActiveX versionsWin32, .NET, ActiveX versions

InkPictureInkPicture Ink-annotate images (.jpg, .bmp, .png, .gif)Ink-annotate images (.jpg, .bmp, .png, .gif) Collect and save ink as inkCollect and save ink as ink Subclass of PictureBox, akin to InkOverlay Subclass of PictureBox, akin to InkOverlay .NET and ActiveX versions only.NET and ActiveX versions only

Platform ComponentsPlatform Components

PenInputPanelPenInputPanel Programmatic access to the PIP (v 1.5)Programmatic access to the PIP (v 1.5)

Binds to any hWnd or ControlBinds to any hWnd or Control Basic control of the TIP (Lonestar)Basic control of the TIP (Lonestar)

Enable / Disable, etc.Enable / Disable, etc. Needed for backward compatibilityNeeded for backward compatibility

Divider (InkDivider)Divider (InkDivider) Divides Ink into words, lines, paragraphs Divides Ink into words, lines, paragraphs

and drawingsand drawings Transforms angled inkTransforms angled ink

Ink Division and the Ink Division and the Divider ComponentDivider Component

New Features in the 1.7 New Features in the 1.7 PlatformPlatform

Tablet PC Input PanelTablet PC Input PanelReplaces TIP and PIPReplaces TIP and PIP

No longer docked to top/bottom of screenNo longer docked to top/bottom of screen

Character Pad panelCharacter Pad panel

OS “just knows” when to show the TIP or OS “just knows” when to show the TIP or a TIP icona TIP icon

Grows as user writesGrows as user writes

On-the-fly recognition and pre-insertion On-the-fly recognition and pre-insertion error correctionerror correction

Context supportContext support

ContextContextWhat is context?What is context? App-specific info you provide to help the recognizerApp-specific info you provide to help the recognizer

Constrains recognition rules/possibilitiesConstrains recognition rules/possibilities Greatly improves recognition accuracyGreatly improves recognition accuracy

Examples of context informationExamples of context information Factoids: Factoids: recoContext.Factoid = “(!IS_DATE)”;recoContext.Factoid = “(!IS_DATE)”;

Word and phrase listsWord and phrase lists Custom input scopes with regular expressionsCustom input scopes with regular expressions

Context-enabled applications in LonestarContext-enabled applications in Lonestar Internet Explorer (Address Bar)Internet Explorer (Address Bar) Outlook 2003 (To:, Cc:, Bcc:)Outlook 2003 (To:, Cc:, Bcc:)

Context Features in Context Features in LonestarLonestar

Context for Context for Non-Ink-Enabled ControlsNon-Ink-Enabled Controls

SetInputScope API (new WIN32 API)SetInputScope API (new WIN32 API) Low-level programmatic solutionLow-level programmatic solution Binaries impactedBinaries impacted White paper available for .NET developmentWhite paper available for .NET development

Context Tagging Tool Context Tagging Tool Non-programmaticNon-programmatic Binaries not impactedBinaries not impacted Persists input scope / field relationships with Persists input scope / field relationships with

XML manifestXML manifest

Web SupportWeb SupportSupport for Partially Trusted callersSupport for Partially Trusted callers Host ink-enabled Windows Forms controls on Web Host ink-enabled Windows Forms controls on Web

pagespages Ink on a Web pageInk on a Web page Send ink to the Web serverSend ink to the Web server Retrieve ink from the Web serverRetrieve ink from the Web server

No-touch deploy Tablet PC applicationsNo-touch deploy Tablet PC applications http://www.Fabrikam.com/claims.exehttp://www.Fabrikam.com/claims.exe

Application examplesApplication examples Inventory managementInventory management Expense reportingExpense reporting Claims handlingClaims handling

Hosting Ink ControlsHosting Ink ControlsCreate the Windows Forms controlCreate the Windows Forms control Ink-enable with InkOverlay objectInk-enable with InkOverlay object Use InkEdit or InkPicture controlsUse InkEdit or InkPicture controls

Deploy the control library to the Web serverDeploy the control library to the Web server

Create the Web page Create the Web page <object> tag to hold the control<object> tag to hold the control Script functions to invoke the control’s Script functions to invoke the control’s

exposed membersexposed members

More info: ASP.NET QuickStart TutorialMore info: ASP.NET QuickStart Tutorial ““Using Windows Forms Controls in Internet Using Windows Forms Controls in Internet

ExplorerExplorer””

StylusInput APIsStylusInput APIsDirect access to digitizer data streamDirect access to digitizer data stream Manipulate packet data in real timeManipulate packet data in real time Separate Real Time Collection and UI threadsSeparate Real Time Collection and UI threads Better performance than InkOverlay, InkEdit, etc.Better performance than InkOverlay, InkEdit, etc.

Build custom componentsBuild custom components Real Time Ink componentsReal Time Ink components

Custom dynamic rendering (e.g., hardware acclerated)Custom dynamic rendering (e.g., hardware acclerated) Custom gesture recognitionCustom gesture recognition Disabled regions (WindowInputRectangle property)Disabled regions (WindowInputRectangle property)

Ink Collection componentsInk Collection components Custom InkOverlay or InkCollector componentsCustom InkOverlay or InkCollector components

RealTimeStylus PluginsRealTimeStylus Plugins

Other ImprovementsOther ImprovementsManipulation of inkManipulation of ink Create stroke with full packet descriptionCreate stroke with full packet description

HTTP User-AgentHTTP User-Agent Now identifies Tablet PC OS and SDK Now identifies Tablet PC OS and SDK

version version

Enhanced SDK DocumentationEnhanced SDK Documentation Lots of new content, samples, etc…Lots of new content, samples, etc…

Development EnvironmentDevelopment EnvironmentInstall on non-tablet devicesInstall on non-tablet devices SDK on Win2K SP3 and aboveSDK on Win2K SP3 and above WinXP Tablet PC Edition OS on non-tablet WinXP Tablet PC Edition OS on non-tablet

hardwarehardware Microsoft Virtual PCMicrosoft Virtual PC Available on MSDNAvailable on MSDN

MSDN: SDKMSDN: SDK MSDN Subscriber Downloads: OSMSDN Subscriber Downloads: OS

Simulate tablet digitizerSimulate tablet digitizer External HID digitizerExternal HID digitizer Mouse (UseMouseForInput property)Mouse (UseMouseForInput property)

Deploying your Tablet Deploying your Tablet Application Application

Redistributable merge modules Redistributable merge modules ConsiderationsConsiderations Recognition Recognition Ink collection w/o a digitizerInk collection w/o a digitizer PIP and TIPPIP and TIP No support for the Win9X platformsNo support for the Win9X platforms

© 2004 Microsoft Corporation. All rights reserved.© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.