Unicode and Keyboards on Windows Michael Kaplan Cathy Wissink Globalization Infrastructure and Font Technology Windows International Microsoft

Unicode and Keyboards on Unicode and Keyboards on WindowsWindows

Michael KaplanMichael KaplanCathy WissinkCathy Wissink

Globalization Infrastructure and Font TechnologyGlobalization Infrastructure and Font Technology

Windows InternationalWindows International

MicrosoftMicrosoft

25 March 200325 March 2003 Prague, Czech Republic (IUC23)Prague, Czech Republic (IUC23) 22

Who is this talk geared towards?Who is this talk geared towards?

This is a high-level introduction to This is a high-level introduction to keyboards on Windows, assuming no prior keyboards on Windows, assuming no prior knowledge.knowledge.Audience:Audience:– Developers new to conceptDevelopers new to concept– People who need to understand keyboards People who need to understand keyboards

enough to understand how applications can enough to understand how applications can work with themwork with them

– Not completely intended to be a “nuts and Not completely intended to be a “nuts and bolts” talkbolts” talk


Keyboards are simple, right?Keyboards are simple, right?

FontsFonts

Rendering enginesRendering engines

135 different locales on Windows135 different locales on Windows

Custom keyboard layoutsCustom keyboard layouts

Turns out they are not so simple!Turns out they are not so simple!


Keyboard Layout, definedKeyboard Layout, defined

A collection of data for each keystroke and A collection of data for each keystroke and shift state combination with a particular shift state combination with a particular keyboard driver (DLL)keyboard driver (DLL)

In other words, it is not the keyboard In other words, it is not the keyboard hardware but the software that connects hardware but the software that connects the hardware to the systemthe hardware to the system


Anatomy of a keystrokeAnatomy of a keystroke

User presses a keyUser presses a key

Key contains a scan codeKey contains a scan code

Keyboard layout DLL maps the scan code Keyboard layout DLL maps the scan code to a virtual key (VK)to a virtual key (VK)

OS sends the VK to the applicationOS sends the VK to the application

If asked, the DLL maps the VK/shift state If asked, the DLL maps the VK/shift state to characters (if they are assigned)to characters (if they are assigned)



Language featuresLanguage features

single character keystrokessingle character keystrokes

ligaturesligatures

dead keysdead keys

shift statesshift states

SGCapSGCap

Caps lock keyCaps lock key


LigaturesLigaturesFor keyboard layouts, not the same as the For keyboard layouts, not the same as the typographic/linguistic termtypographic/linguistic term

A combination of 2 to 4 UTF-16 code points A combination of 2 to 4 UTF-16 code points inserted by a single keystrokeinserted by a single keystroke


Examples of ligatures on keyboardExamples of ligatures on keyboard

Tamil keyboard in the shifted state, showing linguistic characters Tamil keyboard in the shifted state, showing linguistic characters Sri, Ksa, and others as ligaturesSri, Ksa, and others as ligatures


Dead keysDead keys

A combination of two keystrokes to insert A combination of two keystrokes to insert a single charactera single character

Used often in European languages for Used often in European languages for diacriticsdiacritics

Called “dead” since the first keystroke Called “dead” since the first keystroke does not appear to do anythingdoes not appear to do anything

Not intuitive to people who are not used to Not intuitive to people who are not used to themthem


Dead key example:Diaeresis on the Finnish keyboard

Input keysInput keys

Type Type ¨ (deadkey)¨ (deadkey)Type Type ¨ + a ¨ + a Type Type ¨ + A¨ + AType Type ¨ + o¨ + oType Type ¨ + O¨ + O......Type Type ¨ + <space>¨ + <space>

ResultResult

NothingNothingääÄÄööÖÖ......¨ (spacing diaeresis)¨ (spacing diaeresis)


Shift statesShift states

Only 47-48 physical keys to work with on Only 47-48 physical keys to work with on the keyboardthe keyboard

Shift states expand that number by many Shift states expand that number by many timestimes

For languages with casing, usually they For languages with casing, usually they are cased variants of each otherare cased variants of each other

More shift states available (AltGr, Ctrl)More shift states available (AltGr, Ctrl)


The Greek Keyboard, shifted and unshiftedThe Greek Keyboard, shifted and unshifted


Caps lockCaps lock

Usually intended to be a persistent Shift Usually intended to be a persistent Shift keypress (in other words, it acts like a keypress (in other words, it acts like a typewriter caps lock)typewriter caps lock)

Generally used to get at shift state without Generally used to get at shift state without having to press two keys concurrentlyhaving to press two keys concurrently

Often used with cased languagesOften used with cased languages

May do nothing in languages without case, May do nothing in languages without case, or be used for other purposesor be used for other purposes


SGCapSGCap

Advanced usage of the Caps Lock keyAdvanced usage of the Caps Lock key

Used by Swiss German, Czech, and Used by Swiss German, Czech, and Hebrew keyboardsHebrew keyboards

Adds two additional shift statesAdds two additional shift states

Only intuitive to people who understand Only intuitive to people who understand the mechanismthe mechanism


APIsAPIs

keybd_eventkeybd_eventMapVirtualKey[Ex]MapVirtualKey[Ex]OemKeyScanOemKeyScanSendInputSendInputToAscii[Ex]ToAscii[Ex]ToUnicode[Ex]ToUnicode[Ex]VkKeyScan[Ex]VkKeyScan[Ex]

The APIs allow developers to do the same mappings the OS The APIs allow developers to do the same mappings the OS does when moving between scan codes, VKs, and does when moving between scan codes, VKs, and characters.characters.


Factors in keyboard layout creationFactors in keyboard layout creation

National/regional standard?National/regional standard?What languages need to be supported?What languages need to be supported?Is the set of characters needed for the Is the set of characters needed for the language complete?language complete?Focus on code points and not glyphs?Focus on code points and not glyphs?Do all required characters exist in Do all required characters exist in Unicode?Unicode?Consistent use of features such as dead Consistent use of features such as dead keys, ligatures, etc.?keys, ligatures, etc.?


Where do IMEs fit in?Where do IMEs fit in?

For most languages, they don’t!For most languages, they don’t!

For CJK languages, even 47 keys X 8 shift For CJK languages, even 47 keys X 8 shift states is not enough states is not enough

Different mechanisms and APIs are used Different mechanisms and APIs are used for IME interactionfor IME interaction

More information can be found in the More information can be found in the Platform SDK Platform SDK ((http://msdn.microsoft.com/platformsdkhttp://msdn.microsoft.com/platformsdk))


FAQs about keyboardsFAQs about keyboards

Does Microsoft make these layouts up?Does Microsoft make these layouts up?

How do I add every glyph for my How do I add every glyph for my language?language?

Don’t I need an IME?Don’t I need an IME?

How do I replace the layout for my How do I replace the layout for my language if I do not like the one provided?language if I do not like the one provided?

How do I add my own keyboard layout?How do I add my own keyboard layout?


MSKLCMSKLCMicrosoft Keyboard Layout CreatorMicrosoft Keyboard Layout Creator

Create new keyboard layouts:Create new keyboard layouts:– Brand new layoutsBrand new layouts– Modified from existing keyboardsModified from existing keyboards

Deploy the created keyboard layout to Deploy the created keyboard layout to NT4, Win2000, WinXP, or Win Server NT4, Win2000, WinXP, or Win Server 2003 machines2003 machines

Save the definition of the keyboard for Save the definition of the keyboard for later developmentlater development


MSKLC demosMSKLC demos


Other ReferencesOther ReferencesThis tutorial's corresponding paperThis tutorial's corresponding paperWindows Keyboard LayoutsWindows Keyboard Layoutshttp://www.microsoft.com/globaldev/reference/keyboards.aspx

Nadine Kano, Nadine Kano, Developing International SoftwareDeveloping International Software (out of print, but still available on the web)(out of print, but still available on the web)http://microsoft.com/globaldev/dis_v1/disv1.asphttp://microsoft.com/globaldev/dis_v1/disv1.asp

New! New! Developing International SoftwareDeveloping International Software , 2nd , 2nd editioneditionhttp://microsoft.com/globaldev/dis_v2/disv2.asphttp://microsoft.com/globaldev/dis_v2/disv2.asp

Internationalization with Visual BasicInternationalization with Visual Basichttp://i18nWithVB.com/http://i18nWithVB.com/

Get on the MSKLC Beta!Get on the MSKLC Beta!


Questions?Questions?


Don't forget to fill out your Don't forget to fill out your evaluations! evaluations!

Unicode and Keyboards Unicode and Keyboards on Windowson Windows

Documents

Unicode and Keyboards on Windows Michael Kaplan Cathy Wissink Globalization Infrastructure and Font Technology Windows International Microsoft