Designing a Handwriting Recognition Based Writing Environment J C Read, S J MacFarlane, C Casey...

Designing a Handwriting Recognition Based Writing Environment

J C Read, S J MacFarlane, C CaseyDepartment of Computing, University of

Central Lancashire, Preston, UK

JCRead@uclan.ac.uk 2

IntroductionBackground InformationDescription of the Observational StudyFindings from the study

General FindingsFindings relating to the Handwriting RecognitionSatisfaction measuring

Informing the designDescription of the PrototypeConclusions

JCRead@uclan.ac.uk 3

Who? Why? What? Where? How?Janet C Read, lecturer and mother!Elodie, PhD studySpeech, handwriting: - Human Language Technology, Free text not command.Lancs..UK; white rural primary – age 7 – 9Research, Observations, Usability studies

JCRead@uclan.ac.uk 4

Previous WorkQWERTY keyboard difficult (WES2000)HLT attractive to children, HR feasible (HCI2000, HCI2001)Measuring Fun (CandF2000, CandF2001)Participatory design (IDC2002)Errors in HR interfaces (NordiCHI2002)

JCRead@uclan.ac.uk 5

The Observational StudyChildren aged 7 and 8Normal classroom activitiesIn twosLaptop (HR), Desktop (QWERTY), Desk (Pencil)Different writing tasksDifficulties, Errors, Corrections

JCRead@uclan.ac.uk 6

Pen and Paper ErrorsErrors made – missing words, spelling, letters written backwardsError prevention – asked, avoided, lookedError discovery – reading back, self, teacher or another childError repair – rub out, scribble out, cross out, overwrite, re-write, squeeze in , change

JCRead@uclan.ac.uk 7

QWERTY - ErrorsErrors made – missing words, spelling, hit wrong keyError prevention – asked, avoided, lookedError discovery – reading back, self, teacher or another child, wiggly lines!Error repair – position and rub out, rub back to, rub all, retype, change

JCRead@uclan.ac.uk 8

Handwriting Recognition - OverviewHardware – Graphics tablet and penSoftware – Recognition software

Fuzzy computingDisobedient – ambiguousCharacter or word basedOn line – ‘t’ stroke problems

Demonstration

JCRead@uclan.ac.uk 9

Demonstration of handwriting

JCRead@uclan.ac.uk 10

Handwriting Recognition – Errors (1)Child

Errors made – miss words, spellings, letters backwards, pen up

ComputerErrors made – Bad recognition, hardware

ChildError prevention – ask, avoid, look

ComputerError prevention – spell checker (not used)

JCRead@uclan.ac.uk 11

Handwriting Recognition – Errors (2)Child

Error discovery Before Recognition – reading back, teacher, other child After Recognition – as above + wiggly lines!

Error repair Before recognition – scribble out, overwrite, insert letter After Recognition – rub back to, rub all, rewrite all or

some, use QWERTY

JCRead@uclan.ac.uk 12

Satisfaction MeasuringErrors do not imply dissatisfactionWHY?

Sticky – addictive vs. nothing betterFunny – humour with recognition – easy to use

JCRead@uclan.ac.uk 13

Designing a Prototype - methodUsers

Children, environment, characteristics, mental models

TasksGoal oriented – hierarchy

SystemStates – dangerous states

Interface UI design guidelines

JCRead@uclan.ac.uk 14

ChildClassroom based – standard equipment, needs to be easy to use, robust, minimal help neededChildren – varied pen control, different levels of expertise with technology, different reading skills, poor or very good letter formationMental model – see tablet as paper – want to scribble out and insert missed words

JCRead@uclan.ac.uk 15

Child writing

JCRead@uclan.ac.uk 16

User GoalTo produce good written work

PlanningTranslationReviewing and Editing

(Hayes and Flower)

JCRead@uclan.ac.uk 17

Supporting the writer (1)Ideas – pop up in clouds, can have many, child can re-order them and can put them away, use handwriting that is not recognisedTranslation – training supported, lines can be drawn on screen or on the tablet (or both!); recognition can be immediate or delayed;

JCRead@uclan.ac.uk 18

Supporting the writer (2)Reviewing – computer can read back recognised text, child can read recognised or script text; spellings may be highlighted in recognised text – teacher controlsEditing – child can edit with rubber and pen on script, or with keyboard on recognised text

JCRead@uclan.ac.uk 19

System StatesEntry stateRecognition stateEdit state

DANGEROUS STATES Pens that point Cursors that confound Spaces that stop

JCRead@uclan.ac.uk 20

Interface Design (1)Full writing screenAbility to place new pagesMenus at the bottomHaptic boundary preferredTablet matched to screenPen can be turned on and off

JCRead@uclan.ac.uk 21

Interface design (2)Video clip facilityTeachers screenAssistantCustomisableTraining activities

JCRead@uclan.ac.uk 22

And so……………The designs for a product for a small group of users, for a narrow applicationKeyboard interfaceError repairSpeech recognition

JCRead@uclan.ac.uk 23

Thank youJanet C Read

University of Central LancashirePreston

Up North!England

JCRead@uclan.ac.uk