HMM finds behavioral patterns…

Zoltán SzabóEötvös Loránd University

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

2

ContributorsNeural Information Processing Group

György Hévízi (first author)Mihály BiczóBarnabás PóczosBálint TakácsAndrás Lőrincz (head)

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

3

HCIAdaptive interface

User’s actual state?

Behavioral model is needed

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

4

Possibilities for behavioral models

Examples:Markov Chain (MC):

Hidden Markov Model (HMM):

Bayes Network ( ) :

mor

e ge

nera

l

f(Y|X)X

Y

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

5

Our long term goalAdaptation to user by RL: Markov Decision ProcessHMM:

Behavioral components upon practising?Similar patterns for users?Capable of extracting them?

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

6

ToolsDasher:

Pointing-gestures driven text entry solutionBorn at CambridgeOptional: predictive language model

Our solution: headmouse as input deviceFor control experiments: normal desk mouse

HMM: user modelling

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

7

Dasher

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

8

Headmouse

Combines: head detection + trackingTechnical details: Haar wavelets + optic flow

Non-intrusive + cheapAlternative communication toolFree for download:

http://nipg.inf.elte.hu/headmouse/headmouse.html

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

9

User modellingHidden Markov Model:

Observation: cursor speed user movementHidden states: Gaussian emission

Assumption: independence (diagonal covariance)

s

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

10

ExperimentsParticipants:

5 volunteer PhD studentsunexperienced in Dasher

Task: typing short sentences from lyrics with Dasher

e.g.: ,,Children need travelling shoes’’

Cursor trajectories were saved

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

11

Learning graph

Dasher can be learned.

(A)

(B)

(C)

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

12

Hidden states found by HMM

P

Else

Practising

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

13

Interpretation of hidden states

0

10

20

30

40

50

60

70

80

O1 O2 O3 O4 P

OK (% )Mistake (% )

OK Accelerate

Mistake

a

z

a

z

Most probable states by Viterbi:

others

IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University

14

OutlookRecognition of users’ behavioral patterns:

On-line adaptive functionality:Personalization for individual usersAlternative help options

Complex interaction with computer

Relevance: tool for handicapped non-speaking people