HMM finds behavioral patterns…
Zoltán SzabóEötvös Loránd University
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
2
ContributorsNeural Information Processing Group
György Hévízi (first author)Mihály BiczóBarnabás PóczosBálint TakácsAndrás Lőrincz (head)
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
3
HCIAdaptive interface
User’s actual state?
Behavioral model is needed
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
4
Possibilities for behavioral models
Examples:Markov Chain (MC):
Hidden Markov Model (HMM):
Bayes Network ( ) :
mor
e ge
nera
l
f(Y|X)X
Y
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
5
Our long term goalAdaptation to user by RL: Markov Decision ProcessHMM:
Behavioral components upon practising?Similar patterns for users?Capable of extracting them?
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
6
ToolsDasher:
Pointing-gestures driven text entry solutionBorn at CambridgeOptional: predictive language model
Our solution: headmouse as input deviceFor control experiments: normal desk mouse
HMM: user modelling
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
7
Dasher
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
8
Headmouse
Combines: head detection + trackingTechnical details: Haar wavelets + optic flow
Non-intrusive + cheapAlternative communication toolFree for download:
http://nipg.inf.elte.hu/headmouse/headmouse.html
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
9
User modellingHidden Markov Model:
Observation: cursor speed user movementHidden states: Gaussian emission
Assumption: independence (diagonal covariance)
s
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
10
ExperimentsParticipants:
5 volunteer PhD studentsunexperienced in Dasher
Task: typing short sentences from lyrics with Dasher
e.g.: ,,Children need travelling shoes’’
Cursor trajectories were saved
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
11
Learning graph
Dasher can be learned.
(A)
(B)
(C)
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
12
Hidden states found by HMM
P
Else
Practising
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
13
Interpretation of hidden states
0
10
20
30
40
50
60
70
80
O1 O2 O3 O4 P
OK (% )Mistake (% )
OK Accelerate
Mistake
a
z
a
z
Most probable states by Viterbi:
others
IJCNN 2004 Neural Information Processing Group, Eötvös Loránd University
14
OutlookRecognition of users’ behavioral patterns:
On-line adaptive functionality:Personalization for individual usersAlternative help options
Complex interaction with computer
Relevance: tool for handicapped non-speaking people
Recommended
