E. Gilet (1) , J. Diard (2) , R. Palluel-Germain (2) , P. Bessière (1)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Bayesian Action-Perception loop modeling: Application to trajectory generation and recognition using internal motor simulationE. Gilet(1), J. Diard(2), R. Palluel-Germain(2), P. Bessière(1)

(1) Laboratoire d’Informatique de Grenoble – CNRS, France(2) Laboratoire de Psychologie et NeuroCognition – CNRS, France

July, 5, 2010http://diard.wordpress.com/ Julien.Diard@upmf-grenoble.fr

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception of actions

2

(Calvo-Merino et al., 2004)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Reading and writing letters

3

(Longcamp, 2003)

Writing

Reading pseudo letters

Reading letters

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Interpretation• Motor simulation of actions during

perception

• Articulation between perception and action processes

4

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Modeling both reading and writingModeling internal simulation of

movements

5

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Bayesian Action-Perception (BAP) model

6

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

7

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

BAP model structure

8

internal letter representation

perception model

action modelsimulated perception model

coherence variables

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Cartesian and effector spaces

• Common space for perceptive and motor internal representations– Cartesian space

9

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Letter representation: sequences of via-points

10

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

11

Letter representation

« Laplace succession laws »

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Parameter indentification

12

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

13

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

14

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

15

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

16

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception model

17

• Deterministic via-point extraction

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

18

Action model

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

19

Trajectory generation model

• Minimum-acceleration model:– Cost function– Boundary conditions

• Polynomial solution

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

20

Simulated perception model

• Identical to the perception model

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

21

Coherence variables

• Allow to activate or deactivate submodels– « Bayesian switch »

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Coherence variable for controlling submodel activation

• Model– λ binary variable– Joint–

• Inference– P(A) = P(A): value of B does not influence A–

22

A B

λ

A B

A B

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

23

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception: reading letters

24

Correct recognition: 93.36%

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception: writer recognition

25

Correct recognition: 79.5%

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Action: writing letters

26

Variability between writers Variability between trials

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Motor equivalence

27

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Motor equivalence• Writer “style”

– (Wright, 1990)• Common

activated motor areas– (Wing, 2000)

28

(Serratrice. 1993)

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Action: Motor equivalence

29

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

30

Action: Motor equivalence

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception and Action: Copy

31

Trajectory copy Letter copy

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Perception and Action: Reading letters with motor simulation

32

Recall: reading letters without simulation

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

33

Perception and Action: Reading letters with motor simulation

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

34

Perception and Action: Reading letters with motor simulation

• Complete trajectories– Correct recognition score with simulation 93.36%– Correct recognition score without simulation 90.2%

• Incomplete trajectories

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– architecture and definition: overview• Experimental results

– simulation of cognitive tasks• Experimental prediction

35

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Experimental prediction

36

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Preliminary data

37

60

70

80

90

Control Group (motor simulation unaffected)

Motor interference Group (motor simulation affected)

Complete letters

Truncated letters

Recognition Performance (%)

F(1,23) = 3.06, p = 0.093

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Summary• BAP model

– Bayesian model of perception and action

– Includes an internal simulation loop

• Cognitive tasks– Reading without and with

motor simulation– Writer recognition– Writing with different

effectors– Copying letters and

trajectories• Basis for experimental

predictions38

Gilet, Diard, Palluel-Germain & Bessière — LIG & LPNC-CNRSBayesian Action-Perception model

Thank you for your attention !

Questions ?