Project ArteSImit

Artefact Structural Learning through Imitation

(TU München, U Parma, U Tübingen, U Minho, KU Nijmegen)

Giorgio Panin - TUM

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 2 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Artesimit Project Partners

Alois Knoll and colleagues (project coordinator, Robotics, Munich)

Giacomo Rizzolatti and colleagues (Neurophysiology, Parma)

Peter Thier and colleagues (Cognitive Neuroscience, Tübingen)

Wolfram Erlhagen and colleagues (Cognitive Modeling, Minho)

Harold Bekkering and colleagues (Cognitive Neuropsychology, Nijmegen)

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 3 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Central issue of imitation:do we imitate movements (means) or consequences (goals) of the

observed behavior?

Movements somehow need to be directly mapped (via-point approach), whereas goals need to be represented.

Movement mapping assumes homologue bodies, full access of the whole movement and stable environments.

Goal representation assumes an association between an (observed) action and the (observed) consequence.

The concept of goal-directed Imitation

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 4 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

How to work together?

Behavioral measurements (healthy and clinical patients)

Brain research (single-cell, fMRI, MEG, EEG)

Modeling (Dynamic neural fields)

Robotics (Visual analysis, cognition, and artefact control)

THE solution: One common paradigm

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 5 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Common paradigm

Starting position Movement 1

Movement 2A LED gives information about either the goal or the means

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 6 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Motor responses of parietal neurons

Parietal Mirror Neuron findings

Observation task Visual responses of parietal mirror neurons

Motor task

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 7 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Architecture of the dynamic model

PFC: Goal representations Task inputObject properties (e.g. colour)

STS: Visual desciption of grip and trajectory

PF: Sequence of means

F5: Movement primitives

Bridge Paradigm

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 8 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Visual system in action: Task recognition

Complete information acquisition for

the decision and planning tasks

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 9 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Learning I: Association between object properties and goal

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 10 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Learning II: Copying the means of the teacher

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 11 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Goal-directed imitation

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 12 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Partial information: Cueing of the goal

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 13 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Partial information: Cueing of the means

__________________________________________________________________________________________________________________________________________________________________________________

Project funded by the Future and Emerging Technologies arm of the IST Programme 14 FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689

Summary

The project combined ...

• the development of a new paradigm for goal-directed imitation with imaging studies across all modalities

• new findings in experimental neurophysiology and advanced neuromodelling of the structure of the PFC-(STS-PF-F5)-coupling to develop and implement computer-operational models for goal-directed imitation

• the development of a powerful model-based hand posture recogniser (using only one B/W camera)

into ...

• A complete artefact (robot system) capable of mimicking imitation behaviour of creatures: all project goals have been met!