Upload
phamdiep
View
219
Download
0
Embed Size (px)
Citation preview
The JAST Collaborative Human-Robot Dialogue SystemMary Ellen Foster, Markus Rickert, Michael BraunRobotics and Embedded Systems Group, Department of Informatics, Technical University of Munichhttp://www6.informatik.tu-muenchen.de/
Overall goal of project● Build jointly-acting autonomous systems that collaborate and work
intelligently on mutual tasks in dynamic, unstructured environments.
Research areas● Cognitive and neural bases of joint action● Dialogue and joint action● Joint action in autonomous systemshttp://www.jast-net.gr/
Task: User and robot jointly construct Baufix models
... User You should
insert the red bolt in the central hole of this slat.
JAST Which red bolt? There are three red bolts available.
User Any red bolt. JAST [picks up
nearest red bolt] Here is a red bolt. Can you insert it while I hold the slat?
U+J [action] JAST We need a nut
to secure this bolt....
System architecture
EMET
Object referencing andscene overview
Feature linking
Short-termmemory
Long-termmemory
Object recognition
Speech processing
Syntax analysis
Dialoguemanagement
Recipeexecution
Face tracking
Gazetracking
Face positiontracking
Action primitivesequence building
Manipulators
Action primitives
Motion planning
Collision detection
Hand tracking
Pointing detection
Gesture detection
Feedback
Facialexpressions
Speech
Low-levelerror handling
Gaze control
Joint actioncoordination