Wolfram Burgard, Bernhard Nebel,

Martin Riedmiller, Jürgen Sturm, Felix Endres,

Jürgen Hess, Bastian Steder, Christian Dornhege,

Sascha Lange, Stefan Welker

Department of Computer Science

University of Freiburg, Germany

The TidyUpRobot Project

University of Freiburg, Germany

� Freiburg has 200.000 inhabitants

� Freiburg University has 30.000 students

� Traditional university with a technical faculty (CS + MST)

� Strong in AI and robotics

Robotics and AI in FreiburgAutonomous Intelligent SystemsWolfram Burgard

Machine LearningMartin Riedmiller

Social RoboticsKai Arras

Foundations of Artificial Intelligence

Bernhard Nebel

Humanoid RobotsMaren Bennewitz

Autonomous Intelligent Systems

� Mobile robots

� Mapping

� Localization

� Navigation

� Probabilistic robotics

� Multi-robot systems

� Embedded systems

Foundations of Artificial Intelligence

� Action planning

� Theory and practice

� Planning competitions

� Qualitative temporal-spatial reasoning

� RoboCup soccer

� World champion three times

� Autonomous table soccer

� Robocup Rescue League

Machine Learning

� Reinforcement learning

� Neural networks

� Fast, efficient learning

� Neural controllers

� Applications:

� Forecasting systems

� RoboCup soccer

� Industry

Project Goal of TidyUpRobot

� Reliable execution of fetch-and-carry tasks

� Mobile manipulation

� Domestic environments

� Example: Clear table in the living room, place objects in cabinets where they belong

Module Overview

� Re-useable software modules

� Licensed under (L)GPL

Feature-basedObject Recognition

Kinematic Models forArticulated Objects

Generalized MapRepresentation

Symbolic Planner With Semantic Attachments

Reinforcement LearningMotion Controller

� Learn object models

� Detect and localize objects in point clouds

Module: Object Recognition

Approach:

� Match point features from models

� Estimate object candidate pose

� Score and filter candidates

Module: Object Recognition

Module: Articulated Objects

� Detect articulated objects

� Learn kinematic models

� Open doors and drawers

� Use cupboards to stow objects

Module: Generalized Mapping

� Topological maps (multi-floor)

� Augmented with semantic information

� Detected (articulated) objects

� Kinematic models

� Action/skill models

� Symbolic description (for planner)

Module: Semantic Planning

� Plan with incomplete knowledge

� Sensory actions that gain knowledge

� Assertions (pre-/post-conditions)

� Continuous re-planning, triggered by assertions

Module: Semantic Planning

� Problem: Not all real-world facts can be represented efficiently on the logical level

� Solution: Semantic attachments (use the specific algorithms, e.g., geometric path planner)

Module: Action/Skill Learning

� Learning from experience:Reinforcement Learning for optimizing initial trajectories

� Improve available actions

� Acquire and provide new actions

� Identify constraints to be integrated as new semantic attachments

Project Evaluation

� Task execution reliability (>90%)

� Repeatability at three other PR2 sites (≥66%)

� Re-usability of our code (three other groups, using at least one module)

Open-Source Contributions

� Graph-based map representation (LGPL)

� 3D object detection and localization (LGPL)

� Articulated Objects (LGPL)

� Symbolic planner (GPL)

� Manipulation skill learner (LGPL)

� http://code.google.com/p/alufr-ros-pkg

Conclusions

� Joint initiative of the AI and robotics groups at the University of Freiburg

� TidyUpRobot Project

� Reliable execution of fetch-and-carry tasks

� Looking for labs interested in using our modules/application