Probabilistic Control of Human Robot Interaction:Probabilistic Control of Human Robot Interaction:
Experiments with a Robotic Assistant for Nursing HomesExperiments with a Robotic Assistant for Nursing Homes
Joelle PineauMichael MontemerloMartha Pollack *
Nicholas RoySebastian Thrun
Carnegie Mellon University*University of Michigan
The Nursebot Project Joelle Pineau
Introducing Pearl – A mobile robotic assistant for elderly people and nurses
cameras
sonars
handle bars
mobile base
carrying tray
LCD mouth
touchscreen
microphone& speakers
laser
ROLE:
Moving thingsaroundMoving thingsaround
Management support of ADLsManagement support of ADLs
Providing physical assistance
Providing physical assistance
Remotehealth servicesRemotehealth services
Supportingcommunication
Supportingcommunication
Calling for helpin emergenciesCalling for helpin emergencies
Monitoring Rxadherence & safety
Monitoring Rxadherence & safety
Providing info(TV, weather)Providing info(TV, weather)
Reminding to eat,drink, take meds
Reminding to eat,drink, take meds
Linking caregiverand resources
Linking caregiverand resources
The Nursebot Project Joelle Pineau
The Nursebot project in its early days
The Nursebot Project Joelle Pineau
Architecture
Cognitive supportNavigation Communication
High-level controller
The Nursebot Project Joelle Pineau
• Localization and map building(Burgard et al., 1999)
• People detection and tracking(Montemerlo et al., 2002)
Architecture
Cognitive supportNavigation Communication
High-level controller
The Nursebot Project Joelle Pineau
• Autominder system (Pollack et al., 2002)
Architecture
Cognitive supportNavigation Communication
High-level controller
The Nursebot Project Joelle Pineau
• Speech recognition: Sphinx system(Ravishankar, 1996)
• Speech synthesis: Festival system(Black et al., 1999)
Architecture
Cognitive supportNavigation Communication
High-level controller
The Nursebot Project Joelle Pineau
The role of the top-level controller
• Établir les priorités parmi les objectifs des différents modules
• Négocier entre plusieurs objectifs ayant des coûts/gains variés
• Négocier entre l’acquisition d’information et la rencontre des objectifs
• Passer d’une tâche à l’autre en partageant l’information sensorielle
• Planifier malgré la présence d’incertitude
Cognitive supportNavigation Communication
ACTION SELECTION - based on the trade-off between:
- goals from different modules;
- goals with varying costs / rewards;
- reducing uncertainty versus accomplishing goals.
High-level controller
The Nursebot Project Joelle Pineau
Speech recognition with Sphinx
The Nursebot Project Joelle Pineau
Robot control under uncertainty
Belief StateP(st=weather-today)=0.5
P(st=appointment-today )=0.5
USER
Action={ say-weather,update-appointment,clarify-query}
Speech=“today”
Stateweather-today
The Nursebot Project Joelle Pineau
Robot control using Partially Observable MarkovPartially Observable Markov Decision Processes Decision Processes (POMDPs)
Belief state
USER + ENVIRONMENT + WORLD
Actions
ObservationsCosts / Rewards
State
Problem: Which action allows the robot to maximize its reward?
P(s1)P(s2)
The Nursebot Project Joelle Pineau
Methods to solve POMDPs
Objective: Find a policy, (b), which maximizes reward.
Complexity
Performance
QMDP
MDP
FIB
UMDP
AMDP
O(S2A) O(S2AT) O(S2AO )O(S2AO) O(S2AB) T
POMDP
New methods?
The Nursebot Project Joelle Pineau
New approach: A hierarchy of POMDPs
Idea: Exploit domain knowledge to divide one POMDP into many smaller ones.
Motivation: Complexity of POMDP solving grows exponentially with # of actions.
Assumption: We are given POMDP M = {S,A,,b,T,O,R} and hierarchy H
Act
ExamineHealth Navigate
MoveVerifyPulse
ClarifyGoal
North South East West
VerifyMeds
subtask
abstract action
primitive action
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action setNavigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
The Nursebot Project Joelle Pineau
PolCA: Planning with a hierarchy of POMDPs
Step 1: Select the action set
Step 2: Minimize the state set
Step 3: Choose parameters
Step 4: Plan task h
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
STATE FEATURESX-positionY-position
X-goalY-goal
HealthStatus
Navigate
Move ClarifyGoal
South East WestNorth
AMove = {N,S,E,W}
SMove = {X,Y}
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
ACTIONSNorthSouthEastWest
ClarifyGoalVerifyPulseVerifyMeds
PLAN
h
PLAN
h
PARAMETERS
{bh,Th,Oh,Rh}
PARAMETERS
{bh,Th,Oh,Rh}
The Nursebot Project Joelle Pineau
PolCA in the Nursebot domain
• Goal: A robot is deployed in a nursing home, where it provides reminders to elderly users and accompanies them to appointments.
• Domain: |S|=512, |A|=20, |O|=19 • Hierarchy:
The Nursebot Project Joelle Pineau
Sample scenario
The Nursebot Project Joelle Pineau
Results for dialogue system
0.1 0.1
0.18
POMDP policy MDP policy
The Nursebot Project Joelle Pineau
Summary
• We have developed a first prototype robot able to serve as a mobile nursing assistant for elderly people.
• The top-level controller uses a hierarchical variant of POMDPs to select actions.
• This allows it to acquire necessary information and successfully complete assigned tasks.
• Probabilistic techniques have been found to be very useful to flexibly model and track individuals.
The Nursebot Project Joelle Pineau
For more details: www.cs.cmu.edu/~nursebot
The Nursebot team
CMU - Robotics:Greg Armstrong
Michael MontemerloJoelle PineauNicholas RoyJamie Schulte
Sebastian Thrun
CMU - HCI/Design:Francine Gemperle
Jennifer GoetzSarah KieslerAaron Powers
U. of Pittsburgh - Nursing:Jacqueline Dunbar-Jacobs
Sandra EngbergJudith Matthews
U. of Pittsburgh - CS:Don Chiarulli
Colleen McCarthy
U. of Freiburg - CS:Maren BennewitzWolfram Burgard
Dirk Schulz
U. of Michigan - CS:Laura BrownDirk ColbryCheryl OroszBart PeintnerMartha PollackSailesh Ramakrishnan
Standard Robotics:Greg Baltus