Cognition-enabled Robot Control for Mixed ... - Icarus Project · July 2014 Cognition-enabled Robot Control in a Human-Robot Team 32. Artificial Intelligence Conclusions & Future

Artificial Intelligence

Cognition-enabled Robot Control for MixedHuman-Robot Rescue Teams

Fereshta Yazdani,Benjamin Brieber and Michael Beetz

Institute for Artificial IntelligenceUniversitat Bremen

18. July 2014


Simulation-based Rescue Scenario

F. Yazdani18. July 2014

Cognition-enabled Robot Control in a Human-Robot Team2


Contents

• Upcoming Problems

• Cognition-enabled control Framework

– Interpretation of Plan– Interpretation of vague Instructions

• Framework in Use

• Conclusion & Future Work




Motivation

“Go over there”

Upcoming Problems:

• plan execution with differentrobot control systems

• naturalistic task through vague,incomplete and ambigiousmultimodal instructions

• time! forcoordinating/organising therobots




Motivation

“Go over there”

Upcoming Problems:







Motivation

“Go over there”

Upcoming Problems:







Cognition-enabled Control System

“Go over there”




Cognition-enabled Control Framework

“Go over there”




Interpretation of Plan

High-level Plan “Take a picture”

( at− l o c a t i o n ( a− l o c a t i o n ( to−s e e a r t i f a c t ) )( p e r c e i v e a r t i f a c t ) )

( make−d e s i g l o c a t i o n ( ( to−s e e a r t i f a c t ) )( with−camera my−camera )( movement−t y p e d r i v i n g ) ) )

( make−d e s i g l o c a t i o n ( ( to−s e e a r t i f a c t ) )( with−camera my−camera )( movement−t y p e f l y i n g ) ) )




Interpretation of Plan

High-level Plan “Take a picture”

( at− l o c a t i o n ( a− l o c a t i o n ( to−s e e a r t i f a c t ) )( p e r c e i v e a r t i f a c t ) )

( make−d e s i g l o c a t i o n ( ( to−s e e a r t i f a c t ) )( with−camera my−camera )( movement−t y p e d r i v i n g ) ) )

( make−d e s i g l o c a t i o n ( ( to−s e e a r t i f a c t ) )( with−camera my−camera )( movement−t y p e f l y i n g ) ) )




Querying Capabilities of Robot Systems

• KnowRob(Tenorth et al., 2013) provides features for autonomousrobot control

• query for camera properties

:− c o m p o n e n t p r o p e r t i e s ( main camera , Prop , Value ) .Prop = imageSizeX ,Value = 6 4 0 ;Prop = imageSizeY ,Value = 8 0 0 ;. . .

• query for movement-base “flying”

:− c a p a v a i l a b l e o n r o b o t ( f l y i n g b a s e , s e l f ) .t r u e































Interpreting Multimodal Instructions

Interpretation from high-level instructions into low-level descriptions isvery difficult!

• symbolic descriptions called designators in CRAM(Moesenlechner etal., 2010)(Beetz et al., 2012)

• descriptions contain symbolic constraints to restrict solution space

(a location

(visible

(a location (position-of artifact)

(pointed-at

(a gesture (agent-team-leader))))))








(a location

(visible


(pointed-at









(a location

(visible


(pointed-at





Effect-based Action Parameterization

• translating descriptions into action parameters for the robotnavigation routines

(<- (desig ?desig ?loc1)

(desig-prop ?desig visible))

(<- (desig ?desig visible ?loc2)

(desig-prop ?desig (position-of ?obj))

(desig-prop ?desig (pointed-at ?ldr)))

• generating sampling-based solutions with generative model approach!



























setof ?Pose InAreaOf(pointedDirection, ?Pose) ?Poses ∧ member(?P,?Poses) ∧ Pose(CloseToSOF, ?P) ∧ beVisible(ForHumanLeader)






1. setof ?PoseInAreaOf(pointedDirection, ?Pose)?Poses

2. member(?P, ?Poses)

3. Pose(CloseToSOF, ?P)

4. beVisible(ForHumanLeader)






























































Knowledge Framework System

• access to information, e.g. agents capabilities• task of scheduling and organising• assign information to performing robots








Requests for Robot Capabilities

• request all capabilities of the rover

?− c a p a v a i l a b l e o n r o b o t ( Cap , Rover ) .Cap = A r m M o t i o n C a p a b i l i t y ;Cap = B a s e M o t i o n C a p a b i l i t y ;Cap = R e c h a r g e Q u a d r o p t e r C a p a b i l i t y ;

• request a robot with a specific capability

?− c a p a v a i l a b l e o n r o b o t (r e c h a r g e Q u a d r o p t e r C a p a b i l i t y , Rob ) .

Rob = Rover ;

• request missing capabilities for actions

?− m i s s i n g c o m p f o r a c t i o n ( r e c h a r g e−q u a d r o p t e r ,RoverWithoutBox , Comp ) .

Comp = Q u a d r o p t e r D o c k i n g S t a t i o n ;




Conclusions & Future Work

• cognition-enabled control framework for aheterogeneous team

• planinterpretation for different robots

– reasoning about capabilities of robotsystems

• interpretation of multimodal instructions

– reasoning with a generative modelapproach

• scheduling and organising robots withACMS

• Instead of using basic functions,using complex functions!




Thank you for your attention!



Documents

Cognition-enabled Robot Control for Mixed ... - Icarus Project · July 2014 Cognition-enabled Robot Control in a Human-Robot Team 32. Artificial Intelligence Conclusions & Future