ADEADEA Quick Overview andA Quick Overview and

Guide to the Guide to the Agent DevelopmentAgent Development

Environment Environment

Latest updated: October 5, 2010Latest updated: October 5, 2010

Matthias ScheutzMatthias Scheutz

Human-Robot Interaction LaboratoryHuman-Robot Interaction LaboratoryDepartment of Computer ScienceDepartment of Computer Science

Tufts UniversityTufts University

What system is appropriate for What system is appropriate for complex robots?complex robots?

Single agent systems (SAS) where “Single agent systems (SAS) where “agent” means an agent” means an autonomously sensing and acting entity (e.g., a robot autonomously sensing and acting entity (e.g., a robot with sensors, effectors, etc.)?with sensors, effectors, etc.)?

Examples:Examples:SOAR (Laird, Rosenbloom, Newell)SOAR (Laird, Rosenbloom, Newell)ACT-R (Anderson) EPIC (Kieras)ACT-R (Anderson) EPIC (Kieras)Prodigy (Veloso) CogAff (Sloman)Prodigy (Veloso) CogAff (Sloman)Saphira (Konolige) AuRA (Arkin)Saphira (Konolige) AuRA (Arkin)

Cognitive architectures: Cognitive architectures: knowledge representation, reasoning, knowledge representation, reasoning, learning (e.g., chunking)learning (e.g., chunking)

Behavior-based architectures:Behavior-based architectures:behavior representation, sensory behavior representation, sensory processing, learning (e.g., processing, learning (e.g., reinforcement)reinforcement)

Multi agent systems (MAS) where “agent” means an Multi agent systems (MAS) where “agent” means an automonous automonous computational component (i.e., a computational component (i.e., a process) in a distributed computational system?process) in a distributed computational system?

Examples:Examples:JADE (Bellifemine) ZEUS (Nwana) JADE (Bellifemine) ZEUS (Nwana) AgentFactor (Collier) ABE (IBM)AgentFactor (Collier) ABE (IBM)AgentBase Player/Stage (Gerkey)AgentBase Player/Stage (Gerkey)Restina (Sycara) DACAT (Barber)Restina (Sycara) DACAT (Barber)

Distributed system architecture: Distributed system architecture: registry, white pages, yellow pages, registry, white pages, yellow pages, communication protocols and communication protocols and languages, load balancing and languages, load balancing and movability of agents, secure channels movability of agents, secure channels and transactions, resource and transactions, resource management, etc.management, etc.

What system is appropriate for What system is appropriate for complex robots?complex robots?

Would like to implement computationally demanding Would like to implement computationally demanding real-time architectures for complex robots on multiple real-time architectures for complex robots on multiple hosts in a systematic, non-adhoc fashion:hosts in a systematic, non-adhoc fashion:

⇒ ⇒ combine tools from SAS and MAS!combine tools from SAS and MAS!

Specifically, construe components Specifically, construe components of “SAS-agents” as “MAS agents” of “SAS-agents” as “MAS agents” (cp. to Minsky's “Society of Mind”)(cp. to Minsky's “Society of Mind”)

This allows us to distribute This allows us to distribute architectural components of “SAS-architectural components of “SAS-agents” using “MAS-agents”agents” using “MAS-agents”

Combine SAS and MASCombine SAS and MAS

ADE is component based – the basic component is called ADE is ADE is component based – the basic component is called ADE is called called serverserver for historical reasons; a server is a computational for historical reasons; a server is a computational unit representing a functional architectural component on a unit representing a functional architectural component on a possibly computationally independent remote resource, e.g.,possibly computationally independent remote resource, e.g.,

Videre server:Videre server: a representation of the Videre robot within a representation of the Videre robot within ADE with access to its onboard sensors and effectorsADE with access to its onboard sensors and effectors

Laser server:Laser server: a component that processes and provides a component that processes and provides information from a laser range finder sensor information from a laser range finder sensor

ADESim2010 server:ADESim2010 server: a 2D simulated environment that a 2D simulated environment that provides a robot and a laser serverprovides a robot and a laser server

ADEGUI server:ADEGUI server: a graphical user interface that allows the a graphical user interface that allows the user to view all the servers in a running ADE systemuser to view all the servers in a running ADE system

ADERegistry server:ADERegistry server: the main ADE control component that the main ADE control component that manages all servers (including authentication, access control, manages all servers (including authentication, access control, monitoring, error detection, etc.)monitoring, error detection, etc.)

The Agent Development The Agent Development Environment ADEEnvironment ADE

Systems Implemented in ADESystems Implemented in ADE

DIARCDIARC (consists of speech input and output servers, a parser, a (consists of speech input and output servers, a parser, a discourse, goal and action management, planner, laser, vision, discourse, goal and action management, planner, laser, vision, GPS, robot and other servers, and includes various application and GPS, robot and other servers, and includes various application and experiment instances for different robotic and simulation platforms experiment instances for different robotic and simulation platforms and tasks)and tasks)

Grid computing Grid computing (includes integration of SWAGES platform (includes integration of SWAGES platform into ADE with mechanisms for parallel distributed computing)into ADE with mechanisms for parallel distributed computing)

Multi-device scripting (mixed real-virtual) Multi-device scripting (mixed real-virtual) (includes (includes action, device, and simulation servers)action, device, and simulation servers)

Survey takingSurvey taking (includes survey and action servers)(includes survey and action servers)

Video and audio recording and streamingVideo and audio recording and streaming (includes (includes various video/audio recording/playing servers)various video/audio recording/playing servers)

Teaching environmentTeaching environment (including ADESime, ADESim2010 (including ADESime, ADESim2010 and Simbad servers, action, text input and output servers)and Simbad servers, action, text input and output servers)

Verifies servers as they come upVerifies servers as they come up

only recognized only recognized ADE ServersADE Servers are authorized (only ADE are authorized (only ADE servers can exist in ADE systems and use ADE services!)servers can exist in ADE systems and use ADE services!)

Provides “yellow pages” services to serversProvides “yellow pages” services to servers

Monitors the health of serversMonitors the health of servers

can restart a server is something goes wrongcan restart a server is something goes wrong

can even migrate to another host if original is can even migrate to another host if original is unavailableunavailable

Monitors the health of hostsMonitors the health of hosts

Can bootstrap the architectureCan bootstrap the architecture

config file specifies the componentsconfig file specifies the components

registry starts the java processesregistry starts the java processes

The ADE RegistryThe ADE Registry

ADE ServersADE Servers

Every component in an ADE Every component in an ADE system is a (derived) system is a (derived) ADE ServerADE Server

ADE Servers can be distributed ADE Servers can be distributed across multiple hosts and comacross multiple hosts and com--municate with each othermunicate with each other

ADERegistry

NavigationServer

ActionServer

The The ADE RegistryADE Registry (a special server) is used to connect servers to (a special server) is used to connect servers to each other:each other:

on startup, an ADE server contacts the registry and registerson startup, an ADE server contacts the registry and registersthen it is allowed to request resources (i.e., servers) it needsthen it is allowed to request resources (i.e., servers) it needsits requests are forwarded to the remote server and if allowed, the reits requests are forwarded to the remote server and if allowed, the re--questing ADE server (“client”) will receive a reference to the remote questing ADE server (“client”) will receive a reference to the remote server that can be used to access the server's services directlyserver that can be used to access the server's services directlyall servers, however, keep their connection with the registry throughall servers, however, keep their connection with the registry through--out their life-time (to allow for system monitoring, fault detection and out their life-time (to allow for system monitoring, fault detection and error recovery by the registry)error recovery by the registry)

ADE Example:ADE Example:A Simple Ball Following RobotA Simple Ball Following Robot

Implementing architecturualImplementing architecturualcomponents in ADE Serverscomponents in ADE Servers

Mapping between components in the agent Mapping between components in the agent architecture (left) onto the ADE servers in the architecture (left) onto the ADE servers in the implementation (right)implementation (right)

NLU

GoalManager

SpeechProduction

ActionDispatcher

Navigation

SphinxServer

DiscourseServer

ActionServer

FestivalServer

NavigationServer

SegwayServer

ArchitectureArchitecture Implementation in ADEImplementation in ADE

Server APIsServer APIs

ADE servers advertise their services through APIs ADE servers advertise their services through APIs (i.e., “interfaces” in the JAVA implementation)(i.e., “interfaces” in the JAVA implementation)

APIs can be arranged in a directed graph (“hierarchy”) APIs can be arranged in a directed graph (“hierarchy”) in that one API can be a subset of several other APIsin that one API can be a subset of several other APIs(e.g., a “velocity server” used to control rotational and (e.g., a “velocity server” used to control rotational and translational wheel speeds is subsumed in many robot platform translational wheel speeds is subsumed in many robot platform servers)servers)

ADE servers can request other servers based on their ADE servers can request other servers based on their type (i.e., the name of the server class), their instance type (i.e., the name of the server class), their instance (i.e., the name of the server object), or their API type(i.e., the name of the server object), or their API type

The latter allows the ADE Registry to use the API The latter allows the ADE Registry to use the API hierarchy to find an appropriate server hierarchy to find an appropriate server (e.g., a request for a velocity server might result in a reference (e.g., a request for a velocity server might result in a reference to a Pioneer server extending the velocity API being returned)to a Pioneer server extending the velocity API being returned)

Available ADE ServersAvailable ADE Servers

Sensory processingSensory processing

speech recognition (Sphinx, speech recognition (Sphinx, Sonic, Nuance, SailLabs)Sonic, Nuance, SailLabs)

vision (mono and stereo vision (mono and stereo firewire, USB, v4l, color blob firewire, USB, v4l, color blob detection, SIFT object detection, SIFT object detection and tracking, etc.)detection and tracking, etc.)

laser rangefinder (SICK, laser rangefinder (SICK, Hokuyu)Hokuyu)

sound localization (requires sound localization (requires special hardware)special hardware)

SimulatorsSimulators

Stage, Simbad, ADESim, Stage, Simbad, ADESim, ADESim2010, USARsimADESim2010, USARsim

Robot controlRobot control

Mobile wheeled platforms Mobile wheeled platforms (Pioneer, Segway, Videre, (Pioneer, Segway, Videre, James, and others)James, and others)

humanoid torso (Nao, humanoid torso (Nao, Reddy)Reddy)

DevicesDevices

speech production (Festival) speech production (Festival) with emotion modulationwith emotion modulation

Gogo board, various GPS Gogo board, various GPS sensors, IU robotic hand, sensors, IU robotic hand, microphone array, microphone array, physiological sensors, etc.physiological sensors, etc.

Available ADE ServersAvailable ADE Servers

Action and goal managementAction and goal management

action scripts + compileraction scripts + compiler

Natural language processingNatural language processing

Parsers (Link parser, Parsers (Link parser, Malt/Mink parser)Malt/Mink parser)

Discourse and semantic Discourse and semantic engines (Rise, TLDL, Mink, engines (Rise, TLDL, Mink, Chat, TT)Chat, TT)

Sentence production Sentence production (included in discourse (included in discourse engines)engines)

Cognitive ArchitecturesCognitive Architectures

Soar, ACT-R, IcarusSoar, ACT-R, Icarus

PlannersPlanners

Sapa-Replan (preliminary Sapa-Replan (preliminary version of TLPlan)version of TLPlan)

Localization and mappingLocalization and mapping

CarmenCarmen

MiscellaneousMiscellaneous

Survey takingSurvey taking

Video and audio streamingVideo and audio streaming

Grid engine (for cluster-Grid engine (for cluster-based distributed and based distributed and parallel computing)parallel computing)

Liquid State Machine layersLiquid State Machine layers

Generic InterfacesGeneric Interfaces

Mobile base:Mobile base:

VelocityServerVelocityServer

PositionServerPositionServer

PanTiltServerPanTiltServer

OrientationServerOrientationServer

NavigationServerNavigationServer

General percepts:General percepts:

ADEPerceptServerADEPerceptServer

Range sensors:Range sensors:

LaserServerLaserServer

SonarServerSonarServer

BumperSensorServerBumperSensorServer

Humanoids:Humanoids:

HumanoidLegServerHumanoidLegServer

HumanoidHeadServerHumanoidHeadServer

HumanoidHandServerHumanoidHandServer

HumanoidFaceServerHumanoidFaceServer

HumanoidArmServerHumanoidArmServer

NLP:NLP:

SpeechProductionServerSpeechProductionServer

SpeechRecognitionServerSpeechRecognitionServer

NLPServerNLPServer

Other interfaces for Predicates, Other interfaces for Predicates, Symbols, Terms, etc.Symbols, Terms, etc.

We will make a special version of ADE available to youWe will make a special version of ADE available to you

Most recent release:Most recent release: http://ade.sourceforge.net/http://ade.sourceforge.net/

./mkade small./mkade small

creates ADEENVcreates ADEENV

compiles commonly-used serverscompiles commonly-used servers

checks for required software, issues warnings, etc.checks for required software, issues warnings, etc../mkade <package>./mkade <package>

compiles an individual package with all its servers, compiles an individual package with all its servers, e.g., e.g., ./mkade vision./mkade vision

Packages are defined in the Packages are defined in the mkademkade script script

ADE servers can also be compiled manually:ADE servers can also be compiled manually:

javac -cp .:<yourcp> com/yourpkg/YourServerImpl.javajavac -cp .:<yourcp> com/yourpkg/YourServerImpl.java

Working with ADEWorking with ADE

You need:You need:

all host computers and their IP addresses you are going all host computers and their IP addresses you are going to use for ADE servers (they need to allow for ssh access to use for ADE servers (they need to allow for ssh access using your user name!); these need to be put in the using your user name!); these need to be put in the hostsfile the ADERegistry needs on startuphostsfile the ADERegistry needs on startup

paths to the ADE installation and the servers on all paths to the ADE installation and the servers on all computer you use if you like the ADERegistry to start computer you use if you like the ADERegistry to start your architecture; you then also need to create a your architecture; you then also need to create a configuration file that will be passed to the registryconfiguration file that will be passed to the registry

Start the ADERegistry (with the hosts file of allowed hosts):Start the ADERegistry (with the hosts file of allowed hosts):

./runaderegistry -k myhosts.hosts./runaderegistry -k myhosts.hosts

Start an ADE componentStart an ADE component

../runadeserver YourServerImpl -l <yourIP> -r <registryIP>/runadeserver YourServerImpl -l <yourIP> -r <registryIP>

Starting ADEStarting ADE

If you run the whole architecture on a single computer, then If you run the whole architecture on a single computer, then you don't need to specify any IP addresses (as the system you don't need to specify any IP addresses (as the system will default to “localhost” or 127.0.0.1)will default to “localhost” or 127.0.0.1)

You can start a registry manually directly without scripts: You can start a registry manually directly without scripts:

java -Dcomponent=ade.ADERegistryImpl \ java -Dcomponent=ade.ADERegistryImpl \ -Djava.rmi.server.hostname=<hostIP> \ -Djava.rmi.server.hostname=<hostIP> \ -Djava.net.preferIPv4Stack \ -Djava.net.preferIPv4Stack \ -cp .:core/ADEcore.jar:jars/SapaReplan.jar \ -cp .:core/ADEcore.jar:jars/SapaReplan.jar \ ade.ADEServerImpl -l <hostIP> -k <hostfile> ade.ADEServerImpl -l <hostIP> -k <hostfile>

Start a serverStart a server

java -Dcomponent=ade.<yourServerImpl> \ java -Dcomponent=ade.<yourServerImpl> \ -Djava.rmi.server.hostname=<hostIP> \ -Djava.rmi.server.hostname=<hostIP> \ -Djava.net.preferIPv4Stack \ -Djava.net.preferIPv4Stack \ -cp .:core/ADEcore.jar:jars/SapaReplan.jar \ -cp .:core/ADEcore.jar:jars/SapaReplan.jar \ ade.ADEServerImpl -l <hostIP> -r <registryIP> ade.ADEServerImpl -l <hostIP> -r <registryIP>

Running ADERunning ADE

If the ADERegistry is used to start servers for you, create If the ADERegistry is used to start servers for you, create config file (e.g., config/myconfig.config) specifying for each config file (e.g., config/myconfig.config) specifying for each entry:entry:

serverserverhosthostargumentsargumentsclasspathclasspathvmargsvmargs

./runaderegistry -f config/myconfig.config./runaderegistry -f config/myconfig.config -k config/myhosts.hosts -k config/myhosts.hosts

Note: if you want servers to be started automatically and Note: if you want servers to be started automatically and remotely, you will need to set up an remotely, you will need to set up an ssh-agentssh-agent that can that can automatically provide your automatically provide your passhprase passhprase (and remember to (and remember to include all remote hosts in known-hosts file for the registry)include all remote hosts in known-hosts file for the registry)

Running ADERunning ADE

You can always get information about servers by starting You can always get information about servers by starting them with the -h flag, which will print information about them with the -h flag, which will print information about command line optionscommand line options

Many servers support visualizations which can be enable by Many servers support visualizations which can be enable by starting them with the -g flag (e.g., ADESim2010)starting them with the -g flag (e.g., ADESim2010)

You can also always visualize the state of an ADE system You can also always visualize the state of an ADE system and enable visualizations using the ADEGUIand enable visualizations using the ADEGUI

./runadegui -l <hostIP> -r <registryIP>./runadegui -l <hostIP> -r <registryIP>

ADE also supports a system-wide logging mechanism which ADE also supports a system-wide logging mechanism which will allow you to print specific output into special log files will allow you to print specific output into special log files (this can be enabled through the logging call start(this can be enabled through the logging call start

setADEServerLogging(boolean state) // state is: true-on/false-offsetADEServerLogging(boolean state) // state is: true-on/false-off

Log files are placed in the Log files are placed in the logs/logs/ directory directory

Running ADERunning ADE

There are some convenience scriptsThere are some convenience scripts

Not guaranteed to work in all cases, but...Not guaranteed to work in all cases, but...

Start a registryStart a registry

./runaderegistry -k config/myhosts.hosts./runaderegistry -k config/myhosts.hosts

Start a serverStart a server

./runadeserver -cp <yourcp> com.yourpkg.YourServerImpl./runadeserver -cp <yourcp> com.yourpkg.YourServerImpl

Repeat as necessaryRepeat as necessary

These scripts will take any arguments that the manual These scripts will take any arguments that the manual invocation will take and will pass them on, so you can invocation will take and will pass them on, so you can specify:specify:

hosts filehosts file

registry IPregistry IP

etc.etc.

Writing your own ADE serverWriting your own ADE server

Representative PublicationsRepresentative Publications

Scheutz, Matthias and Kramer, James (2007) “Reflection and Scheutz, Matthias and Kramer, James (2007) “Reflection and Reasoning Mechanisms for Failure Detection and Recovery in a Reasoning Mechanisms for Failure Detection and Recovery in a Distributed Robotic Architecture for Complex Robots.” In Distributed Robotic Architecture for Complex Robots.” In Proceedings of the 2007 IEEE International Conference on Robotics Proceedings of the 2007 IEEE International Conference on Robotics and Automation (ICRA'07)and Automation (ICRA'07), Rome, Italy, April 2007, 3699–3704. , Rome, Italy, April 2007, 3699–3704.

Kramer, James; Scheutz, Matthias; and Schermerhorn, Paul (2007) Kramer, James; Scheutz, Matthias; and Schermerhorn, Paul (2007) “'Talk to me!': Enabling Communication between Robotic “'Talk to me!': Enabling Communication between Robotic Architectures and their Implementing Infrastructures.” In Architectures and their Implementing Infrastructures.” In Proceedings of the 2007 IEEE/RSJ International Conference on Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '07)Intelligent Robots and Systems (IROS '07), San Diego, CA, , San Diego, CA, October/November 2007, 3044–3049.October/November 2007, 3044–3049.

Kramer, J. and Scheutz, M. (2007) “Development Environments for Kramer, J. and Scheutz, M. (2007) “Development Environments for Autonomous Mobile Robots: A Survey”. Autonomous Mobile Robots: A Survey”. Autonomous RobotsAutonomous Robots, , 2222, , 22, , 101–132.101–132.

Scheutz, M. (2006). “ADE - Steps Towards a Distributed Scheutz, M. (2006). “ADE - Steps Towards a Distributed Development and Runtime Environment for Complex Robotic Agent Development and Runtime Environment for Complex Robotic Agent Architectures”. Applied Artificial Intelligence, 20, 4-5, 275–304.Architectures”. Applied Artificial Intelligence, 20, 4-5, 275–304.