I NFORMATION AND T ELECOMMUNICATION T ECHNOLOGY C ENTER Use of Multi-Agent Systems in Dynamic, Real-Time Environments Costas Tsatsoulis EECS

INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER

Use of Multi-Agent Systems in Dynamic, Real-Time Environments

Costas Tsatsoulis

EECS

2INFORMATION AND TELECOMMUNICATION TECHNOLOGY CENTER

Problem 1 Description

• Effective resource allocation in distributed autonomous systems under extreme time constraints

• Solve computationally intractable resource allocation problems in real time

• Use negotiation to relax and move boundaries in constraint satisfaction problems

• Relevance:– Autonomous logistics systems– Autonomous target tracking


Quick Summary of Methodology

• Instantiate agent profiles to establish preferences• Use CBR to establish negotiation strategy• Utility theory selects between viable strategies by

evaluating their outcomes based on current world and constraints

• Direct access to system-level resources• Negotiate about system resources as well as sensor

resources• System is part of the resource allocation process


Send messages

Receive messages

Initiate Negotiations

Respond to Negotiations

Read sensor data

Control the sensor

Read system data

Request system resources

Read self data

Perform problem solving


Agent

Agent polls sensor for dataat regular intervals

Vehicle appears and is sensed


Send messages

Receive messages



Control the sensor


Read sensor data

Read system data

Read self data


Read system data

Read self data



Send messages

Receive messages



Read sensor data

Control the sensor

Read system data


Read self data


Perform the task given the sensor data

Control the sensor

Decide if negotiationis needed



Send messages

Receive messages



Read sensor data

Control the sensor

Read system data


Read self data


Instantiate profileInstantiate negotiation neighborhood

Establish negotiation parametersInitiate negotiations

Instantiate profileInstantiate negotiation neighborhood

Establish negotiation parametersInitiate negotiations


Instantiate Negotiation Neighborhood

• Selection based ontask and world view

• Agent needs toknow models of theother agents

• Agent establishescommunication channels


Using Utility

• Use utility theory to sort potential negotiation partners• The utility is a weighted sum of three sets of attributes:

– the time that the target will hit the coverage of a potential partner or the time that the target will leave the coverage of a potential partner

– the past experience of negotiations between the agent and a potential partner (‘have I been helpful to you?’ ‘how many times have we had successful negotiations?’)

– the current relationship between the agent and a potential partner (‘are we already negotiating about some other things?’ ‘how much coverage overlap do we have between your sensor sectors and mine?’ ‘have I already initiated a negotiation request to you?’)

• For each potential partner, we obtain the most useful sector (highest utility) and sort the potential partners based on their most useful sectors.


Send messages

Receive messages



Read sensor data

Control the sensor

Read system data


Read self data


Send messages


Send messages

Receive messages



Read sensor data

Control the sensor

Read system data


Read self data


Receive messages

Read sensor data

Read system data

Read self data


Respond to negotiations

Send messages


CBR-Based Negotiation

• Our negotiation approach is argumentation-based• An initiating agent sends over arguments to support its request

to the responding agent• If the responding agent perceives the arguments as stronger

than its belief in conserving its own resources, then it will agree to give up the resource or to perform the requested task

• Resources may be surrendered completely, partially, or conditionally


Case-Based Negotiation

• An agent uses Case-Based Reasoning (CBR) to quickly find useful negotiation strategies

• An agent forms negotiation coalitions quickly using utility measures based on incomplete information of past and current relationships with its neighbors and their ability to handle the current task

• The agents calculate how long a negotiation is allowed to take using CBR and the current situation– Speed of vehicle allows calculation of when it will enter a radar’s

sensing area, and when and for how long it will be in areas covered by multiple sensors


Argumentative Negotiation 1

• Arguments– information supporting a request sent by an initiator to a responder

• Evidence Support – the cumulative value as a result of a responder evaluating the utility of

the arguments it receives from the initiator

• Persuasion Threshold– if the evidence support is greater than this value, then a responder

agrees to the request in a discrete resource case


Argumentative Negotiation 2

• Persuasion Function– a function based on evidence support such that a responder

may counter-offer to an initiating agent when time is running out in a non-discrete resource situation

– has two parameters:• conceding rate, • willingness factor,


Negotiations: Counteroffers

• Linear and Exponential Persuasion Functions for counter-offers

evidence support

% o

f req

uest

Maximumresourceto give up

Minimumresourcewilling togive up,

conceding rate,

ssflinear )(

evidence support

% o

f req

uest

Maximumresourceto give up

Minimumresourcewilling togive up,

slexponentia esf )(


Tracking




Conclusions

• Developed, implemented and tested a methodology that allows autonomous, intelligent agents to perform real-time negotiation for the allocation of limited resources

• Technique was applied to a distributed sensor network for real-time target tracking


Problem 2 Description• The PRISM (Polar Radar for Ice Sheet Measurement) is developing

mobile, autonomous sensors for measuring the ice sheet mass balance• Sensors include:

– a SAR radar operating in monostatic or bistatic mode at three different frequencies (60, 150 and 350 MHz)

– a wideband dual mode radar employing a depth sounder and an accumulation radar

• Goal is to add intelligence to the SAR and the rover to dynamically determine

– Appropriate operating mode and frequency of the SAR– Distance between the vehicles carrying the SAR receiver and transmitter– Size of measurement swath– Cross-track transect length and along-track spacing – The vehicle speed and scan path

• The system fuses real-time sensor data with a priori observations and rover and radar health status to make decisions


The Need for Data

• Decision making agents require data regarding the geophysical parameters being sensed (e.g. basal condition, ice depth, etc.) and the status of the radars (operating mode, frequency, etc.) and the rover (GPS location, fuel, speed, heading, etc.)

• Decision making requires delivery of different data at different time intervals and for different conditions

• Data sources, communication resources and computational resources may disappear or reappear at any time

• The changing conditions of the decision making require dynamic, adaptive delivery of relevant, timely data


Track vehicle and autonomous rover carrying SAR and wideband radars in field tests in

Greenland during Summer 2004.


Solution

• A multiagent collaborative architecture that involves a number of software agents functioning continuously, autonomously, and intelligently

• There are two kinds of agents: source agents that represent data sources, and decision-making agents, that can perform actions that affect the radar and the rover

• Decision making is performed by a Bayesian Belief Network


The Matchmaker

– A Matchmaker architecture allows data sources to advertise capabilities, and data consumers to advertise needs to a common location

– The advertisement of capabilities gives the name of the source agent, the data it can provide, the maximum frequency of delivery, and the unit

– The advertisement of needs gives the name of the consumer agent and the data type(s) it is interested in

– The Matchmaker tells the consumer which source agents can satisfy its needs

– Consumers register for data with the sources using time- or event-driven requirements

– The Matchmaker periodically pings data sources to ensure they are still alive


Wrapping

• Data sources and decision making or data consuming programs are wrapped by an agent architecture

• The wrapper acts as a proxy of the actual data source

• Data sources are different and have different drivers and APIs

• All wrappers are identical and communicate using the same language, allowing for seamless data exchange without knowledge of the underlying protocols


MatchMaker - Registration

Wrapped data sources for rovers and sensors (GPS, temp, velocity, snow

depth, SAR, etc.)

MatchMakerAdvertisement of capabilities

Wrapped data consumers thatdecide on the parameters of

the radars and rovers

Advertisement of data needs

The MatchMaker connects the producerof the data with the consumer


Consumer-Producer Registration

• Time-driven registration for data– "send me data once"– "send me data during time period T1-T2 every X time units"– "send me data until I tell you to stop every X time units"

• Event-driven registration for data– "send me data when it is equal to/greater than/less than

value V1"– "send me data when within X time units the data changes by

constant/percentage"

• Registration is dynamic: as the needs of the consumer change it can adapt and cancel data subscriptions and start new ones


Implementation

• Agent wrapper and Matchmaker implemented in Java• Communication over a network performed using

Java's Remote Method Invocation (RMI)• The agent language is unique to the PRISM agents,

but is conforms to FIPA standards and to RDF– The FIPA (Foundation for Intelligent Physical Agents) has

defined XML standards to support communicative acts and basic message structure

– RDF (Resource Description Framework) was developed by the World Wide Web Consortium (W3C) as an ontology to support the interchange of information about resources on the web


Example of Data Driven Registration<fipa:Action rdf:ID="temperature_sube_0">

fipa:actor>TemperatureAgent</rdf:actor><fipa:act>subscribeEvent</rdf:act><fipa:argument>

<rdf:seq>

<rdf:li>edu.ku.ittc.prism.intel.agent.rules.Comparator</rdf:li><rdf:li>greaterThan</rdf:li><rdf:li>temperatureCelcius</rdf:li><rdf:li>35.0</rdf:li></rdf:seq>

</fipa:argument></fipa:Action>


Experiments

• Wrapped over 40 data sources on the radars and rover• Wrapper agents and Matchmaker were distributed over a

network• End-to-end testing during Greenland experiment on Summer

2004 showed correct subscription, data source registration, and data delivery

• Consumer agents displayed appropriate and dynamic subscription, cancellation of subscription, and re-subscription to data providers


Short Example from Output Log

2004-07-15 14:42:41,440 MATCHMAKER-0.[] - Fetched collaborator for id [RadarDecisionEngineAgent,101]…2004-07-15 14:43:41,506 MATCHMAKER-0.[] - Starting ping scan of all collaborators...2004-07-15 14:43:43,419 MATCHMAKER-0.[] - Removing [PositionAgent,12] - ping scan revealed that it is not alive and did not deregister properly.2004-07-15 14:43:43,419 MATCHMAKER-0.[] - The collaboration can no longer provide for the [position] capability

2004-07-15 14:43:03,922 TemperatureAgent-11.[] - performing action - updateInternalTemperatureCelcius2004-07-15 14:43:03,922 TemperatureAgent-11.[] - Attempting to connect to RMI temperature sensor 2004-07-15 14:43:03,932 TemperatureAgent-11.[] - Found RemoteTemperatureSensor


Conclusions

• Developed an intelligent Matchmaker that allows wrapped data sources and data consumers to find out about each other

• Developed a language to allow time-driven and data-driven subscription of consumer agents with data agents

• Implemented the complete methodology in a multiagent system using Java, RMI, FIPA and RDF

• Tested the system comprehensively as part of a project to add intelligence to a SAR and an autonomous rover used to measure the mass balance of the Polar ice sheets

Documents

I NFORMATION AND T ELECOMMUNICATION T ECHNOLOGY C ENTER Use of Multi-Agent Systems in Dynamic, Real-Time Environments Costas Tsatsoulis EECS