The Soft Real-Time Agent Control Architecture *

Horling, Bryan and Lesser, VictorUniversity of Massachusetts

Department of Computer ScienceAmherst, MA 01003

{bhorling, lesser)@cs.umass.edu

Vincent, RegisSRI International

333 Ravenswood AvenueMenlo Park, CA 94025

vincent@ai.sri.com

Wagner, ThomasHoneywell Laboratories

Automated Reasoning GroupMinneapolis, MN 55418

wagner _tom@htc.honeywell.com

Abstract

Real-time control has become increasingly importantas technologies are moved from the lab into real worldsituations. The complexity associated with these sys-tems increases as control and autonomy are distributed,due to such issues as precedence constraints, shared re-sources, and the lack of a complete and consistent worldview. In this paper we describe a soft real-time architec-ture designed to address these requirements, motivatedby challenges encountered in a distributed sensor allo-cation environment. The system features the abilityto generate schedules respecting temporal, structuraland resource constraints, to merge new goals with exist-ing ones, and to detect and handle unexpected resultsfrom activities. We will cover a suite of technologiesbeing employed, including quantitative task represen-tation, alternative plan selection, partial-order schedul-ing, schedule consolidation and conflict resolution in anuncertain environment. Technologies which facilitateon-line real-time control, including schedule caching andvariable time granularities are also discussed.

OverviewIn the field of multi-agent systems, much of the researchand most of the discussion focuses on the dynamics andinteractions between agents and agent groups. Just as

*Effort sponsored in part by the Defense Advanced Re-search Projects Agency (DARPA) and Air Force ResearchLaboratory Air Force Materiel Command, USAF, underagreements number F30602-99-2-0525 and DOD DABT63-99-1-0004. The U.S. Government is authorized to reproduceand distribute reprints for Governmental purposes notwith-standing any copyright annotation thereon. This materialis also based upon work supported by the National ScienceFoundation under Grants No. IIS-9812755 and IIS-9988784.Any opinions, findings, and conclusions or recommendationsexpressed in this material are those of the author(s) anddo not necessarily reflect the views of the National ScienceFoundation. Furthermore, the views and conclusions con-tained herein are those of the authors and should not beinterpreted as necessarily representing the official policies orendorsements, either expressed or implied, of the DefenseAdvanced Research Projects Agency (DARPA), Air ForceResearch Laboratory or the U.S. Government.Copyright (E) 2002, American Association for Artificial Intel-ligence (www.aaai.org). All rights reserved.

important, however, is the design and behavior of the in-dividual agents themselves. The efficiency of an agent’sinternal mechanics contribute to the foundation of thesystem as a whole, and the degree of flexibility thesemechanics offer affect the agent’s achievable level of so-phistication, particularly in its interactions with otheragents (Lesser 1991; 1998). We believe that a generalcontrol architecture, responsible for both the planningfor the achievement of temporally constrained goals ofvarying worth, and the sequencing of actions local tothe agent that have resource requirements, can providea robust and reusable platform on which to build highlevel reasoning components. In this article, we will dis-cuss the design and implementation of the Soft RealTime Architecture (SRTA), a generic planning, schedul-ing and execution subsystem designed to address theseneeds.

The SRTA architecture, a sophisticated agent controlengine with relatively low overhead, provides several keyfeatures:

1. The ability to quickly generate plans and schedules forgoals that are appropriate for the available resourcesand applicable constraints, such as deadlines and ear-liest start times.

2. The ability to merge new goals with existing ones, andmultiplex their solution schedules.

3. The ability to efficiently handle deviations in expectedplan behavior that arise out of variations in resourceusage patterns and unexpected action characteristics.

The system is implemented as a set of interacting com-ponents and representations. A domain independenttask description language is used to describe goals andtheir potential means of completion, which includes aquantitative characterization of the behavior of alterna-tives. A planning engine can determine the most appro-priate means of satisfying such a goal within the set ofknown constraints and commitments. This permits thesystem to be able to adjust which goals it will achieve,and how well it will achieve these chosen goals basedon the dynamics of the current situation. Schedulingservices integrate these actions and their resource re-quirements with those of other goals being concurrentlypursued, while a parallel execution module performs theactions as needed. Exception handling and conflict res-olution services help repair and route information when

From: AAAI Technical Report WS-02-15. Compilation copyright © 2002, AAAI (www.aaai.org). All rights reserved.

unexpected events take place. Together, this systemcan assume responsibility for the majority of the goal-satisfaction process, which allows the high-level reason-ing system to focus on goal selection, determining goalobjectives and other potentially domain-dependent is-sues. For example, agents may elect to negotiate overan abstraction of their activities or resource allocations,and only locally translate those activities into a moreprecise form (Mailler et al. 2001). SRTA can thenuse this description to both enforce the semantics ofthe commitments which were generated, and automati-cally attempt to resolve conflicts that were not addressedthrough negotiation.

Based on this architecture, it should be clear that thisresearch assumes sophisticated agents are best equippedto operate and address goals within a resource-bound,interdependent, mixed-task environment. In such a sys-tem, individual agents are responsible for effectively bal-ancing the resources they choose to allocate to their mul-tiple time and resource sensitive activities. A differentapproach addresses these issues through use of groups ofsimpler agents, which may individually act in responseto single goals and only as a team address large-grainedissues. In such an architecture, either the host operat-ing system or increased communication must be used toresolve temporal or resource constraints, and yet morecommunication is required for the agents to effectivelydeliberate over potential alternative plans in context.Decomposing the problem space completely to "simple"agents does not address the problem or remove the in-formation and decision requirements. We feel that sucha design is over-decomposed, and would more effectivelybe addressed by more sophisticated agents capable of di-rectly reasoning over and acting upon multiple concur-rent issues, thereby saving both time and bandwidth.

In recent work on a distributed sensor network appli-cation(Horling et al. 2001), which will be discussed inmore depth below, we exploited SRTA to create a virtualagent organization of simple, single-threaded agents.These "virtual" agents were in fact goals, created asneeded and dynamically assigned to a specific sophis-ticated "real" agent based on information approximat-ing the current resource usage of agents and the typeof resources available at each agent. The "real" agentthen performed detailed planning/scheduling based onlocal resource availability and priority of these goals,and multiplexed among the different goals that it wasconcurrently executing in order to meet soft real-timerequirements.

SRTA operates as a functional unit within a Java-based agent, which itself it running on a conventional(i.e. not real-time) operating system. The SRTA con-troller is designed to be used in a layered architec-ture, occupying a position below the high-level rea-soning component in an agent (Zhang et al. 2000;Bordini et al. 2002) (see Figure 2). In this role, will accept new goals, report the results of the activitiesused to satisfy those goals, and also serve as a knowledgesource about the potential ability to schedule future ac-tivities by answering what-if style queries. Within this

context, SRTA offers a range of features designed to pro-vide support for operating in a distributed, intelligentenvironment. The goal description language supportsquantitative, probabilistic models of activities, includingnon-local effects of actions and resources and a varietyof ways to define how tasks decompose into subtasks.In particular, the uncertainty associated with activitiescan be directly modeled through the use of quantitativedistributions describing the different outcomes a givenaction may produce. Commitments and constraints canbe used to define relationships and interactions formedwith other agents, as well as internally generated limitsand deadlines. The planning process uses this informa-tion to generate a number of different plans, each withdifferent characteristics, and ranked by their predictedutility. A plan is then used to produce a schedule ofactivities, which is combined with existing schedules toform a potentially parallel sequence of activities, whichare partially ordered based on their interactions withboth resources and one another. This sequence is usedto perform the actions in time, using the identified pre-conditions to verify if actions can be performed, andinvoking light-weight rescheduling if necessary. Finally,if conflicts arise, SRTA can use an extendable series ofresolution techniques to correct the situation, in addi-tion to passing the problem to higher level componentswhich may be able to make a more informed decision.

An important aspect of most real-world systems istheir ability to handle real-time constraints. This isnot to say that they must be fast or agile (althoughit helps), but that they should be aware of deadlineswhich exist in their environment, and how to operatesuch that those deadlines are reasoned about and re-spected as much as possible. This notion of real-time isweaker than its relative, strict real-time, who’s adher-ents attempt to rigorously quantify and formally boundtheir systems’ execution characteristics. Instead, sys-tems working in soft real-time operate on tasks whichmay still have value for some period after their dead-lines have passed (Stankovic & Ramamritham 1990),and missing the deadline of a task does not lead to dis-astrous external consequences. Our research addresses aderivative of this concept, where systems are expected tobe statistically fast enough to achieve their objectives,without providing formal performance guarantees. Thisallows it to successfully address domains with highly un-certain execution characteristics and the potential forunexpected events, neither of which are well suited for ahard real-time approach. As its name implies, SRTA op-erates in soft real-time, using time constraints specifiedduring the goal formulation and scheduling processes,and acting to meet those deadlines whenever necessary.In this system, we have sacrificed the ability to provideformal performance guarantees in order to address morecomplex and uncertain problem domains. As will beshown shortly, this technology has been used to success-fully operate in a real-time distributed environment.

To operate in soft-real time, an agent must know whenactions should be performed, how to schedule its activi-ties and commitments such that they can be performed

55

or satisfied, and have the necessary resources on handto complete them. Our solution to this problem ad-dresses two fronts. The first is to implement the tech-nologies needed to directly reason about real-time. Asmentioned above, we begin by modeling the quantita-tive and relational characteristics of the goals and ac-tivities the agent may perform, which can be done apriori and accessed as plan library or through a run-time learning process(Jensen et al. 1999). This infor-mation is represented, along with other goal achieve-ment and alternative plan information, in a T/EMS taskstructure (Decker & Lesser 1993; Horling et al. 1999)(discussed in more detail later). A planning compo-nent, the Design-to-Criteria scheduler (DTC) (Wagner,Garvey, & Lesser 1998; Wagner & Lesser 2000), usesthese T/EMS task structures, along with the quantita-tive knowledge of action interdependence and deadlines,to select the most appropriate plan given current envi-ronmental conditions. This plan is used by the PartialOrder Scheduler process to determine when individualactions should be performed, either sequentially or inparallel, within the given precedence and runtime re-source constraints. In general, we feel that real-timecan be addressed through the interactions of a series ofcomponents, operating at different granularities, speedand satisficing (approximate) behaviors.

The second part of our solution attempts to optimizethe running time of our technologies, to make it easierto meet deadlines. The partial order schedule providesan inherently flexible representation. As resources andtime permit, elements in the schedule can be quickly de-layed, reordered or parallelized. New goals can also beincorporated piecemeal, rather than requiring a compu-tationally expensive process involving re-analysis of theentire schedule. Together, these characteristics reducethe need for constant re-planning, in addition to mak-ing the scheduling process itself less resource-intensive.Learning plays an important role in the long-term vi-ability of an agent running in real time, taking advan-tage of the repetitive nature of its activities. Schedulesmay be learned and cached, eliminating the need to re-invoke the DTC process when similar task structuresare produced, and the execution history of individualactions may be used to more accurately predict theirfuture performance. A similar technique could be usedto track the requisite actions and time needed to devoteto particular goals.

This article will proceed by discussing the problemdomain which motivated much of this system. Func-tional details of the architecture will be covered, alongwith further discussion of the various optimizations thathave been added. We will conclude with a more descrip-tion of SRTA’s ability to adapt to varying conditions,and summarize the significant characteristics of the ar-chitecture. Note also that a more thorough overview ofthis architecture can be found in (Horling et al. 2002).

Problem Domain

A distributed resource allocation domain which moti-vated much of this work (Horling et al. 2001) will be

Figure 1: High-level distributed sensor allocation archi-tecture. A) shows the initial sensor layout, decompo-sition and allocation of sector managers. B) shows thedissemination of scanning tasks. The new track managerin C) can be seen coordinating with sensors to track target, while the resulting data is propagated in D) forprocessing.

used throughout this article to ground the topics whichare discussed and formulate examples. This sectionwill briefly describe the environment and the particu-lar challenges it offers. Components of the SRTA ar-chitecture have also been used successfully in severalother domains, such as intelligent information gathering(Lesser et al. 2000), intelligent home control (Lesser etal. 1999), and supply chain (Horling, Benyo, & Lesser2001).

The distributed resource environment consists of sev-eral sensor nodes arranged in a region of finite area,as can be seen in Figure 1A. Each sensor node is au-tonomous, capable of communication, computation andobservation through the attached sensor. We assumea one-to-one correspondence between each sensor nodeand an agent, which serves locally as the operator of thatsensor. The high level goal of a given scenario in thisdomain is to track one or more target objects movingthrough the environment. This is achieved by havingmultiple sensors triangulate the positions of the targetsin such a way that the calculated points can be usedto form estimated movement tracks. The sensors them-selves have limited data acquisition capabilities, in termsof where they can focus their attention, how quickly thatfocus can be switched and the quality / duration tradeoffof its various measurement techniques. The attention ofa sensor, or more specifically the allocation of a sensor’stime to a particular tracking task, therefore forms an im-portant, constrained resource which must be managedeffectively to succeed.

The real-time requirement of this environment is de-rived from the triangulation process. Under ideal condi-tions, three or more sensors will perform measurementsat the same instant in time. Individually, each sensorcan only determine the target’s distance and velocity rel-ative to itself. Because each node will have seen the tar-get at the same position, however, these gathered datacan then be fused to triangulate the target’s actual loca-tion. In practice, exact synchronization to an arbitrarily

56

high resolution of time is not possible, due to the uncer-tainty in sensor performance and clock synchronization.A reasonable strategy then is to have the sensors per-form measurements within some relatively small windowof time, which will yield positive results as long as thetarget is near the same location for each measurement.Thus, the viable length of this window is inversely pro-portional to the speed of the target (in our scenarios weuse a window of one second for a target moving one footper second).

Competing with the sensor measurement activity area number of other local goals, including sector manage-ment (Figure 1A), target discovery scanning (1B), surement tasks for other targets (1C), and data pro-cessing (1D). We don’t see these as separate agents threads, but rather as different objectives/goals that anagent is multiplexing. To operate effectively, while stillmeeting the deadlines posed above, the agent must becapable of reasoning about and acting upon the impor-tance of each of these activities.

In summary, our real-time needs for this applicationrequire us to synchronize several measurements on dis-tributed sensors with a granularity of one second. Amissed deadline may prevent the data from being fused,or the resulting triangulation may be inaccurate - butno catastrophic failure will occur. This provides indi-vidual agents with some minimal leeway to occasion-ally decommit from deadlines, or to miss them by smallamounts of time, without failing to achieve the overallgoal. At the same time, there is a great deal of un-certainty in when new tasks will arrive, and how longindividual actions will take, so a strict timing policy istoo restrictive. Thus, our notion of real-time here is rel-atively soft, enabling the agents to operate effectivelydespite uncertainty over the behavioral characteristicsof computations and their resource requirements.

Further details on this domain and the multi-agentarchitecture designed to address it can be found in (Hor-ling et al. 2001).

Real-Time Control Architecture

Our previous agent control architecture, used exclu-sively in controlled time environments, was fairly largegrained. As goals were addressed by the problem solvingcomponent, they would be used to generate task struc-tures to be analyzed by the Design-To-Criteria (DTC)scheduler. The resulting linear schedule would then bedirectly used for execution by the agent. Task structurescreated to address new goals would be merged with ex-isting task structures, creating a monolithic view of allthe agent’s goals. This combined view would then bepassed again to DTC for a complete re-planning andre-scheduling. Execution failure would also lead to acomplete re-planning and re-scheduling. This techniqueleads to "optimal" plans and schedules at each point ifmeta-level overheads are not included. As will be dis-cussed in a later section, however, the combinatorics as-sociated with such large structures can get quite high.This made agents ponderous when working with fre-quent goal insertion or handling exceptions, because of

Figure 2: High-level agent control architecture.

the need to constantly perform the relatively expensiveDTC process. In a real-time environment, characterizedby a lot of uncertainty in the timing of actions and thearrival of new tasks, where the agent must constantlyreevaluate their execution schedule in the face of variedaction characteristics, this sort of control architecturewas impractical.

In the SRTA architecture, we have attempted to makethe scheduling and planning process more incrementaland compartmentalized. New goals can be added piece-meal to the execution schedule, without the need to re-plan all the agent’s activities, and exceptions can be typ-ically be handled through changes to only a small subsetof the schedule. Figure 2 shows the new agent controlarchitecture we have developed to meet our soft real-time needs. We will first present an overview of how itfunctions, and cover the implementation in more detailin later sections. In this architecture, goals can arriveat any time, in response to environmental change, localplanning, or because of requests from another agents.The goal is used by the problem solving componentto generate a T/EMS task structure, which quantita-tively describes the alternative ways that goal may beachieved. The T/EMS structure can be generated ina variety of ways; in our case we use a T/EMS "tem-plate" library, which we use to dynamically instantiateand characterize structures to meet current conditions.Other options include generating the structure directlyin code (Lesser et al. 2000), or making use of an ap-proximate base structure and then employing learningtechniques to refine it over time (Jensen et al. 1999).

The Design-To-Criteria component, used in the orig-inal controller described earlier, retains a critical rolein SRTA. Where before it was responsible for both se-lecting an appropriate plan of activities and producinga schedule of actions for monolithic structures, SRTAgenerally exploits only its planning capabilities for dis-crete structures. Using the T/EMS structure mentionedabove, along with criteria such as potential deadlines,

57

Figure 3: The timeline of events for our running sce-nario. Shown are the arrival times for the goals shownin Figures 4 and 5, along with the negotiated deadlinefor Task 2.

minimum quality, and external commitments, DTC se-lects an appropriate plan.

The resulting plan is used to build a partially or-dered schedule, which will use structure details of theT~MS structure to determine precedence constraintsand search for actions which can be performed in par-allel. Several components are used during this finalscheduling phase. A resource modeling component isused during this analysis to ensure that resource con-straints are also respected. A conflict resolution mod-ule reasons about mutually-exclusive tasks and commit-ments, determining the best way to handle conflicts. Fi-nally, a schedule merging module allows the partial or-der scheduler to incorporate the actions derived from thenew goal with existing schedules. Failures in this processare reported to the problem solver, which is expected tohandle them (by, for instance, relaxing constraints suchas the goal completion criteria or delaying its deadline,completing a substitute goal with different characteris-tics, or decommiting from a lower priority goal or thegoal causing the failure).

Once the schedule has been created, an executionmodule is responsible for initiating the various actionsin the schedule. It also keeps track of execution per-formance and the state of actions’ preconditions, poten-tially re-invoking the partial order scheduler when failedexpectations require it. As will be shown later, the par-tial order scheduler uses a shifting mechanism to resolvesuch failures with minimal overhead where possible.

To better explain our architecture’s functionality, wewill work through a example in the next several sec-tions, using simplified versions of task structures in theactual sensor network application. The initial timelinefor this example can be seen in Figure 3. At time 0 theagent recognizes its first goal - to initialize itself. Afterstarting the execution of the first schedule it will re-ceive another goal to track a target and sent the resultsbefore time 2500. Later, a third goal, to negotiate fordelegating tracking responsibility, is received. We willshow how these different goals may be achieved, andtheir constraints and interdependencies respected.

T/EMS GenerationBefore progressing, we must provide some backgroundon our task description language, T/EMS. T/EMS, theTask Analysis, Environmental Modeling and Simulationlanguage, is used to quantitatively describe the alterna-tive ways a goal can be achieved (Decker & Lesser 1993;Horling et al. 1999). A T/EMS task structure is essen-tially an annotated task decomposition tree. The high-est. level nodes in the tree, called task groups, represent

Figure 4: An example T/EMS task structure for track-ing. The expected execution characteristics are shownbelow each method, and the Send-Results method inthis figure has a deadline of 2500.

goals that an agent may try to achieve. The goal ofthe structure shown in Figure 4 is Task2. Below a taskgroup there will be a set of tasks and methods which de-scribe how that task group may be performed, includingsequencing information over subtasks, data flow rela-tionships and mandatory versus optional tasks. Tasksrepresent sub-goals, which can be further decomposed inthe same manner. Task2, for instance, can be performedby completing subtasks Set-Parameters, Track, andSend-Result s.

Methods, on the other hand, are terminal, and repre-sent the primitive actions an agent can perform. Meth-ods are quantitatively described, with probabilistic dis-tributions of their expected quality, cost and duration.These quantitative descriptions are themselves groupedtogether as outcomes, which abstractly represent thedifferent ways in which an action can conclude, costincurred. Set-Parameters, then, is described withtwo potential outcomes, Must-Update-Parameters andAlready-Set-Correctly, each with its relative proba-bility and description of expected duration.

The quality accumulation functions (QAF) below task describes how the quality of its subtasks is com-bined to calculate the task’s quality. For example, themin QAF below Task2 specifies that the quality ofTask2 will be the minimum quality of all its subtasks- so all the subtasks must be successfully performed forthe Task2 task to succeed. On the other hand, the maxbelow Track says that its quality will be the maximumof any of its subtasks.

Interactions between methods, tasks, and affected re-sources are also quantitatively described as interrela-tionships. The enables interrelationships in Figure 4represent precedence relationships, which in this casesay that Set-Parameters, Track, and Send-Resultsmust be performed in-order. An analogous disables in-terrelationship exists, as well as the softer relations ]a-cilitates and hinders. These latter two are particularlyinteresting because they permit the further modeling ofchoice - the agent might choose to perform a facilitatingmethod prior to its target to obtain an increase in thelatter’s quality, or ignore the method to save time.

lock2 and release2 are resource interrelationships,describing, in this case, the consumes and produces ef-fects method Send-Results has on the resource RF.

58

These indicate that when the method is activated, itwill consume or produce some quantity of that resource.The resource effect is further described through the lim-its interrelationship, which defines the changes in themethod’s execution characteristics when that resourceis over-consumed or over-produced. The resource itselfis also modeled, including its bounds and current value(as shown below the RF triangle), and whether it is con-sumable or not (e.g. printer paper is consumable, wherethe printer itself is not).

Together, these descriptions provide the foundationfor the planning process to reason about the effects ofselecting this method for execution, so a planner canchoose correctly when the agent is willing to trade offuncertainty against quality or some other metric.

The problem solver is responsible for translating itshigh-level goals into T/EMS, which serves as a more de-tailed representation usable by other parts of the agent.This can be done by building T/ti~MS structures dynami-cally at runtime, reading static structures from a library,or using a hybrid scheme consisting of a library of tem-plate structures which can be annotated at runtime.

At time 0 the agent will use its template library togenerate the initialization structure seen in Figure 5A.In this structure, the agent must first Init and thenCalibrate its sensor. Properties passed into the tem-plate specify the particular values used in Init, suchas the sensor’s desired gain settings or communicationschannel assignment, as well the number of measure-ments to be used during Calibrate. As specified by theenables interrelationship, Init must successfully com-plete before the agent can Send-Message, reporting itscapabilities to its local manager. Send-Message alsouses resource interrelationships to obtain an exclusivelock on the RF communication resource. Only one ac-tion at a time can use RF to send message, so all mes-saging methods have similar locking interrelationships.As we will see later, this indirect interaction betweenmessaging methods creates interesting scheduling prob-lems. Task2 and Task3, shown in Figures 4 and 5B,respectively, are generated later in the run in a similarmanner.

DTC Planner / Initial Scheduler

Design-to-Criteria (DTC) scheduling is the soft real-time process of evaluating different possible courses ofaction for an intelligent agent and choosing the coursethat best fits the agent’s current circumstances. For ex-ample, in a situation where the RF resource is under agreat deal of concurrent usage, the agent may be unableto send data using the traditional quick communicationsprotocol and thus be forced to spend more time on amore reliable, but slower method to produce the samequality result (analogous to selecting between a UDP orTCP session). Or, in a different situation when bothtime and cost are constrained, the agent may have tosacrifice some degree of quality to meet its deadline orcost limitations. Design-to-Criteria is about evaluatingan agent’s problem solving options from an end-to-endview and determining which tasks the agent should per-

form, when to perform them, and how to go about per-forming them. Having this end-to-end view is crucial forevaluating the relative performance of alternative plansable to satisfy the goal.

One would expect any reasonable planning process toenforce so-called "hard" constraints - ones which mustbe satisfied for a goal to be achieved or a commitmentsatisfied. It is DTC’s additional ability to reason aboutweaker, optional interactions which sets it apart. Thesum QAF in T/EMS , for instance, defines a task who’squality is determined by the sum of all its subtasks’qualities. In a time critical situation, DTC may opt fora shorter, but lower quality plan which only calls for oneof these subtasks to be executed. In more relaxed condi-tions, more may be added to the plan. Similarly, soft in-terrelationships such as facilitates or hinders may be re-spected or not, depending on their specific quantitativeeffects and the current planning co~text. DTC’s behav-ior is governed through the use of a criteria description,which is provided to it along with each T/EMS structure.This criteria specifies, for example, the desired balancebetween plan quality and duration, or what level of un-certainty is tolerable(Wagner, Garvey, & Lesser 1997).More information covering the techniques DTC uses canbe found in (Wagner, Garvey, &: Lesser 1998).

Returning to our example, DTC is used to select themost appropriate set of actions from the initializationtask structure. In this case, it has only one valid plan:Init, Calibrate, and Send-Message. A more interest-ing task structure is seen in Task2 from figure 4, whichhas a set of alternative methods under the task Track.A deadline is associated with Send-Result, correspond-ing to the desired synchronization time specified by theagent managing the tracking process. In this case, DTCmust determine which set of methods is likely to obtainthe most quality, while still respecting that deadline.Because T/EMS models duration uncertainty, the issueof whether or not a task will miss its deadline involvesprobabilities rather than simple discrete points. Thetechniques used to reason about the probability of miss-ing a hard deadline are presented in (Wagner & Lesser2000). It selects for execution the plan Set-Parameters,Track-Medium, and Send-Results. After they are se-lected, the plans will be used by the partial order sched-uler to evaluate precedence and resource constraints,which determine when individual methods will be per-formed.

Partial Order Scheduler

DTC was designed for use in both single agents andagents situated in multi-agent environments. Thus, itmakes no assumption about its ability to communicatewith other agents or to "force" coordination betweenagents. This design approach, however, leads to compli-cations when working in a real-time, multi-agent envi-ronment where distributed resource coordination is anissue. When resources can be used by multiple agents atthe same time, DTC lacks the ability to request commu-nication for the development of a resource usage model.This is the task of another control component that forms

59

I Send-Message I

A) Initialization task structure.

I Neg°tiate-Tracking I I Send-Tracking-Inf° I

0.011000.01 1000.0

B) Tracking goal negotiation task structure.

Figure 5: Two T/EMS task structures, abstractions of those used in our agents.

scheduling constraints based on an understanding of re-source usage. In most applications, these constraintsare formed by rescheduling to analyze the implicationsof particular commitments. In the real-time sensor ap-plication, the rescheduling overhead is too expensive forforming these types of relationships. The solution wehave adopted is to use a subset of DTC’s functional-ity, and then offioad the distributed resource and finegrained scheduling analysis to a different component -the partial order scheduler. Specifically, DTC is usedin this architecture to reason about tradeoffs betweenalternative plans, respect ordering relationships in thestructure, evaluate the feasibility of soft interactions,and ensure that hard duration, quality and cost con-straints are met.

DTC presents the partial order scheduler with a lin-ear schedule meeting the requested deadline. Timing de-tails, with the exception of hard deadlines generated bycommitments to other agents and overall goal deadlines,are ignored in the schedule, which is essentially used asa plan. The partial order scheduler uses this to build apartially ordered schedule, which includes descriptionsof the interrelationships between the scheduled actionsin addition to their desired execution times. This par-tially ordered schedule explicitly represents precedencerelationships between methods, constraints and dead-lines. This information arises from commitments, re-source and method interrelationships, and the QAFs as-signed to tasks, and is encoded as a precedence graph.This graph can then be used both to determine whichactivities may potentially be run concurrently, becausethey have no precedence relation between them or theydo not have interfering resource usage, and where par-ticular actions may be placed in the execution timeline.Of particular significance, this latter functionality allowsthe scheduler to quickly reassess scheduled actions incontext, so that some forms of rescheduling can be per-formed with very low overhead when unexpected eventsrequire it. Much of this information can be directly de-termined from the T/EMS task structure.

Consider the tracking task structure shown in Fig-ure 4. Enables interrelationships between the tasksand methods indicate a strict ordering is necessary forthe three activities to succeed. In addition (althoughnot shown in the figure), a deadline constraint existsfor Send-Result, which must be completed by time2500. Next look at the initialization structure in Fig-ure 5A. While an enables interrelationship orders Init

and Send-Message, it does not affect the Calibratemethod. Internally, the partial order scheduler will usethis information to construct a precedence graph. Inthis example, the graph will first be used to determinethat Calibrate may be run in parallel with the othertwo methods in its structure. Later, when Task2 arrives,the updated graph can be used to find an appropriatestarting time for Set-Parameters which still respectsthe deadline of Send-Result.

While the partial order scheduler may directly rea-son about direct precedence rules as outlined above, amore robust analysis is needed to identify indirect in-teractions which occur through common resource usage.Because of uncertain and probabilistic interactions be-tween resources and actions, both locally and those to beperformed by other agents, a thorough temporal modelis needed to correctly determine acceptable times andlimits for resource usage.

Resource Modeler

In order to bind resources, we use another componentcalled the resource modeler. The partial order sched-uler does this by first producing a description of how agiven method is expected to use resources, if at all. Thisdescription includes such things as the length of the us-age, the quantity that will be consumed or produced,and whether or not the usage will be done throughoutthe method’s execution or just at its start or completion.The scheduler then gives this description to the resourcemodeler, along with constraints on the method’s startand finish time, and asks it to find a point in time whenthe necessary resources are available.

As with most elements in T/EMS the resource us-age is probabilistically described, so the scheduler mustalso provide a minimum desired chance of success to themodeler. At any potential insertion point, the modelercomputes the aggregate affects of the new resource us-age, along with all prior usages up to the last knownactual value of the resource. The expected usage fora given time slot can become quite uncertain, as theprobabilistic usages are carried through from each priorslot. If the probability of success for this aggregate us-age lies above the range specified by the scheduler, thenthe resource modeler assumes the usage is viable at thatpoint. Since a given usage may actually take place overa range of time, this check is performed for all otherpoints in that range as well. If all points meet the suc-cess requirement, the resource modeler will return the

60

valid point in time. After this, the scheduler will insertthe usage into the model, which will then be taken intoaccount in subsequent searches. If a particular point intime is found to be incompatible, the resource modelercontinues its search by looking at the next "interesting"point on its timeline- the next point at which a resourcemodification event occurs. The search process becomesmuch more efficient by moving directly from one poten-tial time point to the next, instead of checking all pointsin between, making the search-time scale with the num-ber of usage events rather than the span of time whichthey cover. Caching of prior results, especially the re-sults of the aggregate usage computation, is also usedto speed up the search process.

This information is used by the resource modeler tosearch for appropriate places where new resource us-ages may be inserted. In general, the scheduling pro-cess will provide a set of resource usage descriptions ex-tracted from methods it is attempting to schedule, whichmay affect multiple different resources at different times,along with start and finish time bounds and a minimumdesired probability of success, and the resource mod-eler will return the first possible match if one is found.These constraints are then used along with direct struc-tural precedence rules and the existing schedule to laydown a final schedule.

Schedule Merging

Once potential interactions, through interrelationships,deadlines or resource uses are determined, the partialorder scheduler can evaluate what the best order of ex-ecution is. Wherever possible, actions are parallelizedto maximize the flexibility of the agent, as was shownearlier. In such cases, methods running concurrentlyrequire less overall time for completion, and thus offermore time to satisfy existing deadlines or take on newcommitments. Once the desired schedule ordering is de-termined, the new schedule must be integrated with theexisting set of actions.

The partial order scheduler makes use of two othertechnologies to integrate the new goal with existingscheduled tasks. The first is a conflict resolution mod-ule, which determines how best to handle un-schedulableconflicts, given the information at hand. A second com-ponent handles the job of merging the new goal’s sched-ule with those of prior goals. The specific mechanismused is identical to that which determines order of ex-ecution. Interdependencies between this large set ofmethods, either direct or indirect, are used to determinewhich actions can be performed relative to one another.This information is then used to determine the final de-sired order of execution.

To this point in our example, the agent has been askedto work towards three different goals, each with slightlydifferent execution needs. Taskl allows some measureof parallelism within itself, as Init and Calibrate canrun concurrently because no ordering constraints existbetween them. Task2, received some time later, mustbe run sequentially, and its method Send-Result mustbe completed by time 2500. Task3 is received later still,

A)

Figure 6: A) Initial schedule produced after all thegoals have been received, with a Send-Result dead-line of 2500, B) the invalid schedule showing thatconstraint broken by the unexpected long duration ofNegotiate-Tracking, and C) the corrected schedule re-specting the deadline.

and also must be run sequentially. All three, however,require the use of the RF resource, for communicationneeds, and are thus indirectly dependent on one another.The partial order scheduler produces the schedule seenin Figure 6A, where all the known constraints are met.Some measure of parallelism can be achieved, seen withSet-Parameters and Send-Message, and also betweenTrack-Medium and the methods in Task3. Note that theresource modeler detected the incompatibility betweenthe methods using RF (shaded gray), however, and there-fore do not overlap.

Conflict Resolution

Suppose next that Negotiate-Tracking is takinglonger than expected, forcing the agent to dynam-ically reschedule its actions. Because the methodSend-Tracking-Info cannot start before the com-pletion of Negotiate-Tracking, due to the en-ables interrelationship shown in Figure 5B, thepartial order scheduler must delay the start ofSend-Tracking-Info. A naive approach would simplydelay Send-Tracking-Info by a corresponding amount.This has the undesirable consequence of also delayingSend-Result, because of the contention over the RF re-source. This will cause Send-Result to miss its deadlineof 2500, as shown in the invalid schedule in Figure 6B.

Fortunately, the partial order scheduler was able todetect this failure, because of the propagation of ex-ecution windows. Send-Result was marked with alatest start time of 2000. This caused the sched-uler to try other permutations of methods, which re-sulted in the schedule shown in Figure 6C, which delaysSend-Tracking-Info in favor of Send-Result. This al-lows the agent to proceed successfully despite a failedexpectation. This process is accomphshed by first de-laying the finish time of the offending method in theschedule to reflect the current state of affairs, and thenrecursively delaying any other methods which are de-pendent on that method until a valid solution is foundor a recursive limit is reached.

61

This type of simple conflict resolution is performedautomatically, through the cooperation of the executionmodule, which detects the unexpected behavior, andthe scheduling component which attempts to repair theproblem using the quick shifting technique shown above.In some cases, particularly when methods actually failto achieve their goal, this sort of simple shifting is notsufficient to repair the problem. To handle these cases,we designed a conflict resolution module capable of an-alyzing a particular situation and suggesting solutions.

Abstractly, the conflict resolution module is a cus-tomizable engine, which applies techniques encoded as"plug-ins" to a particular situation. If the set of tech-niques available is not appropriate for the agent de-signer, they are free to add or remove them as needed.Each technique plug-in is associated with a discrete nu-meric priority rating, typically specified by the designerof the plug-in, which controls the ordering in which thetechniques are applied. When searching for a conflictresolution, the engine will begin by applying all tech-niques marked with the highest priority. If one or moresolutions are suggested, then that set of solutions is re-turned for the caller to select from. If no solutions aresuggested, the engine will apply the techniques at thesecond-to-highest level, and so on. If the techniquesare ordered appropriately, such as with quick or effec-tive techniques first followed by slower or less applicableones, the engine can make efficient use of its time.

As an example, consider the T/EMS structure shownin Figure 4. We will assume three different resolutionplug-ins are in use by the agent, corresponding to sev-eral of the techniques outlined above. At the highestpriority level is Check-Cache, which searches for cachedresolution techniques which are applicable to the cur-rent problem. At the next level is Alternate-Plan, whichlooks for compatible results from the previous schedul-ing activity. At the lowest priority level is Regenerate-Plans, which uses DTC to generate a completely newset of viable plans. The initial schedule generated fromthis structure would be { Set-Parameters, Track-High,Send-Results }. In this instance however, Track-Highfails, forcing the conflict resolution subsystem to findan appropriate solution. Check-Cache has never seenthis problem and context before, so it offers no solu-tion. The prior planning activity, however, returnedthree different plans, so two potentially viable plans re-main for Alternate-Plan to examine. In this case, theplan { Set-Parameters, Track-Low, Send-Results }both avoids the failed method and still fulfills relatedcommitment criteria. This schedule is offered as a so-lution. Since a solution was offered at a lower level,Regenerate-Plans is not invoked. Because only one so-lution is provided, the execution subsystem will instan-tiate the Alternate-Plan solution. If multiple solutionswere provided, they would be discriminated throughtheir respective expected qualities (which can be ob-tained from the task structure). Note that if this prob-lem were seen again, Check-Cache would immediatelyrecognize the context and provide this same solution,avoiding further search.

Optimizations

The high-level technologies discussed above address thefundamental issues needed to run in real-time. Unfor-tunately, even the best framework will fail to work inpractice if it does not obtain the processor time neededto operate, or if activity expectations are repeatedly notmet. A good example of this is the execution subsys-tem. It may be that planning and scheduling have suc-cessfully completed, and determined that a particularmethod must run at a particular time in order to meetits deadline. If, however, some other aspect of the agenthas control of the processor when the assigned start timearrives, the method will not be started on time and maytherefore fail to meet its deadline. In this section wewill cover a pair of techniques which aim to reduce theoverhead of the system, to avoid such situations.

Plan Caching

An issue affecting the agent’s real time performance isthe significant time that meta-level tasks such as plan-ning and scheduling can take themselves. In systemswhich run outside of real-time, the duration performanceof a particular component will generally not affect thesuccess or failure of the system as a whole - at worst itwill make it slow. In real time, this slowdown can becritical, for the reasons cited previously. Complicatingthis issue is the fact that these meta-level activities maybe randomly interspersed with method executions. Newgoals, commitments and negotiation sessions may occurat any time during the agent’s lifetime, and each of thesewill require some amount of meta-level attention fromthe agent in a timely manner. To address this, SRTAattempts to optimize the meta-level activities performedby the agent.

One particular computationally expensive process forour agents is planning, primarily because the DTC plan-ner runs as a separate process, and requires a pair ofdisk accesses to use. Unfortunately, this is an artifactcaused by DTC’s C++ implementation; the remainderof the architecture is in Java. We noticed during ourscenarios that a large percentage of the task structuressent to DTC were similar, often differing in only theirstart times and deadlines, and resulting in very similarplan selections. This is made possible by the fact thatDTC is now used on only one goal at a time, as opposedto our previous systems which manipulated structurescombining all current goals. To avoid this overhead, aplan caching system was implemented, shown as a by-pass flow in Figure 2. Each task structure to be sent toDTC is used to generate a key, incorporating several dis-tinguishing characteristics of the structure. If this keydoes not match one in the cache, the structure is set toDTC, and the resulting plan read in, and added to thecache. If the key does match one seen before, the plan issimply retrieved from the cache, updated to reflect anytiming differences between the two structures (such asexpected start times), and returned back to the caller.This has resulted in a significant performance improve-ment, which leaves more time for low-level activities,and thus increases the likelihood that a given deadline

62

Component Average Calls Execution TimeDTC Scheduler 72.14 300 ms

DTC Cache 31.12 74 msPO Scheduler 531.03 36 ms

Table 1: Average results from 1077 runs of 180 seconds.

1

0.8

06

0.4

0.7

o20 40 60 80

Time Granularity (msec)

0 Ralio ~ se~ d~xs Io lime¯ ~ Redo ol ~ior~ 1o resd~dU~g eve~s

Figure 7: The effect of varying time granularity onagent behavior. A higher time ratio indicates that agreater percentage of sequential time units are seen,which should reduce the need for rescheduling. A higheraction ratio indicates the available time was used moreefficiently.

or constraint will be satisfied. Quantitative effects ofthis system can be seen in Table 1.

To test the caching subsystem, we performed 1077runs in simulation using eight sensors and one target. Asshown in the table, the caching system was able to avoidcalling DTC 30% of the time, resulting in a significantsavings in both time and computational power.

Time Granularity

The standard time granularity of agents running in ourexample environment is one millisecond, which dictatesthe scale of timestamps, execution statistics and com-mitments. Because we run in a conventional (i.e. notreal-time) operating system, in addition to our rela-tively unpredictable activity durations, it becomes al-most impossible to perform a given activity at preciselyits scheduled time. For instance, some action X may bescheduled for time 1200. When the agent first checksits schedule for actions to perform, it is time 1184. Inthe subsequent cycle, 24 milliseconds have passed, andit is now time 1208. To maintain schedule integrity (es-pecially with respect to predicted resource usage), wemust shift or reschedule the method which missed itsexecution time before performing it. Despite existingoptimizations, in large number these actions can stillconsume a significant portion of agents’ operating time.

To compensate for this, we scale the agents’ time gran-ularity by some fixed amount. This theoretically tradesoff prediction and scheduling accuracy for responsive-ness (Durfee & Lesser 1988), but in practice a suitablychosen value has few drawbacks, because the agent iseffectively already operating at a lower granularity due

to the real time missed between agent activity cycles.Using this scheme, if we say that every agent tick cor-responds to 20 milliseconds, the above action would bemapped to run at time 60. At time 1184, the agentwould operate as if it were time 59, while 1208 wouldbecome 60, the correct scheduled time for X, thus avoid-ing the need to shift the action. Clearly we can noteliminate the need for rescheduling, due to the inherentuncertainty in action duration in this environment, butthe hope is to reduce the frequency it is needed. Exper-imentation can find the most appropriate scaling factorfor an agent running on a particular system, by search-ing for the granularity which optimizes the number ofactions which are able to be performed against the num-ber of rescheduling events which must take place. Ourexperiments, the results of which can be seen in Figure7, resulted in a 35% reduction in the number of shiftedor rescheduled activities by using a time granularity be-tween 40 and 60 ms. Ideally, the system should "see"each sequential time click, but as the graph shows, asthe system reaches that point, the coarse timeline un-necessarily restricts the number of actions which maytake place, reducing the overall efficiency.

Adapting to Environmental Conditions

An agent’s ability to adapt to changing conditions isessential in an unpredictable environment. SRTA sup-ports this notion with T/EMS , which provides a rich,quantitative language for modeling alternative plans,and DTC and the partial order scheduler, which canreason about those alternatives. As discussed previ-ously, this combination can also make use of activity andresource constraints in addition to results of completedactions, providing the necessary context for analysis anddecision making.

Consider the model shown in Figure 8, where a vari-ety of strict and flexible options are encoded. BecauseGoal has a seq_sum QAF, it will succeed (e.g. accruequality) if all of its subtasks are completed in sequence.The quality it does accrue will be the sum of the qual-ities of it’s subtasks. The structure indicates that Dmust be performed for Task2 to succeed, and also thatthe agent cannot execute E after F. Task:t and Task2have slightly more flexible satisfaction criteria. Theirsum QAFs specify that they will obtain more qualityas more subtasks are successfully completed - withoutany ordering constraints. Finally, the facilitates rela-tionships between A, B and C model how the agent canimprove C’s performance through the successful priorcompletion of A or B. Specifically, A will augment C’squality by 25%, while B will both increase C’s quality by75% and reduce its cost by 50%.

There are several other classes of alternatives whichare not shown in the figure. Resource interrelationships,for example, may be used to model a variety of effects onboth the resources and the activities using them. Thepresence or absence of nonlocal activities, as discussedin the previous section, can indicate alternative meansof accomplishing a task. Multiple outcomes on methodsmay indicate alternative solutions which may arise from

63

( ) (¯ ) ou~om~OO¢0%), o~4~me(,O00%). outcor~ (1~0%).O: [2.0.1.0] O: [1.0.1.0] O: (1~0.1.0] O: [5.0.1.01 O: I,O.0. ,.0]C: [0.0.1.0] C: [0.0., .0] C: [10.0, 1.0] C; [0.0,1.01 C: [2.0. 1.0]D:[IS 0.1.0] ~ [,S.0.1.0] ~ (2S.o. 1.0] D: (S.0.1.0] D: [10.0.1.0]

Figure 8: A T/EMS task structure modeling several dif-ferent ways to achieve the same goal.

Conditions Schedule Q c D1 Unconstrained ABCDEF 49.9 7.0 90.02 Deadline 40 ADE 17.0 0.0 40.03 Deadline 50 ADEF 27.0 2.0 50.04 Deadline 76 BCDEF 43.5 7.0 75.05 Cost 3 ABDEF 28.0 2.O 65.06 Balanced ADEF 27.0 2.0 50.0

Table 2: A variety of schedules, and their expectedqualities, costs and durations, generated from theT/EMS structure in Figure 8 under different conditions.

a method’s execution, so the probability densities asso-ciated with each outcome provide an additional sourceof discriminating information which can help control theuncertainty of generated plans. The individual proba-bility distributions for the quality, cost and duration ofeach outcome serve in the same capacity, as do analo-gous probabilities modeling the quantitative effects ofinterrelationships. The available time, desired qualityand cost, along with other execution constraints pro-vide the context in which to generate and evaluate thealternative plans such a structure may produce.

To demonstrate how the system adapts to varyingconditions, several plans derived from the task struc-ture in Figure 8 are shown in Table 2. These plansare produced for different environmental conditions thatplace different resource constraints on the agent. Asone would expect, when the agent is completely uncon-strained and has a goal to maximize quality, the planshown in row one is produced. Note that the selectedplan has an expected quality of 49.9, expected cost of7.0, and an expected duration of 90.0. The qualityshown is not a round integer even though the qualitiesshown in Figure 8 are integers because A and B ]acili-tare method C and increase C’s quality when they areperformed before it.

Row two shows the plan selected for the agent undera hard deadline of 40 seconds. This path through thenetwork has the shortest duration enabling the agent toperform each of the major subtasks. Note the differencein quality, cost, and duration between rows one and two.

Row three shows the plan selected for the agent if itis given a slightly more loose deadline of 50 seconds.This case illustrates an important property of schedul-ing and planning with T/EMS - optimal decisions madelocally to a task do not combine to form decisions thatare optimal across the task structure. In this case, the

agent selected methods ADEF. If the agent were plan-ning by simply choosing the best method at each node,it would select method C for the performance of Task 1because C has the highest quality. It would then select Das there is no choice to be made with respect to methodD. It would then select method E because that is theonly method that would fit in the time remaining to theagent. The plan CDE has an expected quality of 25, costof 10, and duration of 50. Planning with T/EMS requiresstronger techniques than simple hill climbing or localdecision making. This same function holds when tasksspan agents and the agents work to coordinate their ac-tivities, evaluate cross agent temporal constraints, anddetermine task value.

Row four shows the plan produced if the agent is givena hard deadline of 76 seconds. What is interesting aboutthis choice is that DTC selected BCDEF over ACDEF eventhough method B has a lower quality than method A andthey both require the same amount of time to perform.The reason for this is that B’s facilitation effect (75%quality multiplier) on method C is stronger than that ofmethod A (which has a 25% quality multiplier). Thenet result is that BCDEF has a resultant expected qualityof 43.5 whereas ACDEF has an expected quality of 39.5.

Row five shows the plan produced by DTC if the agenthas a soft preference for schedules whose cost is underthree units. In this case, schedule ABDEF was selectedover schedules like ADEF because it produces the mostquality while staying under the cost threshold of threeunits. DTC does not, however, deal only in specificconstraints. The "criteria" aspect of Design-to-Criteriascheduling also expresses relative preferences for quality,cost, duration, and quality certainty, cost certainty, andduration certainty. Row six shows the plan producedif the scheduler’s function is to balance quality, cost,and duration. Consider the solution space representedby the other plans shown in Table 2 and compare theexpected quality, cost, and duration attributes of theother rows to that of row six. Even though the solutionspace represented by the table is not a complete space,it shows how the solution in row six falls relative to therest of the possible solutions - a good balance betweenmaximizing quality while minimizing cost and duration.

Note these examples do not illustrate DTC’s abilityto trade-off certainty against quality, cost, and duration.The examples also omit the quality, cost, and durationdistributions associated with each item that is sched-uled/planned for and the distributions that representthe aggregate behavior of the schedule/plan.

Conclusion

The SRTA architecture has been designed to facilitatethe construction of sophisticated agents, working in soft-real time environments possessing complex interactionsand a variety of ways to accomplish any given task.With T/EMS , it provides domain independent mecha-nisms to model and quantify such interactions and alter-natives. DTC and the partial ordered scheduler reasonabout these models, using information from the resourcemodeler and the runtime context to generate, rank and

select from a range of candidate plans and schedules.An execution subsystem executes these actions, track-ing performance and rescheduling or resolving conflictswhere appropriate. The engine is capable of real-timeresponsiveness, allowing these techniques to analyze andintegrate solutions to dynamically occurring goals.

SRTA’s objective is to provide domain independentfunctionality enabling the relatively quick and simpleconstruction of agents and multi-agent systems capa-ble of exhibiting complex and applicable behaviors. It’sability to adapt to different environments, respond tounexpected events, and manage resource and activity-based interactions allow it to operate successfully in awide range of conditions. We feel this type of system canform a reusable foundation for agents working in real-world environments, allowing designers to focus theirefforts on higher-level issues such as organization, nego-tiation and domain dependent problems.

More generally, the significance of the work presentedin this paper comes from its demonstration that it is pos-sible to perform in soft real-time the complex modeling,planning and scheduling that has been described in ourprior research. Previously, these techniques were ana-lyzed only in theory or simulation, and it was not clearthat our heuristic approach would be sufficiently respon-sive and flexible to address real-world problems. TheSRTA architecture shows that engineering can be usedto combine and streamline these approaches to make aviable, coherent solution.

References

Bordini, R.; Bazzan, A.; Jannone, R.; Basso, D.; Vi-cari, R.; and Lesser, V. 2002. Agentspeak(xl): Efficientintention selection in bdi agents via decision-theoretictask scheduling. In Proceedings of the First Interna-tional Joint Conference on Autonomous Agents andMulti-Agent Systems (AAMAS-2002).

Decker, K. S., and Lesser, V. R. 1993. Quantitativemodeling of complex environments. International Jour-nal of Intelligent Systems in Accounting, Finance, andManagement 2(4):215-234. Special issue on "Mathe-matical and Computational Models of Organizations:Models and Characteristics of Agent Behavior".

Durfee, E., and Lesser, V. 1988. Predictability vs.responsiveness: Coordinating problem solvers in dy-namic domains. In Proceedings of the Seventh NationalConference on Artificial Intelligence, 66-71.

Horling, B.; Benyo, B.; and Lesser, V. 2001. Usingself-diagnosis to adapt organizational structures. InProceedings of the Fifth International Conference onAutonomous Agents, 529-536.

Horling, B.; Lesser, V.; Vincent, R.; Raja, A.;and Zhang, S. 1999. The taems white paper.http: / /mas.cs.umass.edu /res-earch /taems /white/.

Horling, B.; Vincent, R.; Mailler, R.; Shen, J.; Becker,R., Rawlins, K.; and Lesser, V. 2001. Distributedsensor network for real time tracking. In Proceedingsof the Fifth International Conference on AutonomousAgents, 417-424.

Horling, B.; Lesser, V.; Vincent, R.; and Wagner, T.2002. The soft real-time agent control architecture.Computer Science Technical Report TR-02-14, Univer-sity of Massachusetts at Amherst.

Jensen, D.; Atighetchi, M.; Vincent, R.; and Lesser,V. 1999. Learning quantitative knowledge for multia-gent coordination. In Proceedings of the Sixteenth Na-tional Conference on Artificial Intelligence. Orlando,FL: AAAI.Lesser, V.; Atighetchi, M.; Horling, B.; Benyo, B.;Raja, A.; Vincent, R.; Wagner, T.; Xuan, P.; andZhang, S. X. 1999. A Multi-Agent System for In-telligent Environment Control. In Proceedings of theThird International Conference on Autonomous Agents(Agents99).Lesser, V.; Horling, B.; Klassner, F.; Raja, A.; Wag-ner, T.; and Zhang, S. X. 2000. BIG: An agent forresource-bounded information gathering and decisionmaking. Artificial Intelligence 118(1-2):197-244. Else-vier Science Publishing.Lesser, V. R. 1991. A retrospective view of FA/C dis-tributed problem solving. IEEE Transactions on Sys-tems, Man, and Cybernetics 21(6):1347-1363.

Lesser, V. R. 1998. Reflections on the nature of multi-agent coordination and its implications for an agent ar-chitecture. Autonomous Agents and Multi-Agent Sys-tems 1(1):89-111.

Mailler, R.; Vincent, R.; Lesser, V.; Shen, J.; and Mid-dlekoop, T. 2001. Soft real-time, cooperative negotia-tion for distributed resource allocation. In Procceedingsof the 2001 AAAI Fall Symposium on Negotiation.

Stankovic, J. A., and Ramamritham, K. 1990. Edito-rial: What is predictability for real-time systems? TheJournal of Real- Time Systems 2:247-254.

Wagner, T., and Lesser, V. 2000. Design-to-CriteriaScheduling: Real-Time Agent Control. In Rana, O.,and Wagner, T., eds., Infrastm~cture for Large-ScaleMulti-Agent Systems, Lecture Notes in Computer Sci-ence. Springer-Verlag, Berlin. A version also appearsin the 2000 AAAI Spring Symposium on Real-TimeSystems and as UMASS CS TR-99-58.

Wagner, T.; Garvey, A.; and Lesser, V. 1997. ComplexGoal Criteria and Its Application in Design-to-CriteriaScheduling. In Proceedings of the Fourteenth NationalConference on Artificial Intelligence, 294-301. Alsoavailable as UMASS CS TR-1997-10.Wagner, T.; Garvey, A.; and Lesser, V. 1998. Criteria-Directed Heuristic Task Scheduling. InternationalJournal of Approximate Reasoning, Special Issue onScheduling 19(1-2):91-118. A version also available UMASS CS TR-97-59.Zhang, X.; Raja, A.; Lerner, B.; Lesser, V.; Oster-weil, L.; and Wagner, T. 2000. Integrating high-leveland detailed agent coordination into a layered archi-tecture. Lecture Notes in Computer Science: Infras-tructure for Scalable Multi-Agent Systems. Springer-Verlag, Berlin. 72-79. Also available as UMass Com-puter Science Technical Report 1999-029.

65