Wagner Aaai Real Time

8/14/2019 Wagner Aaai Real Time

1/10

Design-to-Criteria Scheduling: Real-Time Agent Control

Thomas Wagner

Computer Science DepartmentUniversity of MaineOrono, ME 04469

[email protected]

Victor Lesser

Computer Science DepartmentUniversity of MassachusettsAmherst, MA [email protected]

Abstract

Design-to-Criteria builds custom schedules for agents thatmeet hard temporal constraints, hard resource constraints,and soft constraints stemming from soft task interactions orsoft commitments made with other agents. Design-to-Criteriais designed specifically for online application it copes withexponential combinatorics to produce these custom schedulesin a resource bounded fashion. This enables agents to re-spond to changes in problem solving or the environment asthey arise.

Introduction

Complex autonomous agents operating in open, dynamic en-vironments must be able to address deadlines and resourcelimitations in their problem solving. This is partly due tocharacteristics of the environment, and partly due to thecomplexity of the applications typically handled by softwareagents in our research. In open environments, requests forservice can arrive at the local agent at any time, thus makingit difficult to fully plan or predict the agents future work-load. In dynamic environments, assumptions made when

planning may change, or unpredicted failures may occur1.In most real applications, deadlines or other time constraints

UMass Computer Science Technical Report 1999-58. A ver-sion appears in the Proceedings of 2000 AAAI Spring Symposiumon Autonomous Real-Time Systems.

This material is based upon work supported by the NationalScience Foundation under Grant No. EEC-9209623, and by theNational Science Foundation under Grant No.s IRI-9523419 andIRI-9634938, the Department of the Navy and Office of the Chiefof Naval Research, under Grant No. N00014-95-1-1198. Thiswork is also funded by the National Defense Center of Excellencefor Research in Ocean Sciences (CEROS). CEROS is a part ofthe Natural Energy Laboratory of Hawaii Authority (NELHA), anagency of the Department of Business, Economic Development &

Tourism, State of Hawaii. CEROS is funded by the Defense Ad-vanced Research Projects Agency (DARPA) through grants to andagreements with NELHA. The content of the information does notnecessarily reflect the position or the policy of the Government, theNational Science Foundation, or CEROS, and no official endorse-ment should be inferred.

1This differs from states that are explicitly recognized andplanned for [1] as software agents may be required to perform adifferent set of tasks, as well as having to react to changes in theenvironment.

Figure 1: Modeling and Online Scheduling for Real Time andResource Boundedness

are present on the agents problem solving [16, 8]. For ex-ample, in an anti-submarine warfare information gatheringapplication [3], there is a deadline by which the missionplanners require the information. Resource limitations mayalso stem from agents having multiple different tasks to per-form and having bounded resources in which to performthem. Temporal constraints may also originate with agentinteractions in general, in order for agent

to coordinate

with agent , the agents require mutual temporal informa-tion so that they can plan downstream from the interaction.

In this paper, we focus on the issue of resource boundedagent control. We use the term resource boundedto denote

the existence of deadlines and of other constraints like costlimitations or application specific resource limitations (e.g.,limited network bandwidth). Where it is important to differ-entiate hard and soft deadlines from these other constraints,we refer to them explicitly.

For agents to adapt rationally to their changing problemsolving context, which includes changes in the environment2

and changes to the set of duties for the agent to perform, theymust be able to:

1. Represent or model the time and resource constraints ofthe situation and how such constraints impact their prob-lem solving. We believe this must be done in a quantifiedfashion as different constraints have different degrees of

effect on problem solving.2. Plan explicitly to address the resource limitations. In our

work, this may imply performing a different set of tasks,using alternate solution methods, or trading-off differentresources (or quality), depending on what is available.

2Including resources uncontrollably becoming more or lessconstrained. For example, network latency increasing due to someactivity other than the agents problem solving.


2/10

3. Perform this planning online in the general case, thisimplies coping with exponential combinatorics online insoft real time.

While the first two requirements obviously follow fromthe domain, the third requirement is less obvious. Agentsmust be able to perform real time control problem solvingonline because of the dynamics of the environment. If itis difficult to predict the future and there is a possibility of

failure, or new tasks arriving, agents will, by necessity, haveto react to new information and replan online.

The Design-to-Criteria (DTC) agent scheduler and theTMS task modeling framework are our tools for address-ing these requirements and achieving resource-boundedagent control (Figure 1). TMS provides agents with theframework to represent and reason about their problem solv-ing process from a quantified perspective, including mod-eling of interactions between tasks and resource consump-tion properties. Design-to-Criteria performs analysis of theprocesses (modeled in TMS) and decides on an appro-priate course of action for the agent given its temporaland resource constraints. Design-to-Criteria both producesresource-aware schedules for the agent, and, does this rea-

soning process online in a resource bounded fashion.While the output of Design-to-Criteria is real time in the

sense that the schedules address hard and soft deadlines, andresource constraints, the schedules are not hard real time andare not fault tolerant in the sense that they may contain un-certainty and known potential failure points. Because DTCis applied in domains where failure is expected, and mod-eled, and rescheduling is expected, it may often be prudentto choose a schedule that contains some probability of fail-ure, but, also some probability of higher returns. The issueof uncertainty, and its role in addressing hard deadlines, iscovered in greater detail later. For situations in which a mid-stream schedule failure leads to catastrophic system-widefailure, we have developed an offline variant of DTC thatuses contingency analysis [17, 23] to explore and evaluaterecovery options from possible failure points.

This paper is organized as follows: in Section we presentTMS and describe its role in our domain independent ap-proach to agent control. In Section we describe how DTCreasons about the agents context and makes control deci-sions to produce resource bounded schedules. In Section ,DTCs approximate online solution strategy is presented andin Section we discuss limitations, open questions, and futurework.

TMS Task Models

TMS (Task Analysis, Environment Modeling, and Sim-

ulation) [6] is a domain independent task modeling frame-work used to describe and reason about complex problemsolving processes. TMS models are used in multi-agentcoordination research [24, 11] and are being used in manyother research projects, including: cooperative-information-gathering [14], hospital patient scheduling [5], intelligentenvironments [13], coordination of software process [12],and others [20]. Typically, in our domain-independent agentarchitecture, a domain-specific problem solver or planner

translates its problem solving options in TMS, possibly atsome level of abstraction, and then passes the TMS mod-els on to agent control problem solvers like the multi-agentcoordination modules or the Design-to-Criteria scheduler.The control problem solvers then decide on an appropriatecourse of action for the agent, possibly by coordinating andcommunicating with other agents (that also utilize the samecontrol technologies).

TMS models are hierarchical abstractions of problemsolving processes that describe alternative ways of accom-plishing a desired goal; they represent major tasks and ma-jor decision points, interactions between tasks, and resourceconstraints but they do not describe the intricate details ofeach primitive action. All primitive actions in TMS, calledmethods, are statistically characterized via discrete probabil-ity distributions in three dimensions: quality, cost and dura-tion. Quality is a deliberately abstract domain-independentconcept that describes the contribution of a particular actionto overall problem solving. Duration describes the amountof time that the action modeled by the method will take toexecute and cost describes the financial or opportunity costinherent in performing the action. Uncertainty in each of

these dimensions is implicit in the performance characteriza-tion thus agents can reason about the certainty of particularactions as well as their quality, cost, and duration trade-offs.The uncertainty representation is also applied to task inter-actions likeenablement, facilitationand hinderingeffects, 3

e.g., 10% of the time facilitation will increase the qualityby 5% and 90% of the time it will increase the quality by8%.

The quantification of actions and interactions in TMSis not regarded as a perfect science. Task structure program-mers or problem solver generatorsestimatethe performancecharacteristics of primitive actions. These estimates can berefined over time through learning and reasoners typicallyreplan and reschedule when unexpected events occur.

To illustrate, consider Figure 2, which is a conceptual,simplified sub-graph of a task structure emitted by the BIG[14] resource bounded information gathering agent; it de-scribes a portion of the information gathering process. Thetop-level task is to construct product models of retail PC sys-tems. It has two subtasks, Get-Basicand Gather-Reviews,both of which are decomposed into actions, that are de-scribed in terms of their expected quality, cost, and du-ration. The enables arc between Get-Basic and Gatheris a non-local-effect (NLE) or task interaction; it modelsthe fact that the review gathering actions need the namesof products in order to gather reviews for them. Othertask interactions modeled in TMS include: enablement,facilitation, hindering, bounded facilitation, disablement,

consumes-resource and limited-by-resource. Task interac-tions are important to scheduling because they denote pointsat which a task may be affected, either positively or nega-

3Facilitation and hindering task interactions model soft rela-tionships in which a result produced by some taskmay be beneficialor harmful to another task. In the case of facilitation, the existenceof the result generally increases the quality of the recipient task orreduces its cost or duration.


3/10

Figure 2: Conceptual Information Gathering Task Structure of the BIG Agent

tively, by an outcome elsewhere in the task structure (or atanother agent).

Returning to the example, Get-Basic has two ac-tions, joined under thesum()quality-accumulation-function

(QAF), which defines how performing the subtasks relateto performing the parent task. In this case, either action orboth may be employed to achieve Get-Basic. The same istrue for Gather-Reviews. The QAF forBuild-PC-Product-Objectsis aseq last() which indicates that the two subtasksmust be performed, in order, and that the quality ofBuild-PC-Product-Objectsis determined by the resultant quality ofGather-Reviews. There are nine alternative ways to achievethe top-level goal in this particular sub-structure.4 In gen-eral, a TMS task structure represents a family of plans,rather than a single plan, where the different paths throughthe network exhibit different statistical characteristics ortrade-offs. The process of deciding which tasks/actions toperform is thus anoptimizationproblem rather than a satis-

factionproblem.

TMS also supports modeling of tasks that arrive at par-ticular points in time, parallelism, individual deadlines ontasks, earliest start times for tasks, and non-local tasks (thosebelonging to other agents). In the development of TMSthere has been a constant tension between representationalpower and the combinatorics inherent in working with thestructure. The result is a model that is non-trivial to pro-cess, coordinate, and schedule in any optimal sense (in thegeneral case), but also one that lends itself to flexible and ap-proximate processing strategies. This element of choice andflexibility is leveraged both in designing resource-boundedschedules for agents and in performing online scheduling in

a resource bounded fashion.

4While it might appear per the seq last() QAF that there areonly two possible resultant quality distributions, the enables inter-action betweenBuildandGatheraffects the possible quality valuesforGather.

Modeling and Reasoning about Temporal and

Resource Constraints

TMS tasks may have both soft and hard constraints thatmust be considered when scheduling. In terms of hard tem-

poral constraints, any TMS task may have a hard dead-line, by which some quality must be produced (or it is con-sidered a failure), as well as an earliest-start-time, beforewhich the task may not be performed (or zero quality willresult). These hard constraints may also be caused by hardcommitments5 made with other agents or hard delays be-tween task interactions. The constraints may also be inher-ited from nodes higher in the structure thus a client mayspecify a hard deadline on theBuild-PCtask that applies toall subtasks, or a deadline may be specified on the processof Gathering-Reviews. If multiple temporal constraints arepresent, the tightest or most conservative interpretation ap-plies.

Recall that actions in TMS are characterized using dis-

crete probability distributions. Because durations may beuncertain, and because actions are sequenced in a linearfashion,6 the implication of duration uncertainty is that thereis generally uncertainty in both the start and finish times oftasks even tasks that do not have duration uncertainty oftheir own. When each TMS action is added to a schedule,or considered for a particular schedule point, a data structurecalled aschedule elementis created and the start, finish, andduration distributions for the schedule element are computedas a function of the characteristics of the previous scheduleelement and the action being scheduled. The constraints as-sociated with the action (and higher level task) are then ex-amined and compared to the characteristics that will result ifthe action is performed at the current time or point in the

schedule.One approach for determining whether or not a given

action will violate a hard deadline, for example, is to look

5In contrast to commitments that are soft or relaxable, possiblythrough a decommitment penalty mechanism.

6While DTC supports scheduling of specialized parallel activi-ties, even when activities are scheduled in parallel, they may inherituncertainty from prior activities.


4/10

Figure 3: Reflecting Probability of Missing Deadline in Method Quality

at some single statistic (median, mean, max, min) of theaction and to compare that statistic to the deadline, e.g.,

. This approach is used during some of theapproximation processes of the scheduling algorithm.Another reasonable approach is to compute the proba-bility that the action will violate its hard constraint andcompare the probability to a predetermined threshold, e.g.,

.

However, TMS provides us with a better tool for rea-soning about constraint violation. Because zero quality re-flects failure, and in TMS an action that violates its harddeadline produces zero quality, we can reason about theprobability that a given action violates its hard deadline sim-ply by reflecting said probability in the quality distributionof the action and then treating it like any other TMS ac-tion.7 Enforcing hard deadline constraints on the agents en-tire process (analogous to imposing a deadline on the taskstructure root) is handled in the same way. For example,as shown in Figure 3, if

has a 10% chance of exceed-

ing its deadline (and thus failing), the densities of all themembers of its quality distribution are multiplied by 90%(thus re-weighting the entire distribution) and a new den-

sity / value pair is added to the distribution to reflect the10% chance of returning a result after the deadline. Theleftmost histogram describes

s expected finish time, the

middle histogram describes

s unmodified quality distri-bution, and the rightmost figure shows the modified qual-ity distribution after re-weighting and merging with the new

pair. Through this solution approach, thescheduler may actually select a course of action for the agentthat has some probability of failure, however, the probabil-ity of failure is reflected directly in solution quality so that ifthe risk is not worthwhile (relative to the other solution pathsavailable to the agent) it will not be taken. In other words,a path containing a possible deadline violation will only bechosen if it has a higher quality than the other solutions onan expected value basis.

On the surface, this model is not appropriate for hard realtime applications in which the failure of the action results inno solution for the agent. However, if this is the case, the ac-

7Professor Alan Garvey, developer of a forerunner to Design-to-Criteria, Design-to-Time, first used a similar technique inDesign-to-Time. The technique presented here was developed in-dependently in the DTC research.

tion will serve a key role in the task structure or will interactwith (e.g., enable) other actions in the structure and thus thefailure will result in the quality of the affected actions alsobeing decreased and further lower solution quality. The viewpresented here, if modeled appropriately, gives the schedulera very powerful tool for reasoning about the implications ofpossible failures and their impact on overall problem solv-ing.

In addition to hard temporal constraints, TMS alsomodels hard resource constraints. For example, a given taskmay require the use of a network connection and withoutthis connection, the task may produce zero quality (fail).In TMS, the effects of resource constraints are modeledusing a limits NLE from the resource to the task wherethe NLE describes a multiplier relationship between the re-source and the task. For example, running out of a resourcemay cause the task to take 1.5 times as long to execute, or itmay cause the quality to decrease by 50%, or it may causethe cost to increase, or it may simply cause failure. As withviolating a hard temporal constraint, if a resource constraintcauses action failure, it is reflected in the quality of the ac-tion and any actions or tasks that are acted-upon (e.g., by anenables from the affected action) will also have their quali-ties adjusted to reflect the effects of the resource problem.

Soft constraints in TMS take the form of soft commit-ments made with other agents and soft interactions betweentasks. For example, if task

facilitates , performing before

will positively affect

, possibly by shorting

s

duration, but the facilitation does not need to be leveraged toperform either task. When scheduling for soft constraints as-sociated with actions, the scheduler attempts to utilize themwhen possible (or avoid the in the case of a soft negativeinteraction, e.g., hinders). However, whenever a soft con-straint is violated, either on the positive or negative side, thequality distributions of the involved actions are modified toreflect the situation and thus the scheduler can again rea-son directly about the impact of constraint violation on the

agents process.The scheduler also supports soft constraints on overall

problem solving. In addition to setting hard temporal con-straints, the scheduler client may specify an overall softdeadline, soft cost limit, or soft quality requirement. Thesesoft constraints are members of a package of client prefer-ences calleddesign criteria that describes for the schedulerthe clients objective function. The scheduler then works toproduce a schedule (or set of schedules) to suit the clients


5/10

Figure 4: Different Schedules for Different Clients

needs. The criteria mechanism is soft because, due to the

combinatorics of reasoning about TMS task structures, itis often difficult to predict what types of solutions are possi-ble. Instead, the client describes the desired solution spacein terms of relaxable, relative, design criteria (in quality,cost, duration, uncertainty in each dimension, and limits andthresholds on these) and the scheduler makes trade-off deci-sions as needed to best address the clients needs. The crite-ria metaphor is based on importance sliders for quality, cost,duration, limits and thresholds on these, and certain in eachof these dimensions. The metaphor, the formal mathemat-ics of the criteria mechanism, and the schedulers trade-offcomputations have been fully documented in [22, 21].

Let us revisit BIGs process, shown in Figure 2, and illus-

trate DTCs creation of custom, resource bounded, sched-ules and the role of task interaction in modeling the effectsof failure. Even this simple task structure givesDTC room toadapt BIGs problem solving. Figure 4 shows three differentschedules constructed for different BIG clients that have dif-ferent objectives. For brevity, the detailed distributions asso-ciated with each action are omitted, however, the aggregateschedule statistics are shown. Schedule A is constructedfor a client that has both time and financial resources heor she is simply interested in maximizing overall solutionquality. Schedule B is constructed for a client that wantsa free solution. Schedule C meets the needs of a client in-terested in maximizing quality while meeting a hard dead-line of 6 minutes. Note that schedule C is actually preferred

over a schedule that includes action Query-and-Extract-PC-Connectioneven though said action has a higher expectedquality than Query-and-Extract-PC-Mall. This is becausethe PC-Connection action also has a higher probability offailure. Because of the enables NLE from the task of gettingproduct information to retrieving reviews, this higher proba-bility of failure also impacts the probability of being able toquery the Consumers site for a review. Thus, though the lo-cal choice would be to prefer PC-ConnectionoverPC-Mall

for this criteria, the aggregate effects lead to a different de-

cision. Note also that schedule C also exceeds its deadline10% of the time. The deadline over-run and the enablementfromPC-Mallcontribute to the probability of failure exhib-ited by the schedule (probability of returning a zero qualityresult), i.e., Consumers fails 25% of the time without con-sidering these other constraints. When considering the otherconstraints, probability of failure is: (((25% * .90) + 10%) *.90) + 10% = 39.25%.

Online Scheduling - Coping with Exponential

Combinatorics

As TMS task structures model a family of plans, theDTC scheduling problem has conceptually certain charac-teristics in common with planning and certain characteris-tics of more traditional scheduling problems, and it suffersfrom pronounced combinatorics on both fronts. The sched-ulers function is to read as input a TMS task structure (ora set of task structures) and to 1) decide which set of tasks toperform, 2) decide in what sequence the tasks should be per-formed, 3) to perform the first two functions so as to addresshard constraints and balance the soft criteria as specified bythe client,8 and 4) to do this computation in soft real time (orinteractive time) so that it can be used online.

Meeting these objectives is a non-trivial problem. In gen-eral, the upper-bound on the number of possible schedulesfor a TMS task structure containing actions is given inEquation 1. Clearly, for any significant task structure thebrute-strength approach of generating all possible schedulesis infeasible offline or online. This expression containscomplexity from two main sources. On the planning side,the scheduler must consider the (unordered)

different

8Because there may be alternative ways to perform a given task,and some of the options may not have the same associated dead-lines, the scheduler actually balances both meeting hard constraintsand the design criteria.


6/10

alternative different ways to go about achieving the top leveltask (for a task structure with actions). On the schedul-ing side, the scheduler must consider the

different pos-

sible orderings of each alternative, where is the numberof actions in the alternative.

(1)

In general, the types of constraints present in TMS, andthe existence of interactions between tasks (and the differ-ent QAFs that define how to achieve particular tasks), pre-vent a simple, optimal solution approach. DTC copes withthe high-order combinatorics using a battery of techniques.Space precludes detailed discussion of these, however, theyare documented in [22]. From a very high level, the sched-uler uses:

Criteria-Directed Focusing The clients goal criteria is notsimply used to select the best schedule for execution,but is also leveraged to focus all processing activities onproducing solutions and partial solutions that are mostlikely to meet the trade-offs and limits/thresholds defined

by the criteria.Approximation Schedule approximations, called alterna-

tives, are used to provide an inexpensive, but coarse,overview of the schedule solution space. One alternativemodels one way in which the agent can achieve the toplevel task. Alternatives contain a set of unordered actionsand an estimation for the quality, cost, and duration char-acteristics that will result when the actions are sequencedto form a schedule. This, in conjunction with criteria-directed focusing enables DTC to address the planningside complexity.

Heuristic Decision Making To address the scheduling sidecomplexity, DTC uses a superset of the techniques usedin Design-to-Time [8], namely an iterative, heuristic, pro-cess of sequencing out the actions in a given alternative.These action rating heuristics rate each action and the rat-ings (in DTC) are stratified so that certain heuristics andconstraints dominate others. The net effect is a reductionof the

(

and

by Stirlings Approxima-tion) complexity to polynomial levels in the worst case.

Heuristic Error Correction The use of approximation andheuristic decision making has a price it is possible tocreate schedules that are suboptimal, but, repairable. Asecondary set of improvement [27, 19] heuristics act as asafety net to catch the errors that are correctable.

The Design-to-Criteria scheduling process falls into thegeneral area of flexible computation [9], but differs from

most flexible computation approaches in its use of multipleactions to achieve flexibility (one exception is [10]) in con-trast to anytime algorithms [4, 18, 25]. We have found thelack of restriction on the properties of primitive actions tobe an important feature for application in large numbers ofdomains. Another major difference is that in DTC we notonly propagate uncertainty [26], but we can work to reduceit when important to the client. DTC differs from its pre-decessor, Design-to-Time[8], in many ways. From a client

perspective, however, the main differences are in its use ofuncertainty, its ability to retarget processing at any trade-offfunction, and its ability to cope with both scheduling andplanning side combinatorics.

Design-to-Criteria is not without its limitations; whenadapting the DTC technology for use in potentially time crit-ical domains, such as the CEROS anti-submarine warfare in-formation gathering task, shown in Figure 5, we encountered

an interesting problem. The satisficing focusing methodol-ogy used in Design-to-Criteria leads to poor solutions whencombined with hard deadlines and certain classes of verylarge task structures. Without delving into exhaustive de-tail, the problem is that in order to cope with the high-ordercombinatorics in these particular situations, the schedulingalgorithm must prune schedule approximations, or alterna-tives, and develop only a subset of these. Herein lies theproblem.

Alternatives are constructed bottom-up from the leaves ofthe task hierarchy to the top-level task node, i.e., the alter-natives of a task are combinations of the alternatives for itssub-tasks. Figure 6 shows the alternative set generation pro-cess for a small task structure. Alternatives are generated

for the interior tasks

and

, and these alternatives arecombined to produce the alternative set for the root task, .The complexity of the alternative generation process is pro-nounced. A task structure with actions leads to

possible alternatives at the root level. We control this com-binatorial complexity by focusing alternative generation andpropagation on alternatives that are most likely to result inschedules that best satisfice to meet the clients goal crite-ria; alternatives that are less good at addressing the criteriaare pruned from intermediate level alternative sets. For ex-ample, a criteria set denoting that certainty about quality isan important issue will result in the pruning of alternativesthat have a relatively low degree of quality certainty. Afterthe alternative set for the high-level task is constructed, a

subset of the alternatives are selected for scheduling.For situations in which there is no overall hard deadline,

or in which shorter durations are also preferred, the focusingmechanism works as advertised. However, in the CEROSproject, we are also interested in meeting real-time deadlinesand other hard resource constraints (in contrast to those thatare relaxable), and often these preferences are not accompa-nied by a general preference for low duration or low cost.In these cases, the problem lies in making a local decisionabout which alternatives to propagate (at an interior node)when the decision has implications to the local decisionsmade at other nodes the local decision processes are in-terdependent and they interact over a shared resource, e.g.,time or money. Casting the discussion in terms of Figure 6:

assume has an overall deadline of 5 minutes and s al-ternatives require anywhere from 2 minutes to 20 minutes tocomplete, and s alternatives are similarly characterized.Assume that quality is highly correlated with duration, thusthe more time spent problem solving, the better the result.If the criteria specifies maximum quality within the dead-line, the alternatives propagated from

to

will be those

that achieve maximum quality (and also have high duration).Likewise with the alternatives propagated from

. The re-


7/10

Figure 5: Partial TMS Task Structure for Gathering and Processing ASW Mission Information

Figure 6: Alternative Sets Lead to Cumbersome Combinatorics

sulting set of alternatives,

at node will contain mem-

bers characterized by high quality, but also high duration,and the scheduler will be unable to construct a schedule thatmeets the hard deadline. The optimal solution to this prob-lem is computationally infeasible (

and

) as itamounts to the general scheduling problem because of taskinteractions and other constraints.

Two approximate solutions are possible. One approachis to preprocess the task structure, producing small alterna-tive sets at each node that characterize the larger alterna-tive population for that node. Then examining the rangesof alternatives at each node and heuristically deciding on anallocation or apportionment of the overall deadline or costlimitation to each of the interior nodes. This local-view of

the overall constraint could then be used to focus alterna-tive production on those that will lead to a root-level setthat meets the overall constraint. The other approach, whichwe have employed, is to detect when the local-view of thedecision process is problematic and in those cases samplefrom the population of alternatives, producing a subset thatexhibits similar statistical properties, and propagating thesealternatives. This leads to a less-focused set of root level al-ternatives than the prior approach, but it saves on the added

polynomial level expense of the first approach.

Conclusion, Future Work, and Limitations

We have discussed a class of issues in DTC that pertain tomodeling and scheduling for hard and soft temporal con-straints, resource constraints, and task interactions. Spaceprecludes a full enumeration of the different aspects of DTCthat relate to addressing resource limitations the issue isubiquitous to the design of the DTC algorithm, the TMSmodeling framework, and the decisions made by the DTCscheduler. From a very high level, possibly the most im-portant features that relate to resource boundedness is thedetailed quantified view of actions, and task interactions,afforded by the TMS modeling framework. This, com-

bined with the element of choice present in TMS familiesof plans, sets the foundation for DTCs reasoning about theimplications of failures, failing to acquire resources, and vi-olating hard constraints.

In terms of limitations, DTCs approximate solution ap-proach is clearly not optimal in many circumstances. Asdiscussed, this is particularly true when the alternative setsmust be severely pruned (focused) to produce solutions. Ad-ditionally, in some applications, in which only very specific


8/10

subsets of the features afforded by TMS are employed,custom schedulers may do a better job of balancing the dif-ferent concerns and finding good solutions. In terms of opti-mality, it is difficult to compare the performance of DTC tooptimal as found via exhaustive generation simply because itis not feasible to generate all possible schedules for realistictask structures. Members of our group are currently workingon an MDP-based TMS scheduling tool [17, 23] and we

plan to measure DTCs performance on smaller applicationsthrough this tool.

It is important to note that though DTC takes great painsto produce schedules quickly, the scheduler is not hard realtime itself. We cannot make performance guarantees [15]for a given problem instance, though it would be possibleto produce such guarantees by classifying similar task struc-tures and measuring scheduling performance offline. At is-sue is the constraints present in an arbitrary TMS taskstructure. For certain classes of task structures, guaranteeswithout an in-depth preclassification are possible. In prac-tice, the scheduler (implemented in 50,000 lines of C++)is fast and capable of scheduling task structures with 20-40primitive actions in under 7 seconds on a 600mhz Pentium

III machine running Redhat Linux 6.0. A sampling of appli-cations and runtimes are shown in Table 1.

In the table, the first column identifies the problem in-stance, the second column identifies the number of primitiveactions in the task structure, the third column (UB # R-Alts)indicates the upper bound on the number of root-level alter-natives, the fourth column identifies the upper bound on thenumber of schedules possible for the task structure (N/Cindicates that the value is too large for the variable used tocompute it). The fifth column (# Alts R / Total) identifies thenumber of alternatives actually produced during the schedul-ing run the first number is the number of alternatives pro-duced at the root note and the second number is the totalnumber of alternatives produced during scheduling. The first

number is comparable to the upper-bounds expressed in col-umn three. The sixth column shows the number of schedulesactually produced. The column labeled # D Combinesindi-cates the number of distribution combination operations per-formed during scheduling this is particularly informativebecause nearly all aspects of the scheduling process involveprobability distributions rather than expected values. Thelast three columns pertain to the time spent (in whole sec-onds) doing different activities, namely producing the set ofroot-level alternatives, creating schedules from the alterna-tives, and the total scheduler runtime (which includes somefinal sorting and other output-related operations). Due inpart to the schedulers use of a particular set of clock func-tions, which are integer based, there is no variance when

the experiments are repeated because the variance pertainsto less than whole seconds.

Most of the task structures produced 15 schedules thisis the system default. When fewer schedules are produced itindicates that there are not sufficient alternatives at the rootlevel to produce more schedules. When more than 15 sched-ules are produced, it indicates that the schedulers termina-tion criteria was not met generally caused by a large per-centage of zero quality schedules or by there being alterna-

tives that appear better than any schedules generated thus farper the design criteria. The scheduler will work beyond itspreset number under these conditions but only to some mul-tiple of the preset. The JIL translated structure, for example,contains some modeling problems that produce a very largenumber of zero quality schedules and DTC scheduled up to4*15 and then halted with a small set of viable schedules.

With respect to scheduler computation overhead and on-

line performance, the time required to schedule these taskstructures is reasonable given that the grainsize of the struc-tures themselves is much larger than the seconds required toperform the scheduling operation (generally, scheduler over-head is at most 1% of the total runtime of the agents ap-plication). That being said, however, being appropriatelyfast is not necessarily the long term objective and perfor-mance guarantees and performance estimates are importantresearch avenues for the future.

One promising area of DTC related research is an offlinecontingency analysis tool [17, 23] that uses DTC to explorean approximation of the schedule space for a given TMStask structure. The use of DTC as an oracle enables thecontingency analysis tool to cope with the combinatorics

of the general scheduling problem. The contingency analy-sis methodology determines the criticality of failures withina schedule and for critical points, evaluates the statisticalcharacteristics of the available recovery options. The analy-sis, while expensive, is appropriate for mission-critical harddeadline situations in which a solution must be guaranteedby a particular time. With DTCs approach, it is possibleto start down a solution path (that is appealing even withthe possibility of failure) for which there is no mid-streamrecovery option that will enable the agent to still producesome result by the hard deadline. DTC will always recoverand find whatever options are available, but, because it doesnot plan for contingency and recovery a priori, in hard dead-line situations in which solutions must be guaranteed, there

is some possibility of unrecoverable failure.DTC has many different parameter settings not discussedhere and it can be made to avoid failure if possible. However,while this covers a certain class of the functionalities offeredby the contingency analysis tool, the two are not equivalent.Whereas the best DTC can do is avoid failure if possible,or work to minimize failure, it can only do this for a sin-gle line of control. Using the contingency analysis tool, theagent can select a high risk plan of action that also has somepotential of a high pay off, but, it can also reason a prioriabout the ability to recover from a failure of the plan. WhileDTC can be extremely conservative, it cannot plan for bothhigh-risk and recovery concurrently. The choice betweenDTC and the contingency analysis approach is dependent on

the application. For online, responsive control to unplannedevents, DTC is most appropriate. For mission-critical situ-ations combined with time for a priori offline planning, thecontingency approach is most appropriate.

AcknowledgmentsWe would like to acknowledge the Design-to-Time workof Professor Alan Garvey; said work made this and otherdevelopments in TMS scheduling possible. We would


9/10


10/10

[20] Regis Vincent, Bryan Horling, Thomas Wagner, andVictor Lesser. Survivability simulator for multi-agentadaptive coordination. InProc. of the 1st Intl. Conf. onWeb-Based Modeling and Simulation, 1998.

[21] Thomas Wagner, Alan Garvey, and Victor Lesser.Complex Goal Criteria and Its Application in Design-to-Criteria Scheduling. InProc. of the 14th Natl. Conf.on AI, pages 294301, July 1997. Also as UMASS CS

TR-1997-10.[22] Thomas Wagner, Alan Garvey, and Victor Lesser.

Criteria-Directed Heuristic Task Scheduling. Intl.Journal of Approximate Reasoning, Special Issue onScheduling, 19(1-2):91118, 1998. Also as UMASSCS TR-97-59.

[23] Thomas Wagner, Anita Raja, and Victor Lesser. Mod-eling Uncertainty and its Implications to Design-to-Criteria Scheduling, Under review Special issue of theAI Journal, Also as UMASS CS TR-98-51

[24] Ping Xuan and Victor R. Lesser. Incorporating uncer-tainty in agent commitments. In N.R. Jennings andY. Lesperance, editors, Intelligent Agents VI Proc.

of ATAL-99, Lecture Notes in AI. Springer-Verlag,Berlin, 2000.

[25] S. Zilberstein and S. J. Russell. Optimal compositionof real-time systems. AI, 82(1):181214, December1996.

[26] Shlomo Zilberstein. Using anytime algorithms in in-telligent systems. AI Magazine, 17(3):7383, 1996.

[27] M. Zweben, B. Daun, E. Davis, and M. Deale.Scheduling and rescheduling with iterative repair. InM. Zweben and M. Fox, editors, Intelligent Schedul-ing, Chapter 8. Morgan Kaufmann, 1994.

Documents

Wagner Aaai Real Time