Algorithm Research of Flexible Graphplan Based on Heuristic

Yang Li 1, 2, Yan Sun 1, 2, Chengshan Han1, Xiaodong Wang1, Shuyan Xu1 1 Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences,

Changchun 130033, China 2 Graduate School of the Chinese Academy of Sciences, Beijing 100049, China

Email:victoria_liy@163.com

Abstract

Classic Graphplan has too much restriction on

capturing the full subtlety of many real problems. Flexible Graphplan is defined which supports the soft constraints often found in reality. Heuristic concept is introduced to the process of plan extraction in flexible graphplan in order to improve the efficiency of plan extraction and the quality of plan. A novel algorithm using the new heuristic function which is applied by improved sum mutex heuristic to deal with planning problems is proposed. Sum mutex heuristic which is in common use as heuristic function takes into account only static propositional mutexes, and ignores the mutexes of actions. The performance of the new algorithm on many benchmark problems is remarkably robust. It can solve many planning problems, which can’t be solved by the heuristic state space search planning system using sum mutex heuristic.

Keywords: State space search, flexible graphplan, heuristic, authorization, mutual exclusion. 1. Introduction

The study of plan has become a hotspot in artificial intelligence field in recent years. It is widely used at many areas such as spaceflight, robot control, logistics and so on. The successful application of plan in national defenses and interspaces technology brings great economy and community benefit for many nations especially the developed countries [1].

Along with the process of intelligent planning, researchers find that classical planning cann’t satisfy the demand of practice applications, becasuse there are many indeterminacy factors. Traditionally, planning problems are cast in terms of imperative constraints that are either wholly satisfied or wholly violated, which limits the classical planning algorithms only to solve small-scaled model planning problems. It is

argued herein that this framework is too rigid to capture the full subtlety of many real problems such as aviation and spaceflight problems. In order to sovle the problem above, it gives birth to a new planning named flexible planning which supports the soft constraints often found in reality. This flexible planning problem can only be solved by flexible planner but can’t by the classical planner. Ian Miguel, Peter Jarvis and Qiang shen make the deep research in flexible planning [2] [3]. They exploited the FGP (Flexible Graphplan) and LFGP (Leximin Flexible Graphplan) early or late.

In recent years, the planning community has developed a number of attractive and scalable approaches for solving deterministic planning problems. Prominent among these are the Graphplan algorithm, which is put forward by Blum and Furst [4]. This method is to make use of the planning graph structure to solve planning problems especially which sub-goals influence each other. At the present time, many planners are based on Graphplan. But there is much faultiness. Several techniques have been employed to improve Graphplan. Especially in 2000 years, Michel Cayrol, Pierre Régnier and Vincent Vidal developed a Graphplan-like planner called LCGP (Least Committed Graphplan) [7]. In practice, LCGP rapidly gives a solution on many classical benchmarks (Logistics, Ferry…) where Graphplan is unable to produce a plan after a significant running time.

Another outstanding method is heuristic state space search [5], which uses heuristic function to make the explanation of plan to guide the search in the state space [6]. The superior performance of state space planner comes from the heuristic estimators they use to evaluate the goodness of children states, exemplified by Bonet and Geffner’s HSP-r planner using the sum mutex heuristic. To make the computation tractable, these heuristic estimators make strong assumptions about the independence of subgoals. Because of these assumptions, state search planners often thrash badly in

The 9th International Conference for Young Computer Scientists

978-0-7695-3398-8/08 $25.00 © 2008 IEEE

DOI 10.1109/ICYCS.2008.473

48

problems where there are strong interactions between subgoals. Furthermore, these independence assumptions also make the heuristics inadmissible.

Because of those shortages of Graphplan and heuristic search, XuanLong Nguyen, Subbarao Kambhampati and Romeo S. Nigenda provide a way of successfully combining the advantages of the Graphplan and heuristic state search approaches [6]. In 2005 years, Yang Li etc. developed a new heuristic function-dynamic mutex sum heuristic, which leverages complementary strengths of Graphplan and heuristic state search planning [8]. In 2006 years, Yi Han and Yang Li proposed a method of flexible graphplan using sum mutex heuristic [9]. In this paper, we use dynamic mutex sum heuristic as heuristic function in plan graph extraction in order to get great efficiency and deal with more planning problems.

The rest of the paper is organized as follows. Section 2 reviews the state space approach in planning and the knowledge of flexible planning. Section 3 discusses how the heuristic can be used in plan extraction of flexible graphplan, and redefines mutual exclusion in flexible graphplan. A novel algorithm is brought forward. Section 4 analyzes the experiment result and compares the efficiency of new algorithm to others. Our conclusions are given in Section 5. 2. Background knowledge 2.1. Heuristic search

The efficiency of state search planners and the quality of the solutions that they return depends critically on the information and admissibility of these heuristic estimators, respectively. If the information which taken by the heuristic function is more, the efficiency is better too.

Now let’s take HSP- r as an example to explain how to use heuristic function. HSP-r planner casts the planning problem as search in the regression space of the world states. The heuristic value of a state S is the estimated cost (number of actions) needed to achieve S from the initial state S0. HSP-r follows a variation of the A* search algorithm, called Greedy Best First, which uses the cost function as follows:

f(S) = g(S)+w×h(S),where g(S) is the accumulated cost (number of actions when regressing from goal state) and h(S) is the heuristic value of state S.

In fact, sometimes there are interactions between propositions. HSP-r adopts the notion of mutex relations first originated in Graphplan. But unlike Graphplan, only static propositional mutex is computed. Two propositions p and q form a static mutex when they cannot both be present in any state reachable from the initial state. Since the cost of any

set containing a mutex pair is infinite, a variation of the sum heuristic called sum mutex heuristic is defined as follows:

( ) ( )p S

if p, q S s.t.mutex(p, q)h pH S otherwise

∈

∞ ∃ ∈⎧= ⎨⎩∑

The heuristic cost h (p) of an individual proposition p is computed using an iterative procedure that is run to fixpoint as follows: (1) Initially, each proposition p is assigned a cost 0 if it is in the initial state and ∞ otherwise. (2) For each action a, that adds some proposition p, h(p) is updated as:

h(p)←min{ h(p),h(prec(a))}. Sum mutex heuristic takes into account only static

propositional mutex, and ignores the mutex of actions. Sometimes it is not enough to consider that static mutex of propositions only. In progression state space search, there are many instances such as: the result of the progression of S over an applicable action A is S1 and over B is S2, where H(S1)>H(S2).Using the principle of heuristic, we shall choose action B to guide plan. But there are negative interactions between subgoals. If we choose action B, it may arouse the misguiding to the search. This completely misguides the planner into wrong paths, from which it never recovers. It even makes the planning unable to produce a plan. Therefore we shall introduce the notion of actions’ authorization relation, which is first used in LCGP. We define the dynamic mutex sum heuristic via considering the new mutex of actions.

For an action set Ώ, A, B∈ Ώ, S is the state which progresses over an applicable action A.

( ) ( )( )p S

if action A is mutuallyexclusive with every other actions, or B, B A, but not A B

B, (B A and A B) h p (A B,but not(B A) )

H Sor

∈

⎧⎪ ∞⎪ ∃ ∠= ⎨ ∠⎪ ∃ ∠ ∠⎪ ∠ ∠⎩∑

The dynamic mutex sum heuristic considers the propositions mutex exclusion induced by actions, which is omitted by sum mutex heuristic. So it can effectively void that misguiding. The actions mutual exclusion that the dynamic mutex sum heuristic used is more excellent than that of Graphplan. It can be demonstrated that the performance of the adjusted heuristic is better than the performance of formerly sum mutex heuristic. The adjusted heuristic can discover superior planning solution in many planning problems, where using the sum mutex heuristic fails or worsens the performance for similar reasons such as the travel, mystery, grid, blocks world, and eight puzzle domains.

2.2. Flexible planning introduction

49

A flexible planning problem Ψ, consists of a 4-tuple, <Φ, Ο, Ι, Γ>, denoting sets of plan objects, flexible operators, initial conditions consisting of flexible propositions, and flexible goal, respectively [3].

Boolean propositions are herein replaced by flexible propositions ρ, of the form (ρ φ1, φ2,.., ki), where φi∈Φ and ki is an element of a totally ordered set K, which denotes the subjective degree of truth of the proposition. K is composed of a finite number of membership degrees, k┴, ki,..., k┬. Boolean propositions are captured at the end points of K, with k┴∈K and k┬∈K indicating total falsehood and total truth respectively.

A flexible operator o∈Ο must recognize how well its preconditions are satisfied. Flexible operators are described by fuzzy relations which map from the precondition space to a totally ordered satisfaction scale L and a set of flexible effects propositions. L is also composed of a finite number of membership degrees, l┴, l1,..., l┬. The endpoints l┴ and l┬∈L respectively denote a complete lack of satisfaction and complete satisfaction.

A flexible plan goal γ∈Γ maps from the space of flexible propositions to L. Each goal is defined using a number of clauses. More than one set of mutually-consistent propositions may exist which satisfy the plan goals to some extent. Hence, the satisfaction degree of the plan as a whole must take into account the goal satisfaction degrees as well as those of the flexible operators. The satisfaction degree of a flexible plan is defined as the conjunctive combination of the satisfaction degrees of each operator and each goal used in the plan. The quality of a plan is its satisfaction degree combined with its length, where the shorter of two plans with equivalent satisfaction degrees is better.

3. Plan extraction of flexible graphplan using dynamic mutex sum heuristic

When a consistent flexible proposition set which can satisfied flexible plan goal exists in current level, the extraction process of flexible plan begins. The process of plan extraction in flexible plan graph is similar to that of general plan extraction in Graphplan. If only two satisfied value l┴ and l┬ are used, and two similar truth values are k┴ and k┬, the problem then becomes to a Boolean problem and the plan extraction becomes the plan extraction of Graphplan too.

As for flexible graphplan, the satisfaction degrees of plan should be considered during the procedure of plan extraction. Therefore the extraction procedure is more complex than that in Graphplan in order to get a plan

accord with the required satisfaction scale. And the mutual exclusion must be redefined too. 3.1. Mutual exclusivity and definitions in Flexible graphplan

Graphplan imposes strong constraints on the plans. It uses the independence property to choose actions, which are considered simultaneously. That is, two actions A and B, if A does not delete a precondition and an add effect of B, B does not delete that of A as well we can say they are independent for each other. Here there is no implementing order between two actions. It can be demonstrated that modify this property to relax a part of the constraints on actions of the same set still produce plans. So a more flexible asymmetrical relation between the actions is defined, which is called the authorization relation. Two actions are now mutually exclusive when they have mutually exclusive preconditions or when neither of these two actions authorizes another. Definition 1 (Independence):

Two actions A，B∈I, are independence, where I is the set of actions iff

(1) A≠B and (2) (Add (A) UPrec (A))∩Del(B)=∅and

(Add (B) U Prec (B))∩Del(A)=∅ Definition 2 (Authorization):

An action A∈I authorizes an action B∈I, noted A∠B, where I is the set of actions iff

(1) A≠B and (2) Add(A)∩Del(B)=∅and Prec(B)∩Del(A)=∅. An action A forbids an action B iff the action A

does not authorize B, noted not (A∠B). Definition 3 (Mutual exclusion):

Two actions A ， B ∈I are mutually exclusive, where I is the set of actions iff

(1) A≠B and (2) Each of them forbids the other: not (A∠B) and

not (B∠A), or if a precondition of one is mutually exclusive with a precondition of another.

Authorization and mutual exclusion are introduced to flexible graphplan. The mutual exclusion in our flexible graphplan is redefined.

3.1.1 Action mutual exclusivity

Two flexible actions in the same action level are

mutually exclusive if any of the following hold: (1) Inconsistent effects: An effect of one action

expresses a different truth degree concerning the same core proposition to the effect of the other action in the same proposition;

50

(2) Interference: An effect of one action expresses a different truth degree concerning the same core proposition to a necessary precondition of the other action in the same proposition;

(3) Competing Needs: The actions have mutually exclusive preconditions or each of them forbids the other.

3.1.2 Proposition mutual exclusivity

Two flexible propositions in the same action level

are labeled as exclusive if any of the following hold: (1) Either they express a different truth degree for the

same core proposition; (2) All ways of creating one are exclusive of all ways

of creating the other. 3.2 An example

A flexible planning problem Ψ, Φ= {a, b, c, d, e, f}, Ο= {A, B, C}, Ι={a, b, d}, and Γ={(when c l┬) ( when e l2)}, K={ k┴, k1, k2, k┴}，L={ l┴, l1, l2, l┬}, the partial definition of actions A, B, C is as follows: operator A {when (preconds a b) (effects c f ┐a) (satisfaction l┬)} operator B {when (preconds b d) (effects e ┐b ┐d) (satisfaction l2)} operator C {when (preconds a ) (effects c ┐a) (satisfaction l1)}

Partial flexible planning graph for the problem Ψ is showed in figure 1, the satisfaction scale of propositions and mutual exclusivity are not shown in it.S1 is the first state level which progresses over an action A, and S2 is the first state level progressing over an action B, the first state level which progresses over action C named S3, the second state level progressed over action B noted S4. The difference among the plan extraction without heuristic, using sum mutex heuristic and dynamic mutex sum heuristic can be compared in this example. (1) Without heuristic: plan extraction begins in first

proposition level while the goal set appears without any mutual exclusion, Two actions A and B are not independence, so they can not be selected at the same time. In second proposition level goal set appears again and a valid plan can be found now. Sub-goals, actions, mutual

exclusion and so on should be stored in graph extension. The more plan extension levels, the more memory space and the more time finding a valid plan.

(2) Using sum mutex heuristic: goal set appears in first proposition level. The heuristic value of S1, S2 and S3 are H(S1)=2, H(S2)=1, H(S3)=1. We can only choose action B because the truth degree of c in S2 is l1 which is lower than goal requirement. But now it is impossible to get to goal and to find a valid plan.

(3) Using dynamic mutex sum heuristic: actions authorizations are considered in both plan extension and extraction. Action A authorizes B, but B doesn’t authorize A. Action A and C are mutually exclusive. Action B and C are independence. Goal set appears which graph is extended for first level. We can only choose between S1 and S3 because the truth degree of c in S2 is l1 which is lower than goal requirement. Compute the value of S1 and S3 using dynamic mutex sum heuristic. H(S1)=2, H(S3)=∞. The valid plan is action sequence {A, B}.

l

l

l

l

Figure 1. Partial planning graph for problem

3.3. Algorithm description

The detailed algorithm description of new flexible graphplan based on heuristic named FGP-H (Flexible Graphplan based on Heuristic) is as follows: (1) Plan extension begins from initial conditions I. If

all preconditions of an action appear in I, we progress initial conditions over the action, sign mutual exclusion and subsequence states.

(2) Checking graph continuously, plan extraction begins when goal set appears in one proposition level without any mutual exclusion.

(3) Check goal set firstly and compute heuristic value of states which accord with satisfaction degree using dynamic mutex sum heuristic. The computing course is as follows:

For an action set Ώ, A, B∈ Ώ, S is the state which progresses over an applicable action A.

51

( ) ( )( )p S

if action A is mutually exclusive with every other actions, or B, B A, but not A B

B, (B A and A B) h p (A B,but not(B A) )

H Sor

∈

⎧⎪ ∞⎪ ∃ ∠= ⎨ ∠⎪ ∃ ∠ ∠⎪ ∠ ∠⎩∑

The heuristic cost h(p) of an individual proposition p is computed using an iterative procedure that is run to fixpoint as follows. (1) Initially, each proposition p is assigned a cost 0 if it is in the initial state and ∞ otherwise. (2) For each action a, that adds some proposition p, h(p) is updated as:

h(p)←min{ h(p),h(prec(a))}. (4) Choose appropriate actions to extract valid plan

and output all actions reversely which is employed in arriving goal set from initial conditions. The steps are similar to that of Graphplan.

(5) If there is a proposition level P such that all future proposition levels are exactly the same as P, i.e., they contain the same set of propositions and have the same exclusivity relations, we say the graph has leveled off. When graph has leveled off and goal set doesn’t appear, the plan is failed.

Figure 2 shows the flow chart of new algorithm.

Flexible graphplan Expansion

Program beginning

Goal achievingwithout mutex

Plan outputting

Yes

NO Plan

NO

Flexible graphplan extraction

Heuristic function computing Planning graphs

level off

Yes

NO

Figure. 2. Flow chart of new flexible graphplan

4. Experimentation result analysis

Because there are interactions between propositions, it needs to take into account the mutual exclusion of the propositions. It is not enough to only consider the static mutual exclusion of propositions but not to consider the mutual exclusion of propositions, which is induced by the mutual exclusion of actions. For planning problems where there is a rich interaction between actions and subgoal sequencing, the planning using dynamic mutex sum heuristic as default heuristic

function thrashes well. Specifically, in fact, in real world, there always exists such mutual exclusion in many planning problems. For the omitting of this part mutual exclusion, it maybe leads to the misguiding to the plan, and leads to no valid planning can be found.

Table 1 shows the results obtained when running FGP-H, FG-H, FGP and Graphplan, on benchmark problems from the logistics domain. Graphplan is implemented in C and the other solvers in Java. Hardware used: Pentium (R); CPU2.80GHz; Memory: 512MB; Operating system: Windows XP.

Table1. Comparison of FGP-H, FP-H, FGP and GP

Problem FGP-H FP-H FGP GP Logistics-a 8 10 11 1955 Rescue -a 18 22 25 -- Rescue--b 19 20 20 -- Rocket-a 10 6 14 75 Rocket-b 6 10 21 154 The domain used was a variant of the flexible

logistics domain. The test suite contained 6 problems, with plan lengths at the satisfaction bands as shown in figure 3, which also shows the resultant run-times.

Problem

Plan

(leng

th)

Boolean

1l2l

ll

Figure. 3. Comparison in run-time and plan length

Experiment result shows that FGP-H is the best planner among those flexile planners. It is also better than GP while it deals with Boolean problems. The adjusted dynamic mutex sum heuristic can discover superior planning solution in many planning problems, where using the sum mutex heuristic fails or worsens the performance for similar reasons such as the travel, grid, blocks world, and eight puzzle domains.

52

5. Conclusion

The research of flexible has be give more and more researchers’ attention. In this paper, we use graph notion to deal with flexible planning. Using dynamic mutex sum heuristic in plan graph extraction in order to get great efficiency and deal with more planning problems. The mutual exclusivity relations in our algorithm are redefined and the authorization of actions is introduced. It is found that the algorithm is very efficient through test. The future work is to make our algorithm more perfect and apply the algorithm to actual intelligent system such as spaceflight system which is developed by our institute. 6. References [1] D.L Ding and Y.F. Jiang, “The research of intelligence

plan and its applications”, Computer science, 29(2), 2002, pp. 100-103.

[2] I. Miguel, Q. Shen, and P. Jarvis, “Efficient Flexible Planning via Dynamic Flexible Constraint Satisfaction”, Engineering Applications of Artificial Intelligence, 14 (3), 2001, pp. 301-327.

[3] I. Miguel, P. Jarvis, and Q. Shen, “Flexible Graphplan”, Proceedings of the Fourteenth European Conference on Artificial Intelligence, 2000, pp. 506-510.

[4] A. Blum and M. Furst, “Fast planning through planning-graphs analysis”, Artificial Intelligence, 90, 1997, pp. 281-300.

[5] B. Bonet and H. Geffer, “Planning as heuristic search: New results”, In Proceedings of European Conference on Planning, 1999, pp.360-372.

[6] X.L Nguyen, S. Kambhampati, and R.S. Nigenda, “Planning Graph as the Basis for Deriving Heuristics for Plan Synthesis by State Space and CSP Search”, Artificial Intelligence, 135 (1), 2002, pp. 73-123.

[7] Michel Cayro, Pierre Refnier, and Vincent Vidal, “Least commitment in Graphplan”, Artificial Intelligence, 130(1), 2001, pp. 85-118.

[8] Y. Li, W.X. Gu, M.H. Yin, and Y. Wang, “Planning system based on heuristic”, Proceedings of the Fourth International Conference on Machine Learning and Cybernetics, vol.3, 2005, pp.1385-1390.

[9] Y. Han, W.X. Gu, Y. Li, M.H. Yin, and J.B. Zhang, “Flexible Graphplan based on heuristic searching”, Proceedings of the fifth International Conference on Machine Learning and Cybernetics, Dalian, 2006, pp. 160 -163.

[10] N. Yang, W.F. Tian, and Z.H. Jin. “DGPS integer ambiguity”, Optics and Precision Engineering, 14(5), 2006, pp.891-895.

[11] Z.H. Yang, Z.Q. Qi, and J.C. Fang. “Immune recognition algorithm and its application to air target detection and recognition”, Optics and Precision Engineering, 14(5), 2006, pp.922-927.

[12] Z.Z. Chen, G.Z. Yan, L.M. Lin, and H. Cai, “Novel approach to generating optimal smooth trajectory for a manipulator”, Optics and Precision Engineering, 2001(3), pp. 242-246.

53