Upload
jade-harper
View
212
Download
0
Embed Size (px)
Citation preview
Satisfiability and State-Transition Systems: An AI
Perspective
Satisfiability and State-Transition Systems: An AI
Perspective
Henry Kautz
University of Washington
IntroductionIntroduction
Both the AI and CADE/CAV communities have long been concerned with reasoning about state-transition systems
• AI – Planning
• CADE/CAV – Hardware and software verification
Recently propositional satisfiability testing has turned out to be surprisingly powerful tool
• Planning – SATPLAN (Kautz & Selman)
• Verification – Bounded model checking (Clarke), Debugging relational specifications (Jackson)
Shift in KR&R Shift in KR&R
Traditional approach: specialized languages / specialized reasoning algorithms
New direction: • Compile combinatorial reasoning problems into a
common propositional form (SAT)
• Apply new, highly efficient general search engines
Combinatorial Task
SAT Encoding SAT Solver
Decoder
AdvantagesAdvantages
Rapid evolution of fast solvers• 1990: 100 variable hard SAT problems
• 2000: 100,000 variables
Sharing of algorithms and implementations from different fields of computer science
AI, theory, CAD, OR, CADE, CAV, …
Competitions - Germany 91 / China 96 / DIMACS-93/97/98
JAR Special Issues – SAT 2000
RISC vs CISC
Can compile control knowledge into encodings
OUTLINEOUTLINE
1. Planning Model Checking
2. Planning as Satisfiability
3. SAT + Petri Nets + Randomization = Blackbox
4. State of the Art
5. Using Domain-Specific Control Knowledge
6. Learning Domain-Specific Control Knowledge
GOAL: Overview of recent advances in planning that may (or may not!) be relevant to the CADE community!
1. Planning Model Checking1. Planning Model Checking
The AI Planning ProblemThe AI Planning Problem
Given a world description, set of primitive actions, and goal description (utility function), synthesize a control program to achieve those goals (maximize utility)
most general case covers huge area of computer science, OR, economics
program synthesis, control theory, decision theory, optimization …
STRIPS Style PlanningSTRIPS Style Planning
“Classic” work in AI has concentrated on STRIPS style planning (“state space”)
• Open loop – no sensing• Deterministic actions• Sequential (straight line) plans• SHAKEY THE ROBOT (Fikes & Nilsson 1971)
Terminology• Fluent – a time varying proposition, e.g. “on(A,B)”• State – complete truth assignment to a set of fluents• Goal – partial truth assignment (set of states)• Action – a partial function State State
specified by Operator schemas
Operator SchemasOperator Schemas
Each yields set of primitive actions, when instantiated over a given finite set of objects (constants)
Pickup(x, y)• precondition: on(x,y), clear(x), handempty
• delete: on(x,y), clear(x), handempty
• add: holding(x), clear(y)
Plan: A (shortest) sequence of actions that transforms the initial state into a goal state
• E.g.: Pickup(A,B); Putdown(A,C)
ParallelismParallelism
Useful extension: parallel composition of primitive actions
• Only allowed when all orderings are well defined and equivalent – no shared pre / effects
(act1 || act2)(s) = act2(act1(s)) = act1(act2(s))
• Can dramatically reduce size of search space
• Easy to serialize
• Distinguish:– number of actions in a plan – “sequential length”
– number of sequentially composition operators in a plan – “parallel length”, “horizon”
(a1 || a2); (a3 || a4 || a5) ; a6
- sequential length 6, parallel length 3
Some Applications of STRIPS-Style Planning
Some Applications of STRIPS-Style Planning
Autonomous systems• Deep Space One Remote Agent (Williams & Nayak 1997)
Natural language understanding• TRAINS (Allen 1998)
Internet agents• Rodney (Etzioni 1994)
Manufacturing• Supply chain management (Crawford 1998)
Abdundance of Negative Complexity Results
Abdundance of Negative Complexity Results
Unbounded STRIPS planning: PSPACE-complete• Exponentially long solutions
(Bylander 1991; Backstrom 1993)
Bounded STRIPS planning: NP-complete• Is there a solution of (sequential/parallel) length N?
(Chenoweth 1991; Gupta and Nau 1992)
Domain-specific planning: may depend on whether solutions must be the shortest such plan
• Blocks world –– Shortest plan – NP-hard– Approximately shortest plan – NP-hard
(Selman 1994)
– Plan of length 2 x number blocks – Linear time
Approaches to AI PlanningApproaches to AI Planning
Three main paradigms:• Forward-chaining heuristic search over state space
– original STRIPS system
– recent resurgence – TLPlan, FF, …
• “Causal link” Planning– search in “plan space”
– Much work in 1990’s (UCPOP, NONLIN, …), little now
• Constraint based planning– view planning as solving a large set of constraints
– constraints specify relationships between actions and their preconditions / effects
– SATPLAN (Kautz & Selman), Graphplan (Blum & Furst)
Relationship to Model CheckingRelationship to Model Checking
Model checking – determine whether a formula in temporal logic evaluates to “true” in a Kripke structure described by a finite state machine
• FSM may be represented explicitly or symbolically
STRIPS planning – special case where• Finite state matchine (transition relation) specified
by STRIPS operators– Very compact
– Expressive – can translate many other representations of FSM’s into STRIPS with little or no blowup
Relationship, continuedRelationship, continued
• Formula to be checked is of the form“exists path . eventually . GOAL”
– Reachability
– Distinctions between linear / branching temporal logics not important
Difference:• Concentration on finding shortest plans
• Emphasis on efficiently finding single witness (plan) as opposed to verifying a property holds in all states
– NP vs co-NP
Why Not Use OBDD’s?Why Not Use OBDD’s?
Size of OBDD explodes for typical AI benchmark domains
• Overkill – need not / cannot check all states, even if they are represented symbolically!
O(2n2) states
(But see recent work by M. Velosa on using OBDD’s for non-deterministic variant of STRIPS)
Verification using SATVerification using SAT
Similar phenomena occur in some verification domains
• Hardware multipliers
Has led to interest in using SAT techniques for verification and bug finding
• Bounded – fixed horizon
• Under certain conditions can prove that only considering a fixed horizon is adequate
– Empirically, most bugs found with small bounds
• E. Clarke – Bounded Model Checking– LTL specifications, FSM in SMV language
• D. Jackson – Nitpick– Debugging relational specifications in Z
2. Planning as Satisfiability2. Planning as Satisfiability
Planning as SatisfiabilityPlanning as Satisfiability
SAT encodings are designed so that plans correspond to satisfying assignments
Use recent efficient satisfiability procedures (systematic and stochastic) to solve
Evaluation performance on benchmark instances
SATPLANSATPLAN
axiomschemas instantiated
propositionalclauses
satisfyingmodelplan
length
problemdescription
SATengine(s)
instantiate
interpret
SAT EncodingsSAT Encodings
Target: Propositional conjunctive normal form
Sets of clauses specified by axiom schemas1. Create model by hand
2. Compile STRIPS operators
Discrete time, modeled by integers• upper bound on number of time steps
• predicates indexed by time at which fluent holds / action begins– each action takes 1 time step
– many actions may occur at the same step
fly(Plane, City1, City2, i) at(Plane, City2, i +1)
Solution to a Planning ProblemSolution to a Planning Problem
A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators
Easy to convert back to a STRIPS-style plan
Complete SAT AlgorithmsComplete SAT Algorithms
Davis-Putnam-Loveland-Logeman (DPLL)• Depth-first backtrack search on partial truth assignments
• Basis of nearly all practical complete SAT algorithms– Exception: “Stahlmark’s method”
• Key to efficiency: good variable choice at branch points
– 1961 – unit propagation, pure literal rule
– 1993 - explosion of improved heuristics and implementations
+ MOM’s heuristic
+ satz (Chu Min Li) – lookhead to maximize rate of creation of binary clauses
• Dependency directed backtracking – derive new clauses during search – rel_sat (Bayardo), GRASP (di Silva)
– See SATLIB 1998 / Hoos & Stutzle
Incomplete SAT AlgorithmsIncomplete SAT Algorithms
GSAT and Walksat (Kautz, Selman & Cohen 1993)
• Randomized local search over space of complete truth assignments
• Heuristic function: flip variables to minimize number of unsatisfied clauses
• Noisy “random walk” moves to escape local minima
• Provably solves 2CNF, empirically successful on a broad class of problems
– random CNF, graph coloring, circuit synthesis encodings (DIMACS 1993, 1997)
Planning Benchmark Test SetPlanning Benchmark Test Set
Extension of Graphplan benchmark set
logistics - transportation domain, ranging up to• 14 time slots, unlimited parallelism
• 2,165 possible actions per time slot
• optimal solutions containing 74 primitive actions
• 22000 legal states (60,000 Boolean variables)
Problems of this size not previously handled by any domain-independent planning system
Initial SATPLAN ResultsInitial SATPLAN Results
problem horizon / actions
Graphplan naïve SAT encoding
hand SAT encoding
rocket-b 7 / 30 9 min 16 min 41 sec
log-a 11 / 47 13 min 58 min 1.2 min
log-b 13 / 54 32 min * 1.3 min
log-c 13 / 63 * * 1.7 min
log-d 14 / 74 * * 3.5 min
SAT solver: Walksat (local search)
* indicates no solution found after 24 hours
How SATPLAN Spent its TimeHow SATPLAN Spent its Time
problem instantiation walksat DPLL satz
rocket-b 41 sec 0.04 sec 1.8 sec 0.3 sec
log-a 1.2 min 2.2 sec * 1.7 min
log-b 1.3 min 3.4 sec * 0.6 sec
log-c 1.7 min 2.1 sec * 4.3 sec
log-d 3.5 min 7.2 sec * 1.8 hours
Hand created SAT encodings
* indicates no solution found after 24 hours
3. SAT + Petri Nets + Randomization = Blackbox
3. SAT + Petri Nets + Randomization = Blackbox
Automating EncodingsAutomating Encodings
While SATPLAN proved the feasibility of planning using satisfiability, modeling the transition function was problematic
• Direct naïve encoding of STRIPS operators as axiom schemas gave poor performance
• Handcrafted encodings gave good performance, but were labor intensive to create
– similar issues arise in work in verification – division of labor between user and model checker!
GOAL: fully automatic generation and solution of planning problems from STRIPS specifications
GraphplanGraphplan
Graphplan (Blum & Furst 1995)
Set new paradigm for planning
Like SATPLAN...• Two phases: instantiation of propositional structure,
followed by search
Unlike SATPLAN...• Efficient instantiation algorithm based on Petri-net
type reachability analysis
• Employs specialized search engine
Neither approach best for all domains• Can we combine advantages of both?
BlackboxBlackbox
STRIPSPlan Graph
Petri Net Analysis
CNF
GeneralSAT engines
Solution
SimplifierTranslator
CNF
Component 1: Petri-Net AnalysisComponent 1: Petri-Net Analysis
Graphplan instantiates a “plan graph” in a forward direction, pruning (some) unreachable nodes• plan graph unfolded Petri net (McMillian 1992)
Polynomial-time propagation of mutual-exclusion relationships between nodes• Incomplete – must be followed by search to
determine if all goals can be simultaneously reached
Growing the Plan GraphGrowing the Plan Graph
P0
facts factsactions actions
Growing the Plan GraphGrowing the Plan Graph
P0 A1
B1
P2
R2
facts factsactions actions
Q2
Growing the Plan GraphGrowing the Plan Graph
P0 A1
B1
P2
R2
C3
facts factsactions actions
Q2
Growing the Plan GraphGrowing the Plan Graph
P0 A1
B1
P2
R2
C3
facts factsactions actions
Q2
Growing the Plan GraphGrowing the Plan Graph
P0 A1
B1
P2
R2
C3
facts factsactions actions
Q2
Growing the Plan GraphGrowing the Plan Graph
P0 A1
B1
P2
R2
facts factsactions actions
Q2
Component 2: TranslationComponent 2: Translation
P0 A1
B1
P2
R2
facts factsactions actions
Q2
Action implies preconditions: A1 P0 , B1 P0
Mutual exclusion: A1 B1 , P2 Q2
Initial facts hold at time 0
Goals holds at time n
Component 3: SimplificationComponent 3: Simplification
Generated wff can be further simplified by more general consistency propagation techniques
• unit propagation: is Wff inconsistant by resolution against unit clauses?
O(n)
• failed literal rule: is Wff + { P } inconsistant by unit propagation?
O(n2)
• binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation?
O(n3)
General simplification techniques complement Petri net analysis
Effective of SimplificationEffective of Simplification
Percent vars set byProblem Varsunitprop
failedlit
binaryfailed
bw.a 2452 10% 100% 100%bw.b 6358 5% 43% 99%bw.c 19158 2% 33% 99%log.a 2709 2% 36% 45%log.b 3287 2% 24% 30%log.c 4197 2% 23% 27%log.d 6151 1% 25% 33%
Component 3: Randomized Systematic Solvers
Component 3: Randomized Systematic Solvers
BackgroundBackground
Combinatorial search methods often exhibit
a remarkable variability in performance. It is
common to observe significant differences
between:• different heuristics
• same heuristic on different instances
• different runs of same heuristic with different random seeds
How SATPLAN Spent its TimeHow SATPLAN Spent its Time
problem instantiation walksat DPLL satz
rocket-b 41 sec 0.04 sec 1.8 sec 0.3 sec
log-a 1.2 min 2.2 sec * 1.7 min
log-b 1.3 min 3.4 sec * 0.6 sec
log-c 1.7 min 2.1 sec * 4.3 sec
log-d 3.5 min 7.2 sec * 1.8 hours
Hand created SAT encodings
* indicates no solution found after 24 hours
Preview of StrategyPreview of Strategy
We’ll put variability / unpredictability to our advantage via randomization / averaging.
Cost DistributionsCost Distributions
Consider distribution of running times of backtrack search on a large set of “equivalent” problem instances
• renumber variables
• change random seed used to break ties
Observation (Gomes 1996): distributions often have heavy tails
• infinite variance
• mean increases without limit
• probability of long runs decays by power law (Pareto-Levy), rather than exponentially (Normal)
Heavy TailsHeavy Tails
Bad scaling of systematic solvers can be caused by heavy tailed distributions
Deterministic algorithms get stuck on particular instances
• but that same instance might be easy for a different deterministic algorithm!
• Expected (mean) solution time increases without limit over large distributions
• Log-log plot of distribution of running times approximately linear
Heavy-Tailed DistributionsHeavy-Tailed Distributions
… … infinite variance … infinite meaninfinite variance … infinite mean
Introduced by Pareto in the 1920’s “probabilistic curiosity”
Mandelbrot established the use of heavy-tailed distributions to model real-world fractal phenomena
• stock-market, Internet traffic delays, weather
New discovery: good model for backtrack search algorithms
• formal statement of “folk wisdom” of theorem proving community
Randomized RestartsRandomized Restarts
Solution: randomize the systematic solver• Add noise to the heuristic branching (variable choice)
function
• Cutoff and restart search after a fixed number of backtracks
Provably Eliminates heavy tails
In practice: rapid restarts with low cutoff can dramatically improve performance
(Gomes, Kautz, and Selman 1997, 1998)• Related analysis: Luby & Zuckerman 1993; Alt & Karp 1996
Rapid Restart on LOG.DRapid Restart on LOG.D
1000
10000
100000
1000000
1 10 100 1000 10000 100000 1000000
log( cutoff )
log
( b
ackt
rack
s )
Note Log Scale: Exponential speedup!
Overall insight:Overall insight:
Randomized tie-breaking with
rapid restarts can boost
systematic search algorithms
• Speed-up demonstrated in many versions of Davis-Putnam
– basic DPLL, satz, rel_sat, …
• Related analysis: Luby & Zuckerman 1993; Alt & Karp 1996
Blackbox ResultsBlackbox Results
problem naïve SAT encoding
hand SAT encoding
blackbox
walksat
blackbox
satz-rand
rocket-b 16 min 41 sec 2.5 sec 4.9 sec
log-a 58 min 1.2 min 7.4 sec 5.2 sec
log-b * 1.3 min 1.7 min 7.1 sec
log-c * 1.7 min 15 min 9.3 sec
log-d * 3.5 min * 52 sec
Naïve/Hand SAT solver: Walksat (local search)
* indicates no solution found after 24 hours
4. State of the Art4. State of the Art
Which Strategies Work Best?Which Strategies Work Best?
Causal-link planning• <5 primitive actions in solutions
• Works best if few interactions between goals
Constraint-based planning• Graphplan, SATPLAN, + descendents
• 100+ primitive actions in solutions
• Moderate time horizon <30 time steps
• Handles interacting goals well
1995 – 1999 Constraint-based approaches dominate
• AIPS 1996, AIPS 1998
Graph Search vs. SATGraph Search vs. SAT
SATPLAN
Graphplan
Problem size / complexity
Tim
e
Caveat: on some domains SAT approach can exhaust memory even though direct graph search is easy
Blackbox withsolver schedule
Resurgence of A* SearchResurgence of A* Search
In most of 1980 – 1990’s forward chaining A* search was considered a non-starter for planning
Voices in the wilderness:
• TLPlan (Bacchus) – hand-tuned heuristic function could make approach feasible
• LRTA (Geffner) – can automatically derive good heuristic functions
Surprise – AIPS-2000 planning competition dominated by A* planners!
• What happened?
Solution Length vs HardnessSolution Length vs Hardness
Key issue: relationship between solution length and problem hardness
• RECALL: In many domains, finding solutions that minimize the number of time steps is NP-hard, while finding an arbitrary solution is in P
– Put all the blocks on the table first
– Deliver packages one at a time
• Long solutions minimize goal interactions, so little or no backtracking required by forward-chaining search
• AIPS-2000 Planning Competition did not consider plan length criteria!
Non-Optimal PlanningNon-Optimal Planning
0.01
0.1
1
10
100
1000
10000
100000
easy rocket-a rocket-b
blackbox
hsp
ff
Optimal-Length PlanningOptimal-Length Planning
0.01
0.1
1
10
100
1000
10000
100000
easy rocket-a rocket-b
blackbox
hsp
ff
Which Works Best, ContinuedWhich Works Best, Continued
Constraint-based planning• Short parallel solutions desired• Many interactions between goals• SAT translation a win for larger problems where
time is dominated by search (as opposed to instantiation and Petri net analysis)
Forward-chaining search• Long sequential solutions okay• Few interactions between goals
Much recent progress in domain-independent planning…
but further scaling to large real-world problems requires domain-dependent techniques!
5. Using Domain-Specific Control Knowledge
5. Using Domain-Specific Control Knowledge
Kinds of Domain-Specific Knowledge
Kinds of Domain-Specific Knowledge
Invariants true in every state• A truck is only in one location
Implicit constraints on optimal plans• Do not remove a package from its destination location
Simplifying assumptions• Do not unload a package from an airplane, if the
airplane is not at the package’s destination city– eliminates connecting flights
Expressing KnowledgeExpressing Knowledge
Such information is traditionally incorporated in the planning algorithm itself
Instead: use additional declarative axioms(Bacchus 1995; Kautz 1998; Huang, Kautz, & Selman 1999)
• Problem instance: operator axioms + initial and goal axioms + control axioms
• Control knowledge constraints on search and solution spaces
• Independent of any search engine strategy
Axiomatic FormAxiomatic Form
State Invariant:
at(truck,loc1,i) & loc1 loc2 at(truck,loc2,i)
Optimality:
at(pkg,loc,i) & at(pkg,loc,i+1) & i<j at(pkg,loc,j)
Simplifying Assumption
incity(airport,city) & at(pkg,loc,goal) & incity(airport,city)
unload(pkg,plane,airport)
Adding Control KnowledgeAdding Control Knowledge
ProblemSpecification
Axioms
Domain-specific Control Axioms
Instantiated Clauses
SAT Simplifier
SAT Engine
SAT “Core”
As control knowledge increases, Core shrinks!
Effect of Domain KnowledgeEffect of Domain Knowledge
problem walksat walksat +
Kx
DPLL DPLL +
Kx
rocket-b 0.04 sec 0.04 sec 1.8 sec 0.13 sec
log-a 2.2 sec 0.11 sec * 1.8 min
log-b 3.4 sec 0.08 sec * 11 sec
log-c 2.1 sec 0.12 sec * 7.8 min
log-d 7.2 sec 1.1 sec * *
Hand created SAT encodings
* indicates no solution found after 24 hours
6. Learning Domain-Specific Control Knowledge
6. Learning Domain-Specific Control Knowledge
Learning Control RulesLearning Control Rules
Axiomatizing domain-specific control knowledge by hand is a time consuming art…
• Certain kinds of knowledge can be efficiently deduced
– simple classes of invariants (Fox & Long; Gerevini & Schubert)
• Can more powerful control knowledge be automatically learned, by watching planner solve small instances?
Form of RulesForm of Rules
We will learn two kinds of control rules, specified as temporal logic programs
– (Huang, Selman, & Kautz 2000)
• Select rule: conditions under which an action must be performed at the current time instance
• Reject rule: conditions under which an action must not be performed at the current time instance
incity(airport,city) & GOAL(at(pkg,loc)) &incity(airport,city)
unload(pkg,plane,airport)
Training ExamplesTraining Examples
Blackbox initially solves a few small problem instances
Each instance yields• POSITIVE training examples – states at which
actions occur in the solution
• NEGATIVE training examples – states at which an action does NOT occur, even though its preconditions hold in that state
Note that this data is very noisy!
Rule InductionRule Induction
Rules are induced using a version of Quinlan’s FOIL inductive logic programming algorithm
• Generates rules one literal at time
• Select rules: maximize coverage of positive examples, but do not cover negative examples
• Reject rules: maximize coverage of negative examples, but do not cover positive examples
• Prune rules that are inconsistent with any of the problem instances
– For details, see “Learning Declarative Control Rules for Constraint-Based Planning”, Huang, Selman, & Kautz, ICML 2000
Logical Status of Induced RulesLogical Status of Induced Rules
Some of the learned rules could in principle be deduced from the domain operators together with a bound on the length on the plan
• Reject rules for unnecessary actions
But in general: rules are not deductive consequences
• Could rule out some feasible solutions
• In worst case: could rule out all solutions to some instances
– not a problem in practice: such rules are usually quickly pruned in the training phase
Effect of LearningEffect of Learning
problem horizon blackbox learning
blackbox
grid-a 13 21 4.8
grid-b 18 74 16.6
gripper-3 15 >7200 7.2
gripper-4 19 >7200 260
log-d 14 15.8 5.7
log-e 15 3522 291
mystery-10
8 >7200 47.2
mystery-13
8 161 12.2
AIPS-98 competition benchmarks
SummarySummary
• Close connections between much work in AI Planning and CADE/CAV work on model checking
• Remarkable recent success of general satisfiability testing programs on hard benchmark problems
• Success of Blackbox and Graphplan in combining ideas from planning and verification suggest many more synergies exist
• Techniques for learning and applying domain specific control knowledge dramatically boost performance for planning – could ideas also be applied to verification?