Linear Quantifier Elimination as an

AbstractDecision

ProcedureNikolaj BjørnerMicrosoft Research

What

Why – actually

SMT Applications use Quantifiers

How Interleave Quantifier-Elimination

stepswith DPLL(T) loop.

What, Why and How

Linear QE is cool and macho

Should we call it Quantifier Termination?

Bug found by SLAyerusing Z3’s QE procedure

Linear QE is CLASSICAL

Long history:Presburger, Büchi, Cooper, Oppen, Fischer&Rabin, Pugh, Klaedtke, Boudet&Comon, Boigelot&Wolper, …

Many tools:REDLOG, -package, QEPCAD, LIRA, LDD, LASH, MONA, Mjolnir, Isabelle, HOL-light, ….

A Rough Picture of Current Approach

FourierMotzkin

Omega Test

Loos-Weispfennin

gCooper

Resolution

Case split+ Virtual subst

Abstract Decision

Proc

Abstract Decision

ProcCase split+ Resolution

Opportunity

SMT solvers use are good at Boolean combinations of quantifier free formulas.

is SAT

OpportunityAll-SMT enumerates satisfiable branches

has 8 satisfiable cases. Shorter than

OpportunityAll-SMT enumerates satisfiable branches

Can be used for DNF enumeration For QE procedures tuned to DNF[Monniaux LPAR 2008]

Minimize monomesCompares several different QE procedures

Also suggested in [de Moura, Ruess, Sorea CAV 2003]

OpportunityLinear Quantifier Elimination in Verification

SLAyer: A Separation Logic Prover

Y Symbolic Execution and Abstraction

Predicate Abstraction:[Chaki, Gurfinkel, Strichmann FMCAD 09]Linear Decision Diagrams LDD

Any news?Virtual Substitutions = Bounds + Resolution

Embed QE case splits into DPLL(LA)

A new twist on Presburger QE:Cooper + Resolution from the -testDistributed Divisibility Constraints

Practicalities:Use LA solvers to prune search earlySolve integer equalitiesParallel vs. Sequential EliminationHandling finite range arithmetic efficiently

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

𝑡1 𝑡 3

𝑠1

𝜑 [𝑥<𝑡1 ,𝑥<𝑡2 ,𝑥=𝑡3 ,𝑥>𝑠1 ,𝑥>𝑠2]𝒕𝒓𝒖𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆

𝑡 2

𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆

𝑠2

𝑡1 𝑖𝑠𝑙𝑢𝑏 . 𝑓𝑜𝑟 𝑥

𝑡1 𝑡 3

𝑠1

𝜑 [𝑥<𝑡1 ,𝑥<𝑡2 ,𝑥=𝑡3 ,𝑥>𝑠1 ,𝑥>𝑠2]

𝑡 2

𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆 𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆

𝑠2

𝑡 2<𝑥

𝑡1 𝑡 3

𝑠1

𝜑 [𝑥<𝑡1 ,𝑥<𝑡2 ,𝑥=𝑡3 ,𝑥>𝑠1 ,𝑥>𝑠2]

𝑡 2

𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆 𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆

𝑠2

𝑡 3=𝑥

𝑡1 𝑡 3

𝑠1 𝑠2

𝜑 [𝑥<𝑡1 ,𝑥<𝑡2 ,𝑥=𝑡3 ,𝑥>𝑠1 ,𝑥>𝑠2]

𝑡 2

𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆𝒕𝒓𝒖𝒆 𝒇𝒂𝒍𝒔𝒆𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆 𝒇𝒂𝒍𝒔𝒆𝒕𝒓𝒖𝒆𝒕𝒓𝒖𝒆𝑥𝑖𝑠𝑏𝑖𝑔𝑔𝑒𝑟 h𝑡 𝑎𝑛𝑡 1, 𝑡 2 , 𝑡3 , 𝑠1 ,𝑠2

𝑡1 𝑖𝑠𝑙𝑢𝑏 . 𝑓𝑜𝑟 𝑥 ,𝑡 2𝑖𝑠 𝑙𝑢𝑏 . 𝑓𝑜𝑟 𝑥 , 𝑡3=𝑥 , 𝑥𝑖𝑠𝑏𝑖𝑔𝑔𝑒𝑟 h𝑡 𝑎𝑛𝑡 1, 𝑡 2 , 𝑡3 , 𝑠1 ,𝑠2

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

𝑥𝑖𝑠𝑙𝑎𝑟𝑔𝑒𝑥𝑖𝑠𝑡𝑘𝑙𝑢𝑏 .𝑜𝑓 𝑥 𝑖𝑠 𝑡𝑖

Loos-Weispfenning Abstract QE(LRA)

Terms

Atoms

Formulas

Loos-Weispfenning Abstract QE(LRA)

¿ 𝑖¬ (𝑥<𝑡𝑖 )∧¿𝑘¬ (𝑥=𝑡𝑘 )∧ ¿ 𝑗 𝑥>𝑠 𝑗¿¿

𝑥=𝑡𝑘∧¿𝑘 ′ (𝑥=𝑡𝑘 ′→ 𝑡𝑘=𝑡𝑘 ′ )∧ ¿ 𝑖 (𝑥<𝑡𝑖→𝑡𝑘<𝑡𝑖 )∧ ¿ 𝑗(𝑥>𝑠 𝑗→ 𝑡𝑘>𝑠 𝑗)¿ ¿

𝑥<𝑡𝑖∧¿𝑘¬ (𝑥=𝑡𝑘 )∧¿ 𝑖 ′ (𝑥<𝑡𝑖 ′→𝑡 𝑖≤𝑡 𝑖 ′ )∧¿ 𝑗 (𝑥>𝑠 𝑗→𝑡 𝑖>𝑠 𝑗)¿¿

Loos-Weispfenning Abstract QE(LRA)

¿ 𝑖¬ (𝑥<𝑡𝑖 )∧¿𝑘¬ (𝑥=𝑡𝑘 )∧ ¿ 𝑗 𝑥>𝑠 𝑗¿¿

𝑥=𝑡𝑘∧¿𝑘 ′ (𝑥=𝑡𝑘 ′→ 𝑡𝑘=𝑡𝑘 ′ )∧ ¿ 𝑖 (𝑥<𝑡𝑖→𝑡𝑘<𝑡𝑖 )∧ ¿ 𝑗(𝑥>𝑠 𝑗→ 𝑡𝑘>𝑠 𝑗)¿ ¿

𝑥<𝑡𝑖∧¿𝑘¬ (𝑥=𝑡𝑘 )∧¿ 𝑖 ′ (𝑥<𝑡𝑖 ′→𝑡 𝑖≤𝑡 𝑖 ′ )∧¿ 𝑗 (𝑥>𝑠 𝑗→𝑡 𝑖>𝑠 𝑗)¿¿

𝜑 [ 𝑥↦∞ ]

𝜑 [𝑥↦𝑡 𝑖−𝜖 ]𝜑 [𝑥↦𝑡𝑘 ]

The Abstract Decision Procedure

propagate decide

decide

decide

Eliminate x

𝑥<𝑡𝑖∧¿𝑘¬ (𝑥=𝑡𝑘 )∧¿ 𝑖 ′ (𝑥<𝑡𝑖 ′→𝑡 𝑖≤𝑡 𝑖 ′ )∧¿ 𝑗 (𝑥>𝑠 𝑗→𝑡 𝑖>𝑠 𝑗)¿¿¿ 𝑖¬ (𝑥<𝑡𝑖 )∧¿𝑘¬ (𝑥=𝑡𝑘 )∧ ¿ 𝑗 𝑥>𝑠 𝑗¿¿

[x↦φ

Non-chronological backtracking works across elimination splits

Cooper+ Abstract QE(LIA)Terms

Atoms

Formulas

Cooper+ Abstract QE(LIA)

¿ 𝑖¬ (𝑎𝑥≤ 𝑡𝑖 )∧ ¿ 𝑗 (𝑏𝑥 ≥𝑡 𝑗 )¿

𝑎𝑖𝑥 ≤ 𝑡𝑖∧¿ 𝑖 ′ (𝑎𝑖 ′ 𝑥≤ 𝑡𝑖 ′→𝑎𝑖 ′ 𝑡𝑖≤𝑎𝑖𝑡 𝑖 ′ )∧¿ 𝑗¿¿

𝜑 [ 𝑥↦∞ ]

𝜑 [𝑥↦ ⌊𝑡𝑖𝑎𝑖⌋ 𝑖𝑠 𝑙𝑢𝑏 .]

Cooper+ Abstract QE(LIA)Resolving integer inequalities:

(∃𝑥 .𝑎𝑥≤ 𝑡∧𝑏𝑥 ≥𝑠 )≡𝑟𝑒𝑠𝑜𝑙𝑣𝑒 (𝑎𝑥≤ 𝑡 ,𝑏𝑥 ≥𝑠 )

n x m-ary version in [Pugh 92]

Cooper+ Abstract QE(LIA)

𝛿=𝑙𝑐𝑚 (𝑐𝑘 )−1

𝑥↦𝑥 𝛿+𝑢

Eliminating divisibility

PracticalitiesUse LA solvers to prune search early

Efficient LA solvers eliminate infeasible casesIdentify satisfiable pure formulas

Linear Diophantine Equation solving, e.g., [Pugh 92]

Elimination Order: Sequential vs. Parallel

Handling finite range arithmetic efficiently In context of Z3: Reduce finite range arithmetic to bit-vector theory

𝑦 𝑥∃𝑥𝑦 𝜑 ∃𝑥𝜓 𝜃 𝑦𝑥

∃𝑥𝑦 𝜑 𝜃

Selective ExperimentsFM/-SMS: All-SMT loop +Fourier-Motzkin eliminationLW/C-SMT: All-SMT loop +Cooper/LW eliminationLW/C-Plain: Only SMT on pure formulas.Mix-Model: Use Model to guide split. Mix-SMT: Method presented here.

Would have been much worse without SMT on pure formulasSMT is a waste of time on random formulas

Mix-SMT cheaper than DNF based branching

SummaryLinear Quantifier Eliminination Integrated as an abstract decision procedure.

Similar procedures for other theories:Term AlgebrasArrays (very partially)

Available in Z3 using ELIM_QUANTIFIERS=true

Term Algebra (and co-term algebras)Terms

Atoms

Formulas

𝑢𝑖 ,𝜓 𝑖=𝑠𝑜𝑙𝑣𝑒𝑥(𝑡 ¿¿ 𝑖 [ 𝑥 ]=𝑠𝑖)¿