A graphical strategy language for proof re-use · Motivations • Certain formal methods follow a reﬁnement-based posit-and- prove approach • stepwise development of system •

A graphical strategy language for proof re-use

Gudmund Grov Heriot-Watt University

with contributions from: Lucas Dixon, Alan Bundy, Ewen Maclean & other AI4FM project partners

In summary...From an expert provided proof(s), we want to

• ... extract high-level proof strategy

• ... to be re-used for “similar” conjectures

• ... in order to increase proof automation

background

In summary...From an expert provided proof(s), we want to

• ... extract high-level proof strategy

• ... to be re-used for “similar” conjectures

• ... in order to increase proof automation

background

In this talk we will introduce a graph-based strategy language to achieve this

1. Motivation & background

2. Representation of proof strategies as graphs

3. Strategy extraction in the language

Motivations• Certain formal methods follow a refinement-based posit-and-

prove approach

• stepwise development of system

• user posits a specification and justifies it by proof

• typically utilising interactive and automatic theorem provers

• Industrial application

• hundreds of conjectures require user interaction

• many follow a similar idea/strategy

• specification often changes ↝ slight changes to proof

• refinement-based methods ↝ may happen at abstract step

background

Motivations• Certain formal methods follow a refinement-based posit-and-

prove approach

• stepwise development of system

• user posits a specification and justifies it by proof

• typically utilising interactive and automatic theorem provers

• Industrial application

• hundreds of conjectures require user interaction

• many follow a similar idea/strategy

• specification often changes ↝ slight changes to proof

• refinement-based methods ↝ may happen at abstract step

Can we extract a strategy from one (or a few) proof(s), and re-use this for similar conjectures?

background

Motivating examplesI needed to prove the following lemma lemma 1: fixes S :: "(real^'n) set" assumes "aff_dim S = CARD('n)" shows "affine hull S = (UNIV :: (real^'n) set)" The definition of affine dimension is similar to the definition of dimension in Isabelle (“dim”), ... lemma 2: fixes S :: "(real^'n) set" assumes "dim S = CARD('n)" shows "subspace hull S = (UNIV :: (real^'n) set)"

From Isabelle list [April 2010]:

background

Motivating examplesI needed to prove the following lemma lemma 1: fixes S :: "(real^'n) set" assumes "aff_dim S = CARD('n)" shows "affine hull S = (UNIV :: (real^'n) set)" The definition of affine dimension is similar to the definition of dimension in Isabelle (“dim”), ... lemma 2: fixes S :: "(real^'n) set" assumes "dim S = CARD('n)" shows "subspace hull S = (UNIV :: (real^'n) set)"

• Need a lemma from an almost similar development

• which requires ~50 intermediate lemmas

• User wants to prove lemma by analogy

From Isabelle list [April 2010]:

background

Motivating examples assume "opt1 fN o HeadMap g1 = HeadMap g2o fA" and ... hence "opt1 fN'o opt1 fNo HeadMap g1 = HeadMap g3o fA'o fA" using o_assoc[of "opt1 fN'" "opt1 fN" "HeadMap g1"] and o_assoc[of "opt1 fN'" "HeadMap g2" "fA"] by auto thus "opt1 (fN'o fN)o HeadMap g1 = HeadMap g3o (fA'o fA)" using opt1_o[of "fN'" "fN"] by auto next assume "opt1 fN o TailMap g1 = TailMap g2 o fA" and ... hence "opt1 fN' o opt1 fNo TailMap g1 = TailMap g3o fA'o fA" using o_assoc[of "opt1 fN'" "opt1 fN" "TailMap g1"] and o_assoc[of "opt1 fN'" "TailMap g2" "fA"] by auto thus "opt1 (fN' o fN)o TailMap g1 = TailMap g3 o (fA'o fA)" using opt1_o[of "fN'" "fN"] by auto

From Isabelle list [January 2012]:

background

Motivating examples assume "opt1 fN o HeadMap g1 = HeadMap g2o fA" and ... hence "opt1 fN'o opt1 fNo HeadMap g1 = HeadMap g3o fA'o fA" using o_assoc[of "opt1 fN'" "opt1 fN" "HeadMap g1"] and o_assoc[of "opt1 fN'" "HeadMap g2" "fA"] by auto thus "opt1 (fN'o fN)o HeadMap g1 = HeadMap g3o (fA'o fA)" using opt1_o[of "fN'" "fN"] by auto next assume "opt1 fN o TailMap g1 = TailMap g2 o fA" and ... hence "opt1 fN' o opt1 fNo TailMap g1 = TailMap g3o fA'o fA" using o_assoc[of "opt1 fN'" "opt1 fN" "TailMap g1"] and o_assoc[of "opt1 fN'" "TailMap g2" "fA"] by auto thus "opt1 (fN' o fN)o TailMap g1 = TailMap g3 o (fA'o fA)" using opt1_o[of "fN'" "fN"] by auto

• Two branches within the same proof

• the only difference is TailMap & HeadMap

• User wants to prove second branch by similarity

From Isabelle list [January 2012]:

background

Motivations• There is a need for proving by similarity/analogy

• formal methods

• within proofs & between proofs

• full developments...

• often only parts of the proof can be “reused”

• There are (slight) variation between proofs

• symbols, steps, theorems, intermediate lemmas, rules, theory, ...

• In addition, large developments available

• Gontier’s 4 colour theorem, Hales’ Flyspeck, Isabelle’s AFP

• can automation be extracted from these?

background

LCF in a hurrybackground

• Theorem provers have a strong soundness requirement

• The LCF approach (developed in the 1970’s)

• declare an abstract type thm in the ML language

• create a trusted kernel of

• axioms: thm

• inference rules: thm → thm

• an element of type thm can only be create via these rules

• soundness reduced to the kernel and ML type system

• Many theorem provers follow this approach

• HOL, HOL-light, Coq, Isabelle,...

LCF in a hurry- tactics and tacticals -

background

• Forwards proofs by inference rules

• Backwards proof construction by ML programs called tactics

• tactic : goalstate → goalstate seq

• seq indicates branching

• turns a goal into a list of subgoals

• new tactics from old by a set of tactical combinators, e.g.

• a THEN b - sequential composition of a and b

• REPEAT a - repeats a (typically until failure)

• a OR b - executes a or b

• TRY a ORELSE b - executes b only if a fails

Three levels of proofs

[g1]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

cg5 g6

apply c; [g5,g6,g4]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

cg5 g6

apply c; [g5,g6,g4]

d

apply d; [g6,g4]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

cg5 g6

apply c; [g5,g6,g4]

d

apply d; [g6,g4]

e

apply e; [g4]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

cg5 g6

apply c; [g5,g6,g4]

d

apply d; [g6,g4]

e

apply e; [g4] fg7

apply f; [g7]

background

tactic goal tree


[g1] g1

ag2

apply a; [g2]

bg3 g4

apply b; [g3,g4]

cg5 g6

apply c; [g5,g6,g4]

d

apply d; [g6,g4]

e

apply e; [g4] fg7

apply f; [g7]gapply g; []

background

tactic goal tree

Goal level view- reuse in automatic provers -

• Ignores structure

•Captures theorems + hypothesis used

•Naive bayes + function abstraction

• SVM & graphical kernels

• Successfully used to guide auto provers

• Source: interactive & automatic proofs

•MaLare & Learning2Reason

g1

background

Goal level view- reuse in automatic provers -

• Ignores structure

•Captures theorems + hypothesis used

•Naive bayes + function abstraction

• SVM & graphical kernels

• Successfully used to guide auto provers

• Source: interactive & automatic proofs

•MaLare & Learning2Reason

g1

thms/hyps used

background

Tactic level view- learning regular expression language -

a

b

c

d e

f

g

•Abstracts over goals

• only addresses tactic combinations

• Learning regular expression language

• a,b := t | [a,b] | [ a | b] | a*

• maps to tacticals

•Duncan’s PhD + LearnOmatic

• markov chains

• genetic algorithms

background

Goals vs tacticsstrategy language

• looses all structure!

• replaying strategy

• which tactic created goal?

• goal states large & complex

• which properties are important?

Goal based abstraction

Goals vs tacticsstrategy language

• contains no information of the goals

• which goal goes where?

• which branch does a tactic apply to? e.g. (a THEN ... THEN g)

• termination of iteration?

• when should we apply a strategy?

• tactics are sensitive to small changes

Tactic based abstraction







Goals vs tactics

• We are interested in the structure of proofs

• Both goal & tactic information required

• Abstraction of goals required

strategy language

• contains no information of the goals

• which goal goes where?

• which branch does a tactic apply to? e.g. (a THEN ... THEN g)

• termination of iteration?

• when should we apply a strategy?

• tactics are sensitive to small changes

Tactic based abstraction







A graph based language• Why use a graph?

• combines goal and tactic information

• provides a typing mechanism for tactics

• repetition and goal-branching explicit

• nice inherent abstractions

• symmetry

• abstracts over evaluation and search strategies

• strategy extraction by graph rewriting

• handles “partiality” of strategies

strategy language

Typed open graphs• Developed by Kissinger, Dixon et al

• Implemented in Quantomatic

• Dangling edges via special boundary vertices

• Edges have types

• Semantics for plugging (push-out) & rewriting (double push-out) over boundary

strategy language

Typed open graphs• Developed by Kissinger, Dixon et al

• Implemented in Quantomatic

• Dangling edges via special boundary vertices

• Edges have types

• Semantics for plugging (push-out) & rewriting (double push-out) over boundary

strategy language

Plugging example

[source: Dixon & Kissinger: Open Graph and Monoidal Theories]

Proof strategies as graphs• Vertices encapsulates tactics

• or nest other graphs

• All edges are directed and typed

• illustrated by colours

• description of (properties of ) goals

• ensure correct flow of goals

strategy language

S







strategy language

S

Notation:empty/source target

=boundary vertex







strategy language

S

• Boundary vertices (and types) are used to

• combine strategies correctly (as in open graphs)

• act as I/O for goals during evaluation

• to support partial evaluation

• Only one I/O edge of a given type

Example strategy

induct

simp

goal

base step

rewr

strategy language

Example strategy- natural numbers -

induct

simp rewr

P (x:nat)

P 0 P n ⊢ P s(n)

strategy language

Example strategy- natural numbers (2) -

induct

simp rewr

P (x:nat)

P 1P n ⊢ P s(s(n))

P 0

strategy language

Example strategy- lists -

induct

simp rewr

P (x:nat list)

P [] P xs ⊢ P (x::xs)

strategy language

TENSOR• X TENSOR Y - putting X and Y next to each other

• Push-out with span over the empty boundary

strategy language



strategy language

C

D

A

B

TENSOR =



strategy language

C

D

A

B

TENSOR =



strategy language

C

D

A

B

TENSOR =A

B

C

D



strategy language

• Ins(X TENSOR Y) = Ins(X) + Ins(Y)

• Outs(X TENSOR Y) = Outs(X) + Outs(Y)

C

D

A

B

TENSOR =A

B

C

D

THEN• X THEN Y - putting X after Y

• Combine X out-edges with Y in-edges of same type

• Push-out with boundary over Outs(X) and Ins(Y) with same type

strategy language




strategy language

C

D

A

B

THEN =




strategy language

C

D

A

B

THEN =




strategy language

C

D

A

B

THEN =

A

BC

D




strategy language

• Ins(X THEN Y) = Ins(X) + (Ins(Y) - Outs(X))

• Outs(X THEN Y) = Outs(Y) + (Outs(X) - Ins(Y))

C

D

A

B

THEN =

A

BC

D

Hierarchies• Details can be hidden by nesting graphs

• the graphs are HiGraph

• represented by a rewrite rule

strategy language




strategy language

A

BC

D




strategy language

A

BC

D

C

E

G

F




strategy language

• No loops allowed in set of such “unfold nesting rules”

• Can be used to represent common patterns

A

BC

D

C

E

G

F

OR• Graphs abstracts over evaluation

• ... no natural representation for OR-branching

• OR(X,Y) - a hierarchy with one rewrite rule for each branch

strategy language




strategy language

A

BC

D




strategy language

A

BC

D

C X




strategy language

A

BC

D

C X

C Y




strategy language

• Similar for TRY-ORELSE (different evaluation semantics)

A

BC

D

C X

C Y

The Edge...• Contains information about a goal

• for a graph this means

• correct wiring of graphs ensured

• goals are sent to the right place

• for a technique this means

• input edge - precondition

• output edges - postcondition

• filter of “interesting” parts of the goal state

• A goal must “satisfy” the edge it is on

• Represented by a set of features

• e.g: has_induct_var, embeds_in, of_shape, has_functions

strategy language

The Edge...- features -

• (∀ x. P x ⇒ Q xz) of_shape (∀ x. _)

• A ∧ (B ∨ C) of_shape(_ ∧ _)

of_shape:

strategy language


• (∀ x. P x ⇒ Q xz) of_shape (∀ x. _)

• A ∧ (B ∨ C) of_shape(_ ∧ _)

of_shape:

• embeds_in (f(x), f(g(x))

• embeds_in (f(x,y), h(f(g(x),h(j(y)))

embeds_in:

strategy language


• (∀ x. P x ⇒ Q xz) of_shape (∀ x. _)

• A ∧ (B ∨ C) of_shape(_ ∧ _)

of_shape:

• embeds_in (f(x), f(g(x))

• embeds_in (f(x,y), h(f(g(x),h(j(y)))

embeds_in:

• has_functions([f,g], f(g(x))

• has_functions([],_)

has_functions:

strategy language

Example use of features

induct

simp

goal

base ih |- step

rewr

strategy language

Example use of features

induct

simp

has_induct_var(goal)

not(embeds_in(*,base)) embeds_in(ih,step)

rewr

strategy language

Evaluating a strategystrategy language

• By consuming input goal-nodes nodes

• and producing output goal-nodes

• Achieved by rewriting (with “side-effects” on a partial proof)





















• All goals produced must “satisfy” one output wire - fails if not

• A number of goals may be produced per edge

• Evaluating of vertex may introduce branching

• Hierarchy - evaluate nested graph

• OR - sum of evaluating nested graphs

Evaluating a strategy

S1

S3 S4

S2

g1g1

strategy language


S1

S3 S4

S2

g1

ag2 g3

g2 g3

S1

strategy language


S1

S3 S4

S2

g1

ag2 g3

bg4

c

g3

S1

S3

strategy language


S1

S3 S4

S2

g1

ag2 g3

bg4

c

dg5 g6

g5

g6

S1

S3 S4

strategy language


S1

S3 S4

S2

g1

ag2 g3

bg4

c

d

e

g6

g7 g8

S1

S3 S4

g5

g7g6g5

strategy language


S1

S3 S4

S2

g1

ag2 g3

bg4

c

d

e

g6

g7 g8

S1

S3 S4

g5

g7g6

strategy language

fS2


S1

S3 S4

S2

g1

ag2 g3

bg4

c

d

e

g6

g7 g8

S1

S3 S4

g5

g7

strategy language

gfS2


S1

S3 S4

S2

g1

ag2 g3

bg4

c

d

e

hgf

g5g6

g7 g8

S1

S3 S4

S2

strategy language

Extracting strategy

• Our main goal is to reuse proof

• Meaning from expert provided proof(s) we want to extract high-level proof strategies

• Extraction from one proof:

• try to match subpart with existing known strategies

• apply known generalisation

• match/generalise with new conjecture

• rely on user-annotations (Newcastle)

future work

Extracting strategy








future work

Extracting strategy- utilise existing known strategies -

d e g

g1

ag2

bg3 g4

cg5 g6

fg7

S1

S3

S2

Strategies

S4

future work


d e g

g1

ag2

bg3 g4

cg5 g6

fg7

S1

S3

S2

Strategies

S4

future work


d e g

cg5 g6

fg7

S1

g3 g4S1

S3

S2

Strategies

S4

future work


d e g

cg5 g6

fg7

S1

g3 g4S1

S3

S2

Strategies

S4

future work


d e g

g5 g6

fg7

S1

S3g4

S1

S3

S2

Strategies

S4

future work

S2


d e g

g5 g6

fg7

S1

S3g4

S1

S3

S2

Strategies

S4

future work

S2


g

fg7

S1

S3g4

S1

S3

S2

Strategies

S4

future work

S4


f

S1

S3

S2

g4

g

g7

S1

S3

S2

Strategies

S4

future work

S4


S1

S3

S2 g

S1

S3

S2

Strategies

S4g7

future work

S4


S1

S3

S2 g

S1

S3

S2

Strategies

S4g7

S2

future work

S4


S1

S3

S2

S1

S3

S2

Strategies

S4S2

future work

Extracting strategy








future work

Extracting strategy- known generalisations -

• There may be known rules we can apply to generalise the graph, e.g

future work

repetition:

A

AA



future work

repetition:

A

AA

combine:

A B OR(A,B)



future work

repetition:

A

AA

combine:

A B OR(A,B)

• Would require

• near/fuzzy matching

• generalisation/merging of edges/vertices

• measure of “how general”, e.g. “distance” between two graphs.

Extracting strategy








future work

Extracting strategy- exploit target conjecture -

future work

f(g(z),J(y))f(g(X),h(y))

Unification


future work

f(g(z),J(y))f(g(X),h(y))

Unification

f(g(z),h(y))

{X := z,J :=h} {X := z,J :=h}


future work

f(g(x),h(y))

Anti-unification

f(g(z),i(y))


future work

f(g(x),h(y))

Anti-unification

f(g(X),J(y))

{X := x,J :=h} {X := z,J :=i}

f(g(z),i(y))


• Anti-unification allow us to generalise the shape of a goal

• By anti-unifying with new conjecture one proof may be sufficient

• source has shape f(g(x),h(y))

• target has shape f(g(z),i(y))

• Gives the generalisation: f(g(X),J(y))

• This generalisation can then be applied over the full graph

• i.e. by rewriting: h = J and x = X (in the “source-graph”)

future work

Extracting strategy








future work

Extracting strategies> 1 proof

future work

A

B

C

D

• When several graph we can try to “anti-unify” (sub-)graphs

• least general generalised graph

• Merge/split/add/remove vertices

• Merge/split/add/remove edges


future work

A

B

C

D






future work

A

B

C’

D

C’






future work

A

B

C’

D

C’






future work

A

B

C’

D

C’






future work

C’

• Even stronger need for fuzzy/near matching & distance heuristics





Conclusion & beyond...

• Motivated and introduced a graph based proof strategy language

• indicated how to extract strategies from example proof(s)

• Implementation: combining Isabelle with Quantomatic

• Parsing/translation + annotation of proofs (w/Newcastle)

• “lifting” from goal-state to features

• We hope this could be a new way of writing proof strategies

• user program tactics & draws the connections

• inspect (partial) evaluation - will help with debugging

future work

Documents

A graphical strategy language for proof re-use · Motivations • Certain formal methods follow a reﬁnement-based posit-and- prove approach • stepwise development of system •