Incremental Construction of Minimal Acyclic Deterministic ... · Incremental and semi-incremental algorithms for acyclic automata •Incrementalalgorithmfor(lexicographically)sorteddata(Daciuk,

Incremental Construction of Minimal Finite

State Automata

Jan Daciuk

Gdansk University of Technology

e-mail: [email protected]

Overview

• Incrementality and semi-incrementality

• Trie construction, minimization, synchronization

• Incremental algorithm for sorted data

• Unsorted data and confluence states

• Incremental algorithm for unsorted data

• Performance

Incrementality and automata

• Final automata

� ideal implementation of dictionaries

� very efficient once constructed

� traditional construction needs much memory

• Incremental and semi-incremental construction

requires less memory

• Moore’s law

Traditional construction

• Construct a trie

a i m

a e

n

t

r

a i

sz

i

on s

i

e

n t

si

en t

st

e z

o n s

e z

o n s

t

• Minimize it

a i m

a e

ri

z

o

n a

i

s

i

e

nt

st

e

z

sno

Incremental vs. semi-incremental construction

algorithms

start

add word

minimize

start

add word

last?

minimize

last?

stop

stop

reduce

Incremental and semi-incremental algorithms

for acyclic automata

• Incremental algorithm for (lexicographically) sorted data (Daciuk,

Mihov, Ciura, Deorowicz)

• Incremental algorithm for unsorted data (Aoe, Morimoto, Hase,

Sgarbas, Fakotakis, Kokkinakis, Daciuk, Watson, Revuz. . . )

• Semi-incremental algorithm for data lexicographically sorted on

reversed strings (Revuz)

• Semi-incremental algorithm for data sorted on decreasing length

of strings (Watson)

Construction of the trie

a i m i e z


a i me

ri

e z

i e z


a i me

ri

e z

i e z


a i me

r

a

i

en t

ie z

i e z

What is actually minimization?

• M = (Q,Σ, δ, q0, F ), |M | = |Q|

• M is minimal iff ∀M ′:L(M ′)=L(M) |M | < |M ′|

• −→L (q) = {w : δ∗(q, w) ∈ F}, L(M) =−→L (q0)

• p ≡ q iff−→L (p) =

−→L (q).

• M is minimal iff ∀p,q∈Q p ≡ q ⇔ p = q

• −→L (q) =⋃a:δ(q,a) 6=⊥ a

−→L (δ(q, a)) ∪ ∅ q 6∈ F

ε q ∈ F

What is minimization of a trie?

a i m

a

i

e

n t

st

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s


a i m

a e

n

t

r

a i

sz

i

on s

i

e

n t

si

en t

st

e z

o n s

e z

o n s

t


a i m

a

i

e

n t

st

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s


a i m

a

i

e

n t

st

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s


a i m

a

i

e

n

st

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

t


a i m

a

i

est

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn


a i m

a

i

est

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn


a i m

a

i

e

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

st


a i m

a

i

e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

es

t


a i m

a e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

es

i

t


a i m

a e

n

t

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

es

t

i


a i m

a e

nr

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

es

t

it


a i m

a e

r

a

i

en t

st

ie z

o n ss

z

i e z

o n son s

tn

es

t

in


a i m

a e

r

a

i

en

st

ie z

o n ss

z

i e z

o n son s

tn

es

t

in

t


a i m

a e

r

a

i

es

t

ie z

o n ss

z

i e z

o n son s

n

es

t

in

tn


a i m

a e

r

a

i

st

ie z

o n ss

z

i e z

o n son s

n

es

t

in

t

e


a i m

a e

r

a

i t

ie z

o n ss

z

i e z

o n son s

n

es

t

in

t

e

s


a i m

a e

r

a

i

ie z

o n ss

z

i e z

o n son s

n

es

t

in

t st

e


a i m

a e

r

a ie z

o n ss

z

i e z

o n son s

n

es

t

in

t

i


a i m

a e

ri

e z

o n ss

z

i e z

o n son s

n

es

t

in

t

a

The register

How does one check equivalence of two states?

• Use the recursive definition of−→L (q)

• Visit states using postorder, so that children have unique right

languages

• Keep pointers to states with unique−→L (q) in a sparse table

• Use hash function on finality and transitions

Result: Operations on the register are O(1)

Synchronization

• Add a word to the language of the automaton and minimize

the whole automaton again

� does not pose any restrictions on input data

� the same states have to processed over and over again – slow

• Add a word to the language of the automaton and minimize

the part that will not change in the future

� requires data to be sorted in some way

� faster due to procesing of states only when once

Synchronization – sorted data

o n ss

z

i e z

o n son s

a i me

r

a

i

e

n t

st

n

t

en t

s

ie z

a

i t

Synchronization – sorted data

o n ss

z

i e z

o n son s

a i me

r

a

i

e

n t

st

n

t

ie z

ts

a

i

en t

Incremental construction from sorted data1: function sorted construction;2: w′ ← ε;3: while input not empty do4: s← q0; i← 1; w ← next word;5: while i ≤ |w| and δ(s, wi)! = ⊥ do6: s← δ(s, wi); i← i + 1;7: end while;8: if i ≤ |w′| then repl or reg(δ(s, wi), w′i+1...|w′|); end if;

9: while i ≤ |w| do10: δ(s, wi)← new state; s← δ(s, wi); i← i + 1;11: end while;12: F ← F ∪ {s}; w′ ← w13: end while;14: repl or reg(q0, w′);15: end function;16: function repl or reg(q, v);17: if v 6= ε then18: δ(q, v1)← repl or reg(δ(q, v1), v2...|v|);19: end if;20: if ∃r∈R r ≡ q then21: delete q; return r;22: else23: R← R ∪ {q}; return q;24: end if;25: end function;








9: while i ≤ |w| do10: δ(s, wi)← new state; s← δ(s, wi); i← i + 1;11: end while;12: F ← F ∪ {s}; w′ ← w13: end while;14: repl or reg(q0, w′);15: end function;16: function repl or reg(q, v);17: if v 6= ε then18: δ(q, v1)←repl or reg(δ(q, v1), v2...|v|);19: end if;20: if ∃r∈R r ≡ q then21: delete q; return r;22: else23: R← R ∪ {q}; return q;24: end if;25: end function;





Incremental construction from sorted data –

examples

1: function sorted construction;2: w′ ← ε;3: while input not empty do4: s← q0; i← 1; w ← next word;5: while i ≤ |w| and δ(s, wi)! = ⊥ do6: s← δ(s, wi); i← i + 1;7: end while;8: if i ≤ |w′| then

repl or reg(δ(s, wi), w′i+1...|w′|);end if;

9: while i ≤ |w| do10: δ(s, wi)← new state;

s← δ(s, wi); i← i + 1;11: end while;12: F ← F ∪ {s}; w′ ← w13: end while;14: repl or reg(q0, w′);15: end function;16: function repl or reg(q, v);17: if v 6= ε then18: δ(q, v1)← repl or reg(δ(q, v1), v2...|v|);19: end if;20: if ∃r∈R r ≡ q then21: delete q; return r;22: else23: R← R ∪ {q}; return q;24: end if;25: end function;

a i m

a

i

e

n t

aimaient


examples





a i m

a

i

n t

e

aimais


examples





a i m

a

i

n t

e

aimais


examples





a i m

a

i

n

e

t

aimais


examples





a i m

a

i

n

e

t

aimais


examples





a i m

a

i

e

tn

aimais


examples





t

a i m

a

i

se

n

aimais


examples





a i m

a

i

e

n

s

t

aimait


examples





tn

a i m

a

i

es

aimait


examples





i

tn

a m

a

i

t

es

aimait


examples





i

tn

a m

a

i

est

aime


examples





tn

i

est

a

a i m

aime


examples





tn

i

a

a i m

est

aime


examples





tn

a

a i m

est

i

aime


examples





tn

est

i

a

a i me

aime


examples





tn

est

i

a

a i me

aiment


examples





tn

est

i

a

a i me

n

t

aiment

Unsorted data – hidden dangers

a i m

a e

ri

e

o n

z

o

n a

z

s

i

s

i

es

n

t

t


a

ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

ai


ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

i

a

a


a i m

a e

n

t

r

a i

sz

i

on s

i

e

n t

si

en t

st

e z

o n s

e z

o n s

t


a i m

a e

ri

z

o

n a

i

s

i

e

nt

st

e

z

sno


ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

i

a

a


a

ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

ai

i


a

ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

ai

i


a

ri

e

o n

z

o

n

z

s

i

s

es

n

t

t

a i me

a

st

ii

Incremental construction from unsorted data1: function unsorted construction;2: while input not empty do3: s← q0; i← 0; w ← next word; push(s, P );4: while i ≤ |w| and δ(s, wi) 6= ⊥ and fanin(δ(s, wi)) ≤ 1 do5: s← δ(s, wi); push(s, P ); i← i + 1;6: end while;7: R← R \ {s}; u← i;8: while i ≤ |w| and δ(s, wi) 6= ⊥ do9: δ(s, wi)← clone(δ(s, wi)); s← δ(s, wi); push(s, P ); i← i + 1;

10: end while;11: while i ≤ |w| do12: s← new state; s← δ(s, wi); push(s, P ); i← i + 1;13: end while;14: F ← F ∪ {s}; pop(P );15: while P not empty do16: if ∃r∈R r ≡ s then17: if i = u and i > 0 then R← R \ {top(P )}; u← u− 1; end if;18: delete s; δ(top(P ), wi)← r;19: else20: R← R ∪ {s}; if i = u then break; end if;21: end if;22: i← i− 1; s← pop(P );23: end while;24: reset P ;25: end while;26: end function;















Incr. constr. from unsorted data – examples

1: function unsorted construction;2: while input not empty do3: s← q0; i← 0; w ← next word; push(s, P );4: while i ≤ |w| and δ(s, wi) 6= ⊥

and fanin(δ(s, wi)) ≤ 1 do5: s← δ(s, wi); push(s, P ); i← i + 1;6: end while;7: R← R \ {s}; u← i;8: while i ≤ |w| and δ(s, wi) 6= ⊥ do9: δ(s, wi)← clone(δ(s, wi)); s← δ(s, wi);

push(s, P ); i← i + 1;10: end while;11: while i ≤ |w| do12: s← new state; s← δ(s, wi);

push(s, P ); i← i + 1;13: end while;14: F ← F ∪ {s}; pop(P );15: while P not empty do16: if ∃r∈R r ≡ s then17: if i = u and i > 0 then

R← R \ {top(P )}; u← u− 1;end if;

18: delete s; δ(top(P ), wi)← r;19: else20: R← R ∪ {s};

if i = u then break; end if;21: end if;22: i← i− 1; s← pop(P );23: end while;24: reset P ;25: end while;26: end function;

p e u t

peut






R← R \ {top(P )}; u← u− 1;end if;



p e u t

ve u t

veut






R← R \ {top(P )}; u← u− 1;end if;



p e u t

ve u

t

veut






R← R \ {top(P )}; u← u− 1;end if;



p e u t

ve

u

veut






R← R \ {top(P )}; u← u− 1;end if;



p e u t

v e

veut






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v

veut






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v e

veux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v e u

veux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v e ut

veux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v et

xu

veux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v et

ux

veux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v et

ux

peux






R← R \ {top(P )}; u← u− 1;end if;



e u tp

v e ux

xt

peux






R← R \ {top(P )}; u← u− 1;end if;



e up

v e ux

t

t

x

veux






R← R \ {top(P )}; u← u− 1;end if;



e up

v e

t

xu

veux






R← R \ {top(P )}; u← u− 1;end if;



e up

v

t

xe

veux






R← R \ {top(P )}; u← u− 1;end if;



e up t

xv

peux

Complexity and performance

• Both algorithms keep intermediate automata (almost) minimal

• Both run in time proportional to input data size

• The algorithm for sorted data is faster but less flexible

• Traditional algorithms are slower and use much more memory

• There are extensions of both algorithms to the case off adding

words to a cyclic automaton

Documents

Incremental Construction of Minimal Acyclic Deterministic ... · Incremental and semi-incremental algorithms for acyclic automata •Incrementalalgorithmfor(lexicographically)sorteddata(Daciuk,