Theory of Relations (2)

.

.

. ..

.

.

Theory of Relations (2)Sequential Machines and Finite Automata

Course of MathematicsPusan National University

Yoshhiro Mizoguchi

Institute of Mathematics for IndustryKyushu University, [email protected]

November 3-4, 2011

Y. Mizoguchi (Kyushu Univ.) Theory of Relations (2) Sequential Machines and Finite AutomataNovember 3-4, 2011 1 / 34

Table of Contents

.. .1 Sequential Machine

PreliminaryReachable and ObservableMimimal Realization

.. .2 Finite Automata

Introduction to Theory of AutomataThe Myhill-Nerode theoremMinimal Realization

.. .3 Applications of Relational Calculus to Theory of Automata

Nondeterministic Finite AutomatonCoproduct and Product AutomatonReverse, Concatenate, ClosureExamples

.. .4 References


Sequential Circuit (1)

1 0 0 1 0 0 0

0 1 0 0 0 1 0

1 2 0 1 0 2 00 1 0 0 1 1 0

R

S Q

�@

�@

cc

cc��

��

One of the elementary units of sequential circuits, the RS Flip-flop circuitproduces an output signal sequence according to the sequence of inputsignals. Output signals are 1(on) or 0(of f ) so we write the outputs asY = {0, 1}. There are two input signals S(et) and R(eset). We consider thepair (SR) of S and R, and we denote 0 = (00), 1 = (01) and 2 = (10). Sowe can consider the inputs as X = {0, 1, 2}.The model of a sequential circuit consists of the state set Q = {a, b}, thestate transition function δ : Q × X → Q, and the output functionβ : Q → Y.



The table below is the value of δ and β. The figure is the state transitiondiagram of the RS Flip-flop circuit.

q x δ(q, x)a 0 aa 1 aa 2 bb 0 bb 1 ab 2 b

q β(q)a 0b 1 ��

-i

W Wa/0 b/1

0or1 0or2

21

The labels of a vertex consists of a state and an output symbol. An edgemeans a state transition and the label on an edge corresponds to the inputsymbol.



An sequential circuit can be considered as a function from an input word toan output word. That is we can consider an sequential circuit as a functionf : X∗ → Y∗ where X∗ is the set of words over X including an emptystring ε..Problem..

.

. ..

.

.

Construct an algebraic model (Sequential Machine) of sequentialcircuits using a state set, a state transition function and an outputfunction.

What kind of function from input words to output words is realizableby a finite state sequential machine?

How to construct an efficient sequential machine with small number ofstates.


Epi-Mono Factorization

A function F : X → Y from a set X to a set Y can be represented by acomposition F(x) = m(e(x)) (x ∈ X) of an injective function m : Z → Yand an surjective function e : X → Z. The set Z is uniquely determined byf up to isomorphism. That is Z � F(X) = {F(x) ∈ Y | x ∈ X} andZ � X/ ∼= {[x] | x ∈ X}, where the equivalent relation ∼ on X is definedby [x ∼ x′ ⇔ F(x) = F(x′)] and [x] = {x′ ∈ X | x ∼ x′} is a set ofequivalence class including x.(Note)A function e : X → Z is surjective if there exists an element x ∈ X satisfyinge(x) = z for any element z ∈ Z. A function m : Z → Y is injective if m(z1) , m(z2)for any two elements z1, z2 ∈ Z satisfying z1 , z2. We denote an injection by anarrow with tail�, and a surjective by an arrow with head�.


Sequential Machine

.Definition (Sequential Machine)..

.

. ..

.

.

A sequential mmachine is a sextuple M = (X,Q, δ, q0, Y, β) where

X is the set of inputs,Q is the set of states,δ : Q × X → Q is the transition function,q0 ∈ Q is the initial state,Y is the set of outputs, andβ : Q → Y is the output map.

(Note) This sequential machine (SM) is called Moore style SM. The Mealy styleSM is defined by an alternate output map λ : Q × X → Y insted of β. These twomodel are equivalent. Mealy style does not have an output for the initial state, butthe rest of relations between inputs and outputs are mutually transformable.We sometime define SM by pentad without an initial state.


Run map and Response map

.Definition (Run map)..

.

. ..

.

.

Let δ : Q × X → Q be a transition function. We define its run map to bethe unique map δ∗ : Q × X∗ → Q defined inductively by δ∗(q, ε) = q, andδ∗(q, xw) = δ∗(δ(q, x),w) (q ∈ Q, x ∈ X, w ∈ X∗).

Let f : X∗ → Y be a function. We also define the function f∗ : X∗ → Y∗defined inductively by f∗(ε) = f (ε), and f∗(wx) = f∗(w) f (wx) (x ∈ X,w ∈ X∗)..Definition (Response map)..

.

. ..

.

.

Let M = (X,Q, δ, q0, Y, β) be a sequential machine. We define itsresponse map to be the map ( fM)∗ : X∗ → Y∗ where fM : X∗ → Y isdefined by fM(w) = β(δ∗(q0,w)) (w ∈ X∗).


Realization (1)

.Definition..

.

. ..

.

.

Let t : X∗ → Y∗ be a function. If there exists a sequential machine Msuch that t = ( fM)∗, then M is called a realization of t.

We call t : X∗ → Y∗ is realizable if there exisits a function f : X∗ → Ysuch that t = f∗..Proposition..

.

. ..

.

.

A function t : X∗ → Y∗ is realizable if and only if for any w ∈ X∗, x ∈ Xthere exists y ∈ Y such that t(wx) = t(w)y.

The condition is equivalent that the value t(wx) is depending on only wand is not depending on x. We call a function t : X∗ → Y∗ satisfying thecondition as a sequential function.


Realization (2)

.Proposition..

.

. ..

.

.

Let t : X∗ → Y∗ be a sequential function. Then there exist a sequentialmachine M which is a realization of t.

Let f : X∗ → Y be a function and t = f∗. We introduce two kinds ofsequential machines which is a realization of t.

MI = (X, X∗, δI, ε, Y, f )δI(w, x) = wx (w ∈ X∗, x ∈ X).

MT = (X, YX∗ , δT, f, Y, βT)YX∗ is the set of all maps from X∗ to Y, that is { f | f : X∗ → Y},δT( f, x) : X∗ → Y is defined by δT( f, x)(w) = f (xw) (x ∈ X, w ∈ X∗),and βT( f ) = f (ε) ( f ∈ YX∗).

We can verify easily f = fMI = fMT . We note that both MI and MT is nota finite sequential machine. That is the state set is not a finite set.


Finite Realization

.Problem..

.

. ..

.

.

What kind of sequential function f : X∗ → Y which have a finite staterealization?

The state sets of MT is infinite but the most of states are unreachable fromthe initial state f . If the states reachable from f is finite, then we canconstruct a finite sequential machine from MT. That is the condition tohave a finite representation is that the set Z = {δ∗

T( f,w) ∈ YX∗ |w ∈ X∗} is

finite.We define a function F : X∗ → YX∗ by F(w) = δ∗

T( f,w) (w ∈ X∗). F is

divided to the composition of a surjection and an injection. SinceZ = F(X∗), we have Z = X∗/ ∼ by the equivalence relation ∼ defined by[w ∼ w′ ⇔ δ∗

T( f,w) = δ∗

T( f,w′)]. We note δ∗

T( f,w)(z) = δ∗

T( f,w′)(z) is

f (wz) = f (w′ z) for any z ∈ X∗.If the number of the equivalence class is finite, then there exists a finiterealization.


Reachable and Observable

Let M = (X,Q, δ, q0, Y, β) be a sequential machine and its response mapfM : X∗ → Y. We define F : X∗ → YX∗ by F(w) = δ∗

T( fM,w).

F is divided into a composition of fe : X∗ → Q and fm : Q → YX∗ suchthat

F(w) = fm( fe(w)),

where

fe(w) = δ∗(q0,w), and

fm(q)(w) = β(δ∗(q,w)) (w ∈ X∗, q ∈ Q).

If fe is a surjection then we call M as reachable.

If fm is an injection then we call M as obserbable or reduced.

We note that a reachable and obserbable sequential machine isminimal representation of fM.


Minimal Realization

Let M = (X,Q, δ, q0, Y, β) be a sequential machine. and F = fm ◦ fe.Assume fe : X∗ → Q is a surjection. If fm : Q → YX∗ is not an injection,then we can construct a minimal realization using the epi-monofactorization of fm.The equivalence relation [q ∼ q′ ⇔ fm(q) = fm(q′)] on Q is thatfm(q)(w) = fm(q′)(w) for any w ∈ X∗. That is β(δ∗(q,w)) = β(δ∗(q′,w)). IfQ is a finite set and |Q| = n, then it is sufficient to check the conditionβ(δ∗(q,w)) = β(δ∗(q′,w)) for finite number of words w with |w| < n. So wecan check q ∼ q′ for any state q and q′ in finite steps, and we canconstruct a minimal state sequential machine.


Introduction to Theory of Automata

The first published study on automata [6] is the Automata studies in 1956edited by C. E. Shanon who is famous as an originator of the informationtheory and J. McCarthy who is a famous researcher of the fields artificialintelligence.At that time, they formalized an abstract model of a sequentil circuit andinvestigated relationship between inputs and outputs analyzing a statetransition functions.Once the notion of accept states is introduced, a machine is considered asan acceptor and investigations of recognized language are started. This isthe origin of the theory of language and automata.The first paper [4] about finite state automata is ’finite Automata and TheirDecision Problems’ by M. O. Rabin and D. Scott pubshed in 1959. Theywere awarded an ACM Turing aword in 1976 for this research.


Alphabet, Word, Concatenation

An alphabet is a finite, nonempty set. The elements of an plphabet arefeferred to as letters, or symbols. A word over an alphabet is a finitestring consisting of zero or more letters of the alphabet, in which the sameletter may occur several times. The string consisting of zero letters iscalled the empty word, written ε. The length of a word w, denoted by |w|,is the number of letters in w. Again by definition, |ε| = 0.Let Σ be an alphabet. The set of all words over an alphabet Σ is denotedby Σ∗. The sets Σ∗ is infinite for any Σ. Algebraically speaking, Σ∗ is thefree monoid with the identity ε generated by Σ.For words w1 and w2, the juxtaposition w1w2 is called the concatenationof w1 and w2. The empty word is an identity with respect to concatenation,εw = wε = w holds for all word w.


Kleene Closure

Subsets of Σ∗ are referred to as formal languages, or briefly, languagesover Σ.The concatenation (or product) of two languages L1 and L2 is defined byL1L2 = {w1w2 |w1 ∈ L1, w2 ∈ L2}.The (Kleene) closure L∗ of a language L is defined to be the union of allpowers of L, that is L∗ = ∪∞

n=0Ln where L0 = {ε}, and Ln = Ln−1 · L(n ≥ 1).The closure of all words Σ is Σ∗ and there is no confusions defined as aset of all words over Σ.


Deterministic Automata (1)

.Definition..

.

. ..

.

.

Deterministic Automata A determinisiic finite automaton is a pentadM = (Σ,Q, δ, q0, F) where

Σ is the alphabet,Q is the finite set of states,δ : Q × Σ → Q is the transition function,q0 ∈ Q is the initial state, andF ⊂ Q is the set of accept states.


Deterministic Automata (2)

Controler(q)

Finite Automaton M

s1 s2 · · · sn

6 -

inputs

move head to right after a state transition

A finite automaton is illustrated as above figure. The input letters are oninput tape. The first state of controller is the initial state q0. If the state is qand input letter is s then the state is changed to δ(q, s). After changing thestate the head is moved to right. Repeating these procedures until the endof an input word. If the head reached to the end of an input word, thencheck the state is accept state or not.


Recognized Language

Let δ : Q × Σ → Q be a state transition function. The functionδ∗ : Q × Σ∗ → Q is uniquely determined by δ∗(q, ε) = q, andδ∗(q,wa) = δ(δ∗(q,w), a) (w ∈ Σ∗, a ∈ Σ)..Definition..

.

. ..

.

.

The recognized language L(M) ⊂ Σ∗ is defined by

L(M) = {w ∈ Σ∗ | δ∗(q0,w) ∈ F}.

L(M) is referred to as the language accepted by M.


Example

.Example..

.

. ..

.

.

Let M = (Q,Σ, δ, q0, F) be a finite automaton where Q = {q0, q1, q2},F = {q1}, q0 ∈ Q. The state transition function δ : Q × Σ → Q is definedas follows

δ(q0, a) = q1, δ(q0, b) = q2,δ(q1, a) = q2, δ(q1, b) = q0,δ(q2, a) = q2, δ(q2, b) = q2.

For example, the word w = aba is acceptable δ∗(q0, aba)= δ∗(δ(q0, a), ba) = δ∗(q1, ba) = δ∗(δ(q1, b), a) = δ∗(q0, a) = δ∗(δ(q0, a), ε)= δ∗(q1, ε) = q1 ∈ F.


State Transition Diagram

A finite automaton M is denoted by a following figure called state transitiondiagram. Vertices are states and the initial state has an arrow withoutlabel. According to the input letters follow arrows with same label withinput letter. The vertex corresponding to an accept state has double circleand if the following the input letters ended at the vertices with double circlethen the input word is accepted.

��

�?

��)i

�-

�

q2

q0

q1

a, bb

aa

b


The Myhill-Nerode theorem (1)

.Definition..

.

. ..

.

.

An equivalence relation ∼ on Σ∗ is said to be right invariant if”w1 ∼ w2 ⇒ w1 z ∼ w2 z (∀z ∈ Σ∗)” for any w1, w2 ∈ Σ∗.An equivalence relation ∼ is finite index if the number of equivalenceclasses is finite. That is {[x] | x ∈ Σ∗} is finite set where[x] = {x′ | x ∼ x′}.

Let L j Σ∗. We define a relation w1 ∼L w2 on Σ∗ by

w1 z ∈ L ⇔ w2 z ∈ L (∀z ∈ Σ∗).

The relation ∼L is a right invariant equivalent relation.


The Myhil-Nerode theorem (2)

.Theorem (Nerode)..

.

. ..

.

.

Let L j Σ∗ be a language on Σ. Then, the following three conditions (1),(2) and (3) are equivalent.

(1) The set L is acpted by some finite automaton.

(2) L is the union of some of the equivalent classes of a right invariantequivalence relation of finite index.

(3) The right invariant equivalence relation ∼L induced by L is of finiteindex.

Let L be the language L in (3) and Q = {[w] |w ∈ Σ∗}, δ([w], a) = [wa],q0 = [ε], F = {[w] |w ∈ L}. Then M = (Q,Σ, δ, q0, F) is a finite automatonand L = L(M). Further, if L = L(M′) for some finite automataM′ = (Q′,Σ, δ′, q′

0, F′) then |Q| ≤ |Q′|. That is M is a minimal state

automaton with L = L(M).


Minimal Realization

Let M = (X,Q, X, δ, q0, Y, β) be a sequential machine and Y = {0, 1}.There is an one-to-one coresspondence between an output mapβ : Q → {0, 1} and a subset F = {q ∈ Q | β(q) = 1} of Q. That is a finiteautomata is considered as a sequential machine with Y = {0, 1}.A finite sequential machine is exactly a finite automaton. Further, afunction f : X∗ → Y is corresponde to a subset of X∗ that is a recognizedlanguage.In the previous section, we construct a minimal realization of f using anequivalence relation w ∼ w′ defined by f (wz) = f (wz′) (∀z ∈ X∗). This isthe equivalence relation ∼L induced by L.


Example

.Example (Minimalize)..

.

. ..

.

.

The automton M′ in right figure is the minimalized automaton of M in leftfigure. We note p0 = {q0, q2}, p1 = {q1, q3}, and p2 = {q4, q5}.

� ��

� �� q0

q1q2

q3

q4q5

�?

HHja ��9a ��:bzb

XXyb

*b :a*a

W

bW

b

Wa

R

a

� �� p1

p0

p2

�?

�a HHjb-b

-aW

a, b

M M′


Nondeterministic Finite Automaton

Let Σ be an alphabet, I one point set {∗}..Definition (Kawahara 1988[3])..

.

. ..

.

.

A nondeterministic finite automaton (NFA) M = (Q, τ, δa(a ∈ Σ), β) is aquadruple where

Q is a finite set of states,τ : I ⇁ Q is the inital state relation,δa : Q ⇁ Q is the set of state-transition relations, andβ : Q ⇁ I is the final state relation.

For an input string w = σ1σ2 · · ·σn ∈ Σ∗ (σi ∈ Σ, 1 ≤ i ≤ n, 0 < n), the iterativestate-transition relation δw : Q ⇁ Q is defined by δw = δσ1δσ2 · · · δσn..Definition..

.

. ..

.

.

The language accepted by a NFA M is defined by

L(M) = {w ∈ Σ∗ | τδwβ♯ = idI}.


Coproduct Automaton

.Definition..

.

. ..

.

.

Let M = (Q, τ, δa(a ∈ Σ), β) and M′ = (Q′, τ′, δ′a(a ∈ Σ), β′) be NFAs. Thecoproduct automaton of M and M′ is defined byM + M′ = (Q + Q′, τ̂, δ̂a(a ∈ Σ), β̂) where τ̂ = τ ⊥ τ′, δ̂a = δa + δ

′a, and

β̂ = β ⊥ β′.

.Proposition..

.

. ..

.

.

Let M + M′ be the coproduct automaton of M and M′. Then

(a) δ̂w = δw + δ′w for w ∈ Σ∗.

(b) τ̂δ̂wβ̂♯ = τδwβ

♯ ⊔ τ′δ′w(β′)♯.(c) L(M + M′) = L(M) ∪ L(M′).


Product Automatan

.Definition..

.

. ..

.

.

Let M = (Q, τ, δa(a ∈ Σ), β) and M′ = (Q′, τ′, δ′a(a ∈ Σ), β′) be NFAs. Theproduct automaton of M and M′ is defined byM × M′ = (Q × Q′, τ̂, δ̂a(a ∈ Σ), β̂) where τ̂ = τ⊤τ′, δ̂a = δa × δ′a, andβ̂ = β⊤β′.

.Proposition..

.

. ..

.

.

Let M + M′ be the coproduct automaton of M and M′. Then

(a) δ̂w = δw × δ′w for w ∈ Σ∗.(b) τ̂δ̂wβ̂

♯ = τδwβ♯ ⊓ τ′δ′w(β′)♯.

(c) L(M × M′) = L(M) ∩ L(M′).


Reverse Automaton

.Definition..

.

. ..

.

.

Let M = (Q, τ, δa(a ∈ Σ), β) be a NFA. The reverse automaton of M isdefined by MR = (Q, τR, δR

a (a ∈ Σ), βR) where τR = β, δRa = δ

♯a and

βR = τ.

.Proposition..

.

. ..

.

.

Let MR be the reverse automaton of M. Then

(a) δRw = (δwR)♯ where wR is a reverse string of w ∈ Σ∗.

(b) τRδRw(βR)♯ = (τδwRβ♯)♯.

(c) L(MR) = L(M)R where L(M)R = {wR |w ∈ L(M)}.


Concatenate Automatan

.Definition..

.

. ..

.

.

Let M = (Q, τ, δa(a ∈ Σ), β) and M′ = (Q′, τ′, δ′a(a ∈ Σ), β′) be NFAs. Theconcatenate automaton of M and M′ is defined byM · M′ = (Q + Q′, τ̂, δ̂a(a ∈ Σ), β̂) where γ = β♯τ′, τ̂ = τ(i ⊔ γ j),δ̂a = i♯δai ⊔ i♯γδ′a j ⊔ j♯δ′a j, and β̂ = β′(γ♯i ⊔ j).

The function δow : Q ⇁ Q′ (w ∈ Σ∗) is uniquely determined by δo

ε = 0QQ′

and δowa = (δwγ ⊔ δo

w)δ′w..Proposition..

.

. ..

.

.

Let M · M′ be the concatenate automaton of M and M′. Then

(a) δ̂w = i♯δwi ⊔ i♯δow j ⊔ j♯δ′w j.

(b) δwγ ⊔ δow ⊔ γδ′w = ⊔w=uvδuγδ

′v.

(c) τ̂δ̂w(β̂)♯ = ∪w=uvτδuγδ′v(β′)♯.

(d) L(M · M′) = L(M) · L(M′).


Closure Automaton

.Definition..

.

. ..

.

.

Let M = (Q, τ, δa(a ∈ Σ), β) be a NFA. The coosure automaton of M+ isdefined by M+ = (Q, τ, δ+a (a ∈ Σ), β) where γ = β♯τ, and δ+a = (idQ ⊔ γ)δa.

.Proposition..

.

. ..

.

.

Let M+ be the closure automaton of M. Then

(a) δ+w = ⊔w=u1···uk,uk,ε,k>0δu1γδu2γ · · · γδuk for w , ε.

(b) τδ+wβ♯ = ⊔w=u1···uk,k>0τδu1γδu2γ · · · γδukβ

♯ for w , ε.(c) L(M+) = L(M)+.


Examples

.Proposition..

.

. ..

.

.

Let Mϕ = (I, τ, δa(a ∈ Σ), β) be a NFS where β = 0II. ThenL(Mϕ) = ϕ.

Let Mε = (I, τ, δa(a ∈ Σ), β)) be a NFS where τ = idI, δa = 0II(a ∈ Σ), and β = idI. Then L(Mε) = {ε}.Let σ ∈ Σ and Mσ = (I + I, δa(a ∈ Σ), β) where τ = i, δσ = i♯ j,δa = 0I+I,I+I (a , σ), and β = j. Then L(Mσ) = {σ}.

The Kleene closure automaton of M is defined by M∗ = Mε + M+. Thenwe haveT(M∗) = L(Mε + M+) = T(Mε) ∪ T(M+) = {ε} ∪ T(M)+ = T(M)∗.


Regular Language

.Definition..

.

. ..

.

.

A language L ⊂ Σ∗ is a regular if there exists a NFA M such thatL = L(M).

.Proposition..

.

. ..

.

.

Let L and L′ be regular languages. Then

L = ϕ, L = {ε} and L = {σ} (σ ∈ Σ) are regular languages.

Σ∗ − L, LR, L+ and L∗ are regular languages.

L ∩ L′, L ∪ L′ and L · L′ are regular languages.


References

[1] M.A.Arbib and E.G.Manes: Machines in a cagegory, an expositoryintroduction, SIAM Review, 16(1974), 163-192.

[2] J.E.Hopcroft and J.D.Ullman: Formal Languages and Their Relation toAutomata (2nd. Ed.), Addison-Wesley(2001).

[3] Y. Kawahara, Applications of relational calculus to computermathematics, Bulletin of Informatics and Cybernetics, 23(1988),67-78.

[4] M.O.Rabin and D.Scott: Finite Automata and Their DecisionProblems, IBM Journal, 3(1959), 114-125.

[5] A. Salomaa: Computation and Automata, Cambridge UniversityPress(1985).

[6] C.E.Shannon and J.Mac Carthy (eds.): Automata Studies, PrincetonUniversity Press(1956).


Education

Theory of Relations (2)