Lecture 16:

Modeling Computation

Languages and Grammars

Finite-State Machines with Output

Finite-State Machines with No Output

Natural Language vs. Formal Language

the large rabbit hops quickly

articleadjective

nounverb

adverbsentence

noun-phrase verb-phrase

article noun verb-phrase article adjective noun verb-phrase

article noun verb

article noun verb adverb

article adjective noun verb

article adjective noun verb adverb

Verifying a Sentence

Phrase-Structure Grammars

A phrase-structure grammar is a 4-tuple G = (V, T, S, P)

where

V = a vocabulary which is a finite, non-empty set of symbols comprised of a set of non-terminals, N and a set of terminals, T where V = N + T

P = a finite set of production rules. X -> Y where X and Y are each one or more symbols in V. Each production must contain at least one non-terminal on the left side.

S = a start symbol that is an element of V.

An Example

Show that "ba" is a word in the language of G.

Find another word in the language of G.

S -> ABa

A -> BB

B -> ab

AB -> b

S -> ABa -> ba

S -> ABa -> BBBa -> BBaba -> Bababa -> abababa

Example

S -> aA

S -> b

A -> aa

Example

S -> 11S

S -> 0

Example

Example

A grammar is is used to generate or evaluate members of a formal language.  the context-free grammar G1 below, generates the language {0n1n for n>=0}.

A -> 0A1

A -> f (the empty string)

A grammar is a set of production rules as shown above.  A production rule is comprised of a variable, a right-arrow, and a string of variables and terminals. 

The variables are used to control the arrangement of possible substitutions and the terminals are symbols from the alphabet of the language being generated or recognized. 

By convention, variables are represented by uppercase letters.  One of the variables is designated as the start variable, usually the left-hand symbol of the first rule in the grammar.

Introduction to Grammars

There are usually choices for applying production rules, which means that in order to create a particular member of a language we need to specify an order of application of the rules of the grammar, called a derivation. 

For example, the following is a derivation to show that 00001111 is a member of the language defined by G1.

start     A

rule 1    0A1

rule 1    00A11

rule 1    000A111

rule 1    0000A1111

rule 2    00001111

Applying Production Rules

All the member strings that can be generated through derivations using G1 are collectively referred to as the language of the grammar G1 or L(G1). 

We say that a context-free grammar is one in which candidate strings are accepted or rejected as members of L(G1) based strictly on their structures and not based on the context in which they appear.  The syntax of a programming language is (mostly) definable with a context-free grammar. 

For example, the reserved words of a programming language are parsed and recognized as reserved words regardless of their context. 

(Can you think of an example in a programming language in which context is important?)

The Language of the Grammar

Formal Definition of a Context-Free Grammar

A context-free grammar is a 4-tuple (V,S,R,S), where

1. V is a finite set of variables

2. S is a finite set, disjoint from V, called the terminals

3. R is a finite set of rules, with each rule being a variable and a string of variables and terminals, and

4. S is an element of V called the start variable.

If u, v, and w are strings of variables and terminals, and A -> w is a rule of the grammar, we say that uAv yields uwv, written uAv -> uwv. 

Write u v if u=v or if a sequence u1,u2,. . ., uk exists for k>=0 and,

u -> u1 -> u2 -> . . . -> uk -> v.

->*

A Practical Example - The following context-free grammar can be used to derive (or recognize) in-fix arithmetic expressions in x,y and z.  The start variable is S and the termials are +, -, *, /, (, ), x, y, and z.

S -> T+S | T-S | T T -> T*T | T/T | (S) | x | y | z

Use the grammar above to show that (x+y)/(x-y)*z is a valid in-fix expression.

S -> T -> T/T -> T/T*T -> ... -> (S)/(S)*T -> ... -> (T+S)/(T-S)*T -> ... -> (x+y)/(x-y)*z

A Practical Example

How do we show a particular string is not derivable with a context-free grammar?

How would we use a grammar to show that (x+y)*/z is not a valid in-fix expression? 

We have no means to choose which rules to apply or in which order to apply them. 

If a grammar generates the same string in more than one way, the grammar is ambiguous. 

Using a grammar to generate a candidate string can lead to an exponential number of alternate paths to test. 

What we need is a systematic method for applying the rules to show that a particular string is or is not derivable.

The Problem with Grammars

Chomsky Normal Form

A context-free grammar is said to be in Chomsky normal form (CNF) when every rule has the form

A -> BC A -> a

where a is any terminal and A, B, and C are any variables except B and C are not the start variable. A rule that replaces the start variable with nothing (i.e. S -> e ) is permitted. 

We can show that "Any context-free language is generated by a context-free grammar in Chomsky normal form."  Perhaps more important to us than the proof itself is the proof idea which we can use to convert a context-free grammar into a CNF grammar.

CNF grammars are very useful since they simplify derivations making them more easily implemented as computer algorithms.  The method of conversion of a context-free grammar into CNF is detailed below:

1. Create a new start variable.

2. Eliminate all rules of the form A -> .

3. Eliminate all unit rules of the form A -> B.

4. Convert the remaining rules into the form A -> BC or A -> a

Converting a Grammar into CNF

Example:  Lets work through an example of this conversion process. In the following, rules being eliminated will be shown in blue, rules being added will be shown in red.  Given the grammar,

we apply Step 1, creating a new start variable S0,

Now we need to eliminate rules of the form A->.  In order to do this without changing the grammar, we must also create replacement rules to permit the same derivations that were possible when the epsilon rules were present. 

For example, removing B -> requires that we add a rule S -> a to cover the derivation S-> aB -> a that would have been possible with the rule B -> .  Similarly we need to include the rule A -> to account for the A -> B -> derivation.

An Example

We still need to remove the A -> rule.

Next we remove all unit rules.  Removing  S0 -> S we must add S0 -> SA | AS | ASA | aB | a. We can remove S -> S without adding any other rules. 

When we remove A -> B we must add the rule A -> b.  Removing A -> S we must add A -> SA | AS | ASA | aB | a.

This gives us,

Finally we must make sure that each rule is in one of the two forms allowed in CNF.  Specifically we can have only rules that replace a single variable with a pair of variables or rules that replace a single variable with a single terminal.

Therefore we must modify the rules, S0 -> ASA, S -> ASA, S -> aB, A -> ASA, and A -> aB.  We create new variables and new rules to ensure that the grammar derivation is not changed.

The original grammar is shown on the left and the CNF of our grammar is on the right.

The CNF grammar is larger and less readable than our original grammar, so what is the advantage of the CNF grammar? We may learn more if we attempt to derive candidate strings using each form.

Derivation using the Original Grammar

The value of the CNF grammar begins to be revealed in these sample derivations.  Although the CNF grammar has a larger rule set, its derivations are significantly shorter.

Derivation using the CNF Grammar

Moore Machine - A Moore machine is a 6-tuple (Q, , , , q0, ) where,

(1) Q is a finite set of states.

(2) is a finite set of symbols called the input alphabet.

(3) is a finite set of symbols called the output alphabet.

(4): Q x -> Q is the transition function

(5) q0 is an element of Q called the start state, and

(6) : Q -> is the output function mapping the current state to the output.

Moore Machine

Example Moore Machine

Each of the six elements of this example Moore machine are given below,

Q = {1, 2, 3} = ([1,X]->2, [1,Y]->3, [1,Z]->1, [2,X] ->3, [2,Y]->1, [2,Z]->2, [3,X]->1, [3,Y]->3, [3,Z]->3)

= { X, Y, Z} q0 = 1

= {A, B} = (1->A, 2->B, 3->A)

Mealy Machine - A Mealy machine is a 6-tuple (Q, , , , q0, ) where,

(1) Q is a finite set of states.

(2) is a finite set of symbols called the input alphabet.

(3) is a finite set of symbols called the output alphabet.

(4): Q x -> Q is the transition function

(5) q0 is an element of Q called the start state, and

(6) : Q x -> is the output function mapping the current transition to the output.

Mealy Machine

Now let's look at how we build an FSM to find all occurrences of a particular substring in an input binary string. The first issue we need to resolve is whether we wish to allow overlap in the substrings. We can use the substring 1101 to illustrate substring overlap. Permitting overlap means that one or more bits can be part of more than one instance of a substring.

Consider the input string,

1101101101101101101

If we permit overlap, we find that the string contains six instances of the substring 1101. However, if we do not permit overlap, this string contains only three instances of the substring.

Moore Machine Substring Recognizer

Moore Machine: Recognizer for '1101' with Overlap

As a final example of bit string recognizers, we will build a Mealy machine to find the substring 1101 with overlap. Recall that a Mealy machine generates its output on the transition rather than upon entering a state. This may seem like a trivial difference, but it can sometimes permit us to reduce the number of states required.

Mealy Machine: Recognizer for "1101" with Overlap

Upon reading the final bit in the substring sequence, this machine outputs a 1, otherwise it outputs a 0 each time it reads an input bit.

This Mealy machine has one less state than the Moore machine designed to solve the same problem.

Mealy machines will always require the same or fewer states to solve the same problem as Moore machines.

Finite State Machines

Finite State Machines (FSMs) are theoretical models of computational systems. Finite state machines, also called finite automata, are used in the study of the theory of computation.

R = rightL = leftU = upD = down

Definition of a Finite State Machine - Formally, a finite Automaton (FA) is defined as a 5-tuple (Q,, , q0, F) where,

(1) Q is a finite set of states.

(2) is a finite set of symbols or the alphabet.

(3): Q x -> Q is the transition function

(4) q0 is an element of Q called the start state, and

(5) F is a subset of Q called the set of accept states.

Formal Definition

Q = {1, 2, 3, 4, 5, 6}

q0 = 1 (start state)

= {U, D, L, R}

F = {6}

An Example

An FSM to Recognize (Accept) Integers

Valid Integers Not an Integer 123 123.456 123456 12+345 -543 Hello There 9 9 3 5

How to Implement an FSM in a Computer Program

Example Binary String Recognizeraccepts strings containing at least three 1's

The double circle indicates the accept state, the start state is labeled (0), and the transitions between states are labeled according the input symbol currently being read. The binary strings below, are accepted or rejected as indicated,

01010000 reject

01001110 accept

0111 accept

110000 reject

Example Binary String Recognizeraccepts strings containing three consecutive 1's

Applying this FSM to the sample binary strings listed below gives the indicated results.

0000110011001100 reject

0001110000000000 accept

1010101010000000 reject

11111111 accept

Building Finite State MachinesConsider an FSM that accepts (or recognizes) strings containing the substring "1101". We create a start state and a state for each bit in the substring for which we are searching.

Accepting Binary Encoded Valuesdivisible by four

We can check the parity of 1's or 0's or both in a binary value using an FSM. In this example, we want an FSM that accepts binary strings with an even number of 0's and an even number of 1's. The start state is an accept state since the empty string has zero 0's and zero 1's (and zero is even).

Dual Parity Tester

Computational Limits of Finite State MachinesCounting 1's and 0's - Consider the design of an FSM that accepts all binary strings that have an equal number of 1's and 0's. At first this may seem to be a relatively simple problem; however, we need to deal with the possibility that the count of the number of 1's or 0's could grow without bound.

We can produce a partial FSM for recognizing binary strings with the same number of 1's and 0's, but a consecutive number of 1's or 0's could occur which would exceed any finite number of states in the FSM.

In our example problem, we want to design an FSM that accepts all binary strings that have the same sequence of 1's and 0's when read either left-to-right or right-to-left. Again, we are limited by the fact that an input string can be arbitrarily long. In this case however, the FSM is impractical even for strings of finite length. For binary strings of maximum length Nmax, our palindromic binary string recognizer uses 2Nmax+1-1 states

FSM for Recognizing Binary Palindromes

Describe the Language Recognized by this FSM