1 Chapter 1 Introduction to the Theory of Computation

1

Chapter 1Chapter 1

Introduction to the Theory of Introduction to the Theory of ComputationComputation

2

Theory of ComputationTheory of Computation

Can we mathematically model the computation?Can we mathematically model the computation? Automata Theory

Regular languages: DFA, NFA, Regular expressions Context-free languages: Context-free grammar,

pushdown automata Turing machines: the most powerful model of

computation Can the computer solve all problems?Can the computer solve all problems?

Computability Theory Halting problem

How efficient the solution is?How efficient the solution is? Complexity Theory

Time and space complexity (Big O notation)

3

1.11.1 Mathematical Preliminaries Mathematical Preliminaries

Sets (1)Sets (1) A A setset is a collection of objects is a collection of objects

Order and repetition is irrelevant X = {1, 2, 3} X = {n | 1 n 3, n N } Y = { } = Y is an empty set

The The power setpower set of X is the set of all subsets of X of X is the set of all subsets of X (X) = 2X = { Y | Y X} |(X)| = 2|X|

Example X = {1, 2, 3}Example X = {1, 2, 3} 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}} {1,2} X is a proper subset of X {1,2,3} X is NOT a proper subset of X {1,2,3} is a superset of {1, 2}

4

Sets (2)Sets (2)

The The symbol signifies membership (belong to): symbol signifies membership (belong to): x X indicates that x is a member or element

of the set X x X indicates that x is not a member or

element of the set X Assume X = {1, 2, 3}, then 1 X, but 4 X

Two sets are equal if they contain the same Two sets are equal if they contain the same members (elements)members (elements) If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} Then X = Y, but X Z, and so Y Z

5

Sets (3)Sets (3)

Assume X = {1,2,3} and Y = {2,4}Assume X = {1,2,3} and Y = {2,4}

UnionUnion X Y = {z | z X or z Y} = {1,2,3,4}

IntersectionIntersection X Y = {z | z X and z Y} = {2}

DifferenceDifference X - Y = {z | z X and z Y} = {1,3}

6

Sets (4)Sets (4)

Complement Complement Let X be a subset of Y The complement of X ( X ) with respect to Y is

the set of elements in Y but not in X X = { z | z Y and z X } = Y - X

Assume X = {1,2,3} is a subset of Y = {1,2,3,4}Assume X = {1,2,3} is a subset of Y = {1,2,3,4} X = {4}

DeMorgan’s LawsDeMorgan’s Laws

(X Y) = X Y

(X Y) = X Y

7

Inductive Proofs

Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts: Basis Step:

Prove for one or several small values of X directly Assumption step :

Assume S(Y) for Y smaller than X Prove S(X) using that assumption

8

Inductive Proofs (EX1)

Prove by induction thatProve by induction that S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2

Basis step:Basis step: n = 1

1 ( 1 + 1) / 2 = 1 Assumption step:Assumption step:

Assume that S is true for all series of length < n Prove that S is true for a series of length n?Prove that S is true for a series of length n?

The series of length n is 1+2+3+…+n-1+n By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 Add n to the summation proves S.

9


Example A complete binary tree with n leaves has 2n - 1

nodes

10


Basis: If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK

Assumption:Assumption: Assume that the statement is true for all trees that have leaves

less than n Proof: Proof: Proof that the statement is true for trees that have n leavesProof that the statement is true for trees that have n leaves

T has two subtrees, U and V U has u leaves and V has v leaves According to the assumption: U has (2u – 1) of nodes and

V has (2v – 1) of nodes T must be a root plus two subtrees U and V. The number of

leaves in T equal number of leaves in U + number of leaves in V = u + v

The number of nodes in T = (2u – 1) + (2v – 1) + 1 = 2 ( u + v) - 1

11

If-And-Only-If Proofs

Often, a statement we need to prove is of the form “X if and only if Y”

We are then required to do two things: Prove the if-part: Assume Y and prove X Prove the only-if-part: Assume X, prove Y

Remember: The if and only-if parts are converses of each

other

One part, say “if X then Y” says nothing about whether Y is true when X is false.

An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.

12

Equivalence of Sets

Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set

To prove sets S and T are the same, prove: x is in S if and only if x is in T. That is:

Assume x is in S; prove x is in T . Assume x is in T ; prove x is in S.

13

FunctionsFunctions

Let Let ff be a function defined on a set be a function defined on a set AA and taking values to and taking values to a set a set BB. Set . Set AA is called the is called the domaindomain of of ff; set ; set BB is called the is called the rangerange of of ff..

The The arityarity of the functionsof the functions The number and type of input arguments the number

and type of output Examples:

The arity of the Multiplication: Num × Num Num The arity of the SquareRoot : Num Num

14

1.2:1.2: Strings and Languages (1)Strings and Languages (1)

AlphabetAlphabet: Finite, nonempty set of symbols: Finite, nonempty set of symbols Examples:

= {0, 1}: binary alphabet = {a, b, c, …, z}: the set of all lower case letters The set of all ASCII characters

StringString: Finite sequence of symbols from an : Finite sequence of symbols from an alphabet alphabet Examples:

01101 where = {0, 1} abracadabra where = {a, b, c, …, z}

15


Empty StringEmpty String: The string with : The string with zerozero occurrences of occurrences of symbols from symbols from and is denoted and is denoted or or

Length of StringLength of String: Number of symbols in the string: Number of symbols in the string The length of a string w is usually written |w| |1010| = 4 || = 0 |uv| = |u| + |v|

ReverseReverse : : wwRR

If w = abc, wR = cba

16


ConcatenationConcatenation: if : if xx and and yy are strings, then are strings, then xyxy is the is the string obtained by placing a copy of string obtained by placing a copy of yy immediately immediately after a copy of after a copy of xx x = a1a2 …ai, y = b1b2 …bj xy = a1a2 …aib1b2 …bj Example: x = 01101, y = 110, xy = 01101110 x = x = x

17


Power of an AlphabetPower of an Alphabet: : kk = the set of strings of length k = the set of strings of length k with symbols from with symbols from Example: = {0, 1} 1 = = {0, 1} 2 = {00, 01, 10, 11} 0 =

QuestionQuestion: How many strings are there in : How many strings are there in 33??

The set of all strings over The set of all strings over is denoted is denoted ** * = 0 1 2 3 …

Also Also + = 1 2 3 … * = + + = * -

18


SubstringSubstring: any string of consecutive characters in : any string of consecutive characters in some string some string ww If w = abc , a, ab, abc are substrings of w

PrefixPrefix and and suffixsuffix:: if w = vu v is a prefix of w u is a suffix of w Example

If w = abc a, ab , abc are prefixes of w c, bc, abc are suffixes of w

19


Suppose: S is the string banana Prefix : ban, banana Suffix : ana, banana Substring : nan, ban, ana, banana

20


LanguageLanguage: set of strings chosen from some alphabet: set of strings chosen from some alphabet A language is a subset of A language is a subset of **

Example of languages:The set of valid Arabic wordsThe set of strings consisting of n 0’s followed by n 1’s

– {, 01, 0011, 000111, …}

The set of strings with equal number of 0’s and 1’s– {, 01, 10, 0011, 0101, 1010, 1001, 1100, …}

Empty languageEmpty language: : = { } = { } The language The language consisting of the empty string consisting of the empty string Note: Note:

21


Can concatenate languagesCan concatenate languages L1L2 = {xy | x L1, y L2} Ln = L concatenated with itself n times L0 = {}; L1 = L

Star-closureStar-closure L* = L0 L1 L2 L+ = L* - L0

Languages can be finite or infiniteLanguages can be finite or infinite L = {a, aba, bba} L = {an | n > 0}

22


OPERATION DEFINITION

union of L and M written L M

concatenation of L and M written LM

Kleene closure of L written L*

positive closure of L written L+

L M = {s | s is in L or s is in M}

LM = {st | s is in L and t is in M}

L+ = Li, i=1,..,

L* denotes “zero or more concatenations of “ L

L* = Li , i=0,..,

L+ denotes “one or more concatenations of “ L

L+ = LL*

23


L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9}L D = the set of letters and digits

LD = all strings consisting of a letter followed by a digit

L2 = the set of all two-letter strings

L4 = L2 L2 = the set of all four-letter strings

L* = { All possible strings of L plus }, L+ = L* -

D+ = set of strings of one or more digits

L (L D ) = set of all strings consisting of a letter followed by a a letter or a digit

L (L D )* = set of all strings consisting of letters and digits beginning with a letter

24


The language The language LL consists of strings over consists of strings over {a,b}{a,b} in in which which eacheach string begins with an string begins with an aa should have an should have an eveneven length length

aa, ab L aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb L

baa L a L

25


The language The language LL consists of strings over consists of strings over {a,b}{a,b} in in which each occurring of which each occurring of bb is is immediately immediately precededpreceded by an by an aa

L a L abaab L

bb L bab L abb L

26


Let X = {a,b,c} and Y = {abb, ba}. ThenLet X = {a,b,c} and Y = {abb, ba}. Then XY = {aabb, babb, cabb, aba, bba, cba} X0 = {} X1 = X = {a,b,c} X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc} X3 = XXX =

{aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}

27


The language L = {a,b}The language L = {a,b}**{bb}{a,b}{bb}{a,b}** = = **{bb}{bb}**

consists of the strings over {a,b} that contain the substring bb

bb L abb L bbb L aabb L bbaaa L bbabba L abab L bab L b L

28


Let Let LL be the language that consists of all strings that be the language that consists of all strings that beginbegin with with aaaa or or endend with with bbbb L1 = {aa}{a,b}*

L2 = {a,b}*{bb} L = L1 L2 = {aa}{a,b}* {a,b}*{bb} bb L abb L bbb L aabb L bbaaa L bbabba L abab L bab L ba L

29


Let Let LL11 = {bb} and = {bb} and LL22 = { = {, bb, bbbb} over , bb, bbbb} over bb

The languages LThe languages L11**

and L and L22**

both contain precisely both contain precisely the strings consisting of an the strings consisting of an eveneven number of number of bb’s.’s.

, with length zero, is an element of L, with length zero, is an element of L11**

and L and L22**

30


What is the language of all What is the language of all even-lengtheven-length strings over strings over {a,b}{a,b} L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*

What is the language of all What is the language of all odd-lengthodd-length strings over strings over {a,b}{a,b} L = {a,b}* - {aa, bb, ab, ba}* or L = {a,b}{aa, bb, ab, ba}* or L = {aa, bb, ab, ba}* {a,b}

Documents

1 Chapter 1 Introduction to the Theory of Computation