Sr1 Toc Intro

8/12/2019 Sr1 Toc Intro

1/56

CSE 211: Introduction to the Theory of Computation

Md. Shaifur Rahman

Dept. of CSEBangladesh University of Engineering & Technology

Week 1 Class 1

Shaifur (BUET) CSE211: TOC-Intro Week 1 Class 1 1 / 29


2/56

Outline

1 Necessity of CS Theory Courses



3/56

Outline


2 What is Theory of Computation?



4/56

Outline


2 What is Theory of Computation?

3 Course Outline



5/56

Why Theory? (again!)

We dont like theory, because:One has to memorize a lot of theorems, lemmas, proofs

We like non-theory courses,



6/56



It is inherently boring!




7/56




Where is the output?




8/56





How is it related to our career-path?We like non-theory courses,



9/56






We like programming in C, C++, JAVA, Python, Android etc.



10/56







The outcome is visible, projects done in these courses brighten our

resume



11/56


12/56







The outcome is visible, projects done in these courses brighten our

resumeThe skills are necessary for the career-path of a developer

Achieving grades are comparatively easy :)



13/56

Theory vs. Non-theory

Difficult to separate theories from all courses!

Course Theory-bias

CSE 103 (Discrete Math) 90%

CSE 105 (Structured Programming Language) 30%

CSE 201 (Object Oriented Programming Language) 20%

CSE 203 (Data Structure) 80%CSE 205 (Digital Logic Design) 60%

In this semester:

Course Theory-bias

CSE 207 (Algorithms) 95%CSE 209 (Digital Electronics and Pulse Technique) 40%

CSE 211 (Theory of Computation) 100%

CSE207 is thefraternal brotherof CSE211



14/56

The Power of Theory

Write a C program that finds all the factors of a 512-bit integer

For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3



15/56

The Power of Theory


For input 24, Output is: 1, 2, 3, 4, 6, 8, 12 and 24 , since 24 = 1 23 3But wait! 512-bit number!So the input ranges from 1 to 2512 1thats roughly 1.3 10154

The highest input can be of155-digits!

Input: 12345676449393993. . .

930049922(155 digits)



16/56

The Power of Theory




Input: 12345676449393993. . .

930049922(155 digits)

3 hoursgone, your C-program is not coming up with any answer?Your program worked perfectly for 1, 2, 10, 20 even 50 digit inputs!


Th P f Th


17/56

The Power of Theory




Input: 12345676449393993. . .

930049922(155 digits)


So whats wrong??Is your program erroneous?


Th P f Th


18/56

The Power of Theory




Input: 12345676449393993. . .

930049922(155 digits)



Is the computation power of the machine insufficient?


Th P f Th


19/56

The Power of Theory




Input: 12345676449393993. . .

930049922(155 digits)



Is the computation power of the machine insufficient?

Is there a probem in the nature of the input?


Th P f Th (C td )


20/56

The Power of Theory (Contd.)

Case: Your program is erroneousHow do you know? The program is not even stopping!It may indeed stop after 10 hours and show the correct output!


Th P f Th (C td )


21/56



Case: Computation Power of the Machine is insufficient

We may use multi-core processors clocked at a very high speedlike: 10-core processor each core clocked at 3GHz


The Power of Theory (Contd )


22/56



Case: Computation Power of the Machine is insufficient

We may use multi-core processors clocked at a very high speedlike: 10-core processor each core clocked at 3GHzCan we afford that machine?What if the number is even bigger? Can the machine scale up to the

input?Caveat: Even with todays fastest computer, the computation for a500-digit input will take 5 billion years!




23/56


Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!




24/56


Case: Nature of InputSome numbers are too big or too smallAre you crazy? 500-digit number? Thats pretty big!So is:

the number of stars in the universe

So, data can be very large or too small? You never can tell




25/56



the number of stars in the universethe number of electrons in your body





26/56




the energy in electron volt when two stars collide





27/56




the energy in electron volt when two stars collide

the number of combinatorial choices of pairings of almost half-million

genes in our cellSo, data can be very large or too small? You never can tell




28/56


Case: Nature of Computation

Can we break down the processing into multiple blocks and distribute

those among multiple computers?




29/56



Can we break down the processing into multiple blocks and distributethose among multiple computers?

Can we predict the time required by analyzing the nature of input(input-domain) and the method of computation (algorithm)?




30/56



Can we break down the processing into multiple blocks and distributethose among multiple computers?

Can we predict the time required by analyzing the nature of input(input-domain) and the method of computation (algorithm)?

Can we identify certain problems asreally hardproblems?


Really Hard Problems


31/56

Really Hard Problems

General Tile Fitting Problem

Given a finite set of tiles and an area on a plane, is it possible to cover thecomplete area using each type of tiles at least once?


Really Hard Problems (Contd.)


32/56

y ( )

Sometimes, there is a solutionSometimes, there is no solutionSometimes, we cannot tell for sure- if there is a solution! (undecidable)




33/56

y ( )

There are many problems that cannot be guaranteed to be solved by anycomputer! These are calledundecidable problems

David Hilbert

Entscheidungsproblem

Give an algorithmAthat takes as inputa hypothesis Hand answersTrueorFalsebased on a finite set of axioms SA.In other word, AlgorithmAdecideswhether H logically follows from SA.

Hypothesis: A machine works withoutconsuming any energy.Decision: False (according to the set ofaxioms of Physics)




34/56

y ( )

Alonzo Church & Alan Turing

Alonzo Church and Alan Turing independently proved respectively in 1936

and in 1937 that

Hilberts Entscheidungsproblem isundecidable

This is known asChurch-Turing Thesis.




35/56

y ( )

More undecided problems:

Mortal Matrix ProblemGiven a finite set ofn n matrices with integer entries, determinewhether they can be multiplied in some order, possibly with repetition, toyield the zero matrix.

Conjugacy problem

Given two words xand y of a Group G, whether there exists a z such that:y=zxz1

*A group is an algebraic structure consisting of a set with one or moreoperators such that the result of applying the operators on one or moreelements of the is also an element in the set. For example, the set ofinteger I is a group.




36/56

( )

Some problems are hard but still solvable i.e. NOT undecidable

(Euclidian) Travelling Salesman Problem

Given a set of vertices and their coordinates, find a tour or cycle whichvisits each vertex exactly once and is of the minimum length.




37/56

Some problems are hard but still solvable i.e. NOT undecidable

Chromatic Number or Graph Coloring ProblemGiven a graph G= (V,E), find the minimum number of different colorsneeded for this graph such that two adjacent vertices are not given thesame color.

Total three different colors are needed



38/56



39/56

There is relationships among the hard solvable problems, we can comparetheir hardness!

Circuit satisfiability problem(Circuit-SAT) can bereducedtoTravelling Salesman

Problem (TSP)If there is agoodsolution toCircuit-SAT, there is also agoodsolution to TSP




40/56

Moral of the story

If the given problem is proved to be undecidable or unsolvable, dontinvest your valuable effort to write program to solve it.




41/56

Moral of the story


If the given problem is solvable but still hard, it is not your programsfault that the computation is taking too much time.




42/56

Moral of the story


If the given problem is solvable but still hard, it is not your programsfault that the computation is taking too much time.

If the given problem is solvable andnot hard, and your program failsto solve it in the reasonable amount of time, only theninvest effortfor improvement of your program!


Theory of Computation


43/56

Theory of computation

The branch that deals with whether and how efficiently problems can besolved on amodel of computation, using an algorithm




44/56



The field is divided into three major branches:

Automata Theory

Computability Theory and

Computational Complexity Theory




45/56



The field is divided into three major branches:

Automata Theory

Computability Theory and


There are many models of computation:

Turing Machine

Combinatory logic

Markov Algorithm etc.



46/56

Branches of Theory of Computation


47/56

Automata Theory

the study ofabstract machines( in our case, Turing Machine) andthe problems that can be solved using these machines

If the abstract machine cannot solve the problem, no computer on

earth, no matter how powerful it is, can ever solve the problem

Topics include: Finite Automaton (FA), Push-down Automaton(PDA), Turing Machine (TM) etc.

studied in the theory of computation, compiler design, verification

systems, robotics etc.


Branches of Theory of Computation (Contd.)


48/56

Computability Theory

deals with the question of the extent to which a problem is solvableon a computer

Topics include: Turing Degree (level of unsolvabiltiy), Reducibility(the ability to transform one hard problem to another) etc.

studied in mathematical logic theories like recursion theory, artificialintelligence, machine learning etc.


Branches of Theory of Computation (Contd.)


49/56


deals with classifying computational problems according to theirinherent difficulty

Two major aspects are considered: time complexity (how many stepsthe algorithm takes) and space complexity (how much memory isrequired by the program)

studies complexity classes: P, NP, NPCand relationships amongthese classes


Course Outline


50/56

Course Name: CSE 211 (Theory of Computation)Credit Hour: 2.0Number of Class Tests: 3

Course Teachers:Md. Shaifur Rahman(first seven weeks before mid-term)and

Sukarna Barua(after mid-term)


Class Outline


51/56

Week Day Topics

1

1 Introduction to TOC

2 Deterministic Finite Automaton (DFA)

2 1 DFA (Contd.)

2 Operators of Finite Automaton

3 1 Non-deterministic Finite Automaton (NFA)

2 Equivalence of DFA and NFA

4 1 Regular Expressions

2 Class Test-1

5 1 Regular Expressions (Contd.)

2 Pumping Lemma

6 1 Context-free Grammar (CFG)2 CFG (Contd.)

7 1 Overall Discussion

2 Class Test-2



52/56

Textbooks (Contd.)


53/56

Supporting textbooks

Introduction to Automata Theory,Languages & Computation

(3rd Edition)by

Hopcroft et. al.


Textbooks (Contd.)


54/56

Supporting textbooks

Elements of the Theory ofComputation(2nd Edition)

by

Lewis et. al.


Course Website


55/56

http://sites.google.com/site/cse211february2013

http://sites.google.com/site/cse211february2013http://sites.google.com/site/cse211february2013


56/56

Question?


Documents

Sr1 Toc Intro