Module 1: Course Overview

1

Module 1: Course Overview

• Course: CSE 460

• Instructor: Dr. Eric Torng

• Grader/TA: To be determined

2

What is this course?

• Philosophy of computing course– We take a step back to think about computing

in broader terms

• Science of computing course– We study fundamental ideas/results that shape

the field of computer science

• “Applied” computing course– We learn study a broad range of material with

relevance to computing today

3

Philosophy

• Phil. of life– What is the purpose of

life?

– What are we capable of accomplishing in life?

– Are there limits to what we can do in life?

– Why do we drive on parkways and park on driveways?

• Phil. of computing– What is the purpose of

programming?

– What can we achieve through programming?

– Are there limits to what we can do with programs?

– Why don’t debuggers actually debug programs?

4

Science

• Physics– Study of fundamental

physical laws and phenomenon like gravity and electricity

• Engineering– Governed by physical

laws

• Our material– Study of fundamental

computational laws and phenomenon like undecidability and universal computers

• Programming– Governed by

computational laws

5

Applied computing

• Applications are not immediately obvious– In some cases, seeing the applicability of this

material requires advanced abstraction skills• Every year, there are people who leave this course

unable to see the applicability of the material

– Others require more material in order to completely understand their application

• for example, to understand how regular expressions and context-free grammars are applied to the design of compilers, you need to take a compilers course

6

Some applications

– Important programming languages• regular expressions (perl)

• finite state automata (used in hardware design)

• context-free grammars

– Proofs of program correctness– Subroutines

• Using them to prove problems are unsolvable

– String searching/Pattern matching– Algorithm design concepts such as recursion

7

Fundamental Theme *

• What are the capabilities and limitations of computers and computer programs?– What can we do with computers/programs?– Are there things we cannot do with

computers/programs?

8

Module 2: Fundamental Concepts

• Problems

• Programs– Programming languages

9

Problems

We view solving problems as the main application for computer

programs

10Inputs Outputs

(4,2,3,1)(3,1,2,4)

(7,5,1)

(1,2,3)

(1,2,3,4)

(1,5,7)

(1,2,3)

Definition • A problem is a mapping or function between

a set of inputs and a set of outputs

• Example Problem: Sorting

11

How to specify a problem

• Input– Describe what an input instance looks like

• Output– Describe what task should be performed on the

input– In particular, describe what output should be

produced

12

Example Problem Specifications*

– Sorting problem• Input

– Integers n1, n2, ..., nk

• Output– n1, n2, ..., nk in nondecreasing order

– Find element problem• Input

– Integers n1, n2, …, nk

– Search key S

• Output– yes if S is in n1, n2, …, nk, no otherwise

13

Programs

Programs solve problems

14

Purpose

• Why do we write programs?

• One answer– To solve problems– What does it mean to solve a problem?

• Informal answer: For every legal input, a correct output is produced.

• Formal answer: To be given later

15

Programming Language

• Definition– A programming language defines what

constitutes a legal program– Example: a pseudocode program may not be a

legal C++ program which may not be a legal C program

– A programming language is typically referred to as a “computational model” in a course like this.

16

C++

• Our programming language will be C++ with minor modifications– Main procedure will use input parameters in a

fashion similar to other procedures• no argc/argv

– Output will be returned• type specified by main function type

17

Maximum Element Problem

• Input– integer n ≥ 1

– List of n integers

• Output– The largest of the n integers

18

C++ Program which solves the Maximum Element Problem*

int main(int A[], int n) {int i, max;

if (n < 1)

return (“Illegal Input”);

max = A[0];

for (i = 1; i < n; i++)if (A[i] > max)

max = A[i];

return (max);

}

19

Fundamental Theme

Exploring capabilities and limitations of C++ programs

20

Restating the Fundamental Theme *

• We will study the capabilities and limits of C++ programs

• Specifically, we will try and identify– What problems can be solved by C++ programs– What problems cannot be solved by C++

programs

21

Question

• Is C++ general enough?

• Or is it possible that there exists some problem such that– can be solved by some program P in some

other reasonable programming language – but cannot be solved by any C++ program?

22

Church’s Thesis (modified)

• We have no proof of an answer, but it is commonly accepted that the answer is no.

• Church’s Thesis (three identical statements)– C++ is a general model of computation– Any algorithm can be expressed as a C++

program– If some algorithm cannot be expressed by a C+

+ program, it cannot be expressed in any reasonable programming language

23

Summary *• Problems

– When we talk about what programs can or cannot “DO”, we mean what PROBLEMS can or cannot be solved

24

Module 3: Classifying Problems

• One of the main themes of this course will be to classify problems in various ways– By solvability

• Solvable, “half-solvable”, unsolvable

• We will focus our study on decision problems– function (one correct answer for every input)

– finite range (yes or no is the correct output)

25

Classification Process

• Take some set of problems and partition it into two or more subsets of problems where membership in a subset is based on some shared problem characteristic

Set of Problems

Subset 1 Subset 2 Subset 3

26

Classify by Solvability

• Criteria used is whether or not the problem is solvable– that is, does there exist a C++ program which

solves the problem?Set of All Problems

Solvable Problems Unsolvable Problems

27

Function Problems

• We will focus on problems where the mapping from input to output is a function

Set of All ProblemsNon-Function Problems Function Problems

28

General (Relation) Problem

• the mapping is a relation– that is, more than one output is possible for a

given input

Inputs Outputs

29

Criteria for Function Problems

• mapping is a function– unique output for each input

Inputs Outputs

30

Example Non-Function Problem

• Divisor Problem– Input: Positive integer n– Output: A positive integral divisor of n

Inputs Outputs

9 13

9

31

Example Function Problems

• Sorting

• Multiplication Problem– Input: 2 integers x and y– Output: xy

Inputs Outputs

2,5 10

32

Another Example *

• Maximum divisor problem– Input: Positive integer n– Output: size of maximum divisor of n smaller

than n

Inputs Outputs

9 13

9

33

Decision Problems

• We will focus on function problems where the correct answer is always yes or no

Set of Function ProblemsNon-Decision Problems Decision Problems

34

Criteria for Decision Problems• Output is yes or no

– range = {Yes, No}

• Note, problem must be a function problem– only one of Yes/No is correct

Inputs Outputs

Yes

No

35

Example

• Decision sorting– Input: list of integers– Yes/No question: Is the list in nondecreasing

order?

Inputs Outputs

Yes

No

(1,3,2,4)

(1,2,3,4)

36

Another Example

• Decision multiplication– Input: Three integers x, y, z– Yes/No question: Is xy = z?

Inputs Outputs

Yes

No

(3,5,14)

(3,5,15)

37

A Third Example *

• Decision Divisor Problem– Input: Two integers x and y– Yes/No question: Is y a divisor of x?

Inputs Outputs

Yes

No

(14,5)

(14,7)

38

Focus on Decision ProblemsSet of All Problems


DecisionProblems

OtherProbs

• When studying solvability, we are going to focus specifically on decision problems– There is no loss of generality, but we will not explore

that here

39

Finite Domain Problems

• These problems have only a finite number of inputs

Set of All ProblemsFinite Domain Problems Infinite Domain Problems

40

Lack of GeneralitySet of All Problems


• All finite domain problems can be solved using “table lookup” idea

FiniteDomain

InfiniteDomain

Empty

41

Table Lookup Program

int main(string x) {

switch x {

case “Bill”: return(3);

case “Judy”: return(25);

case “Tom”: return(30);

default: cerr << “Illegal input\n”;

}

42

Key Concepts

• Classification Theme

• Decision Problems– Important subset of problems– We can focus our attention on decision

problems without loss of generality– Same is not true for finite domain problems

• Table lookup

43

Module 4: Formal Definition of Solvability

• Analysis of decision problems– Two types of inputs:yes inputs and no inputs

– Language recognition problem

• Analysis of programs which solve decision problems– Four types of inputs: yes, no, crash, loop inputs

– Solving and not solving decision problems

• Classifying Decision Problems– Formal definition of solvable and unsolvable decision

problems

44

Analyzing Decision Problems

Can be defined by two sets

45

Decision Problems and Sets• Decision problems consist of 3 sets

– The set of legal input instances (or universe of input instances)

– The set of “yes” input instances – The set of “no” input instances

Yes Inputs No InputsSet of All Legal Inputs

46

Redundancy *

• Only two of these sets are needed; the third is redundant– Given

• The set of legal input instances (or universe of input instances)

– This is given by the description of a typical input instance

• The set of “yes” input instances – This is given by the yes/no question

– We can compute• The set of “no” input instances

47

Typical Input Universes

• *: The set of all finite length strings over finite alphabet – Examples

• {a}*: {/\, a, aa, aaa, aaaa, aaaaa, … }• {a,b}*: {/\, a, b, aa, ab, ba, bb, aaa, aab, aba, abb, … }• {0,1}*: {/\, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, … }

• The set of all integers• If the input universe is understood, a decision

problem can be specified by just giving the set of yes input instances

48

Language Recognition Problem

• Input Universe– * for some finite alphabet

• Yes input instances– Some set L subset of *

• No input instances– * - L

• When is understood, a language recognition problem can be specified by just stating what L is.

49

Language Recognition Problem *

• Traditional Formulation– Input

• A string x over some finite alphabet

– Task• Is x in some language L

subset of ?

• 3 set formulation• Input Universe

– * for a finite alphabet • Yes input instances

– Some set L subset of *

• No input instances– * - L

• When is understood, a language recognition problem can be specified by just stating what L is.

50

Equivalence of Decision Problems and Languages

• All decision problems can be formulated as language recognition problems– Simply develop an encoding scheme for

representing all inputs of the decision problem as strings over some fixed alphabet

– The corresponding language is just the set of strings encoding yes input instances

• In what follows, we will often use decision problems and languages interchangeably

51

Visualization *

Yes Inputs

Original DecisionProblem

No Inputs

EncodingScheme overalphabet

Language L

* - L

CorrespondingLanguage RecognitionProblem

52

Analyzing Programs which Solve Decision Problems

Four possible outcomes

53

Program Declaration

• Suppose a program P is designed to solve some decision problem . What does P’s declaration look like?

• What should P return on a yes input instance?

• What should P return on a no input instance?

54

Program Declaration II

• Suppose a program P is designed to solve a language recognition problem . What does P’s declaration look like?– bool main(string x) {

• We will assume that the string declaration is correctly defined for the input alphabet

– If = {a,b}, then string will define variables consisting of only a’s and b’s

– If = {a, b, …, z, A, …, Z}, then string will define variables consisting of any string of alphabet characters

55

Programs and Inputs

• Notation– P denotes a program

– x denotes an input for program P

• 4 possible outcomes of running P on x– P halts and says yes: P accepts input x

– P halts and says no: P rejects input x

– P halts without saying yes or no: P crashes on input x• We typically ignore this case as it can be combined with rejects

– P never halts: P infinite loops on input x

56

Programs and the Set of Legal Inputs

• Based on the 4 possible outcomes of running P on x, P partitions the set of legal inputs into 4 groups– Y(P): The set of inputs P accepts

• When the problem is a language recognition problem, Y(P) is often represented as L(P)

– N(P): The set of inputs P rejects

– C(P): The set of inputs P crashes on

– I(P): The set of inputs P infinite loops on• Because L(P) is often used in place of Y(P) as described above, we

use notation I(P) to represent this set

57

Illustration

All Inputs

Y(P) N(P) C(P) I(P)

58

Analyzing Programs and Decision Problems

Distinguish the two carefully

59

Program solving a decision problem

• Formal Definition:– A program P solves decision problem if and only if

• The set of legal inputs for P is identical to the set of input instances of

• Y(P) is the same as the set of yes input instances for • N(P) is the same as the set of no input instances for

– Otherwise, program P does not solve problem • Note C(P) and I(P) must be empty in order for P to solve

problem

Y(P) N(P) C(P) I(P)

60

Solvable Problem

• A decision problem is solvable if and only if there exists some C++ program P which solves – When the decision problem is a language

recognition problem for language L, we often say that L is solvable or L is decidable

• A decision problem is unsolvable if and only if all C++ programs P do not solve – Similar comment as above

61

Illustration of Solvability

Inputs of Program P

Y(P) N(P) C(P) I(P)C(P) I(P)

Inputs of Problem

Yes Inputs No Inputs

62

Program half-solving a problem

• Formal Definition:– A program P half-solves problem if and only if

• The set of legal inputs for P is identical to the set of input instances of

• Y(P) is the same as the set of yes input instances for • N(P) union C(P) union I(P) is the same as the set of no

input instances for

– Otherwise, program P does not half-solve problem • Note C(P) and I(P) need not be empty

Y(P) N(P) C(P) I(P)

63

Half-solvable Problem

• A decision problem is half-solvable if and only if there exists some C++ program P which half-solves – When the decision problem is a language

recognition problem for language L, we often say that L is half-solvable

• A decision problem is not half-solvable if and only if all C++ programs P do not half-solve

64

Illustration of Half-Solvability *

Inputs of Program P

Y(P) N(P) C(P) I(P)

Inputs of Problem

Yes Inputs No Inputs

65

Hierarchy of Decision Problems

Solvable

The set of half-solvable decision problems is a proper subset of the set of all decision problems

The set of solvable decision problems is a proper subset of the set of half-solvable decision problems.

Half-solvable

All decision problems

66

Why study half-solvable problems?

• A correct program must halt on all inputs

• Why then do we define and study half-solvable problems?

• One Answer: the set of half-solvable problems is the natural class of problems associated with general computational models like C++ – Every program half-solves some decision problem

– Some programs do not solve any decision problem

• In particular, programs which do not halt do not solve their corresponding decision problems

67

Key Concepts

• Four possible outcomes of running a program on an input

• The four subsets every program divides its set of legal inputs into

• Formal definition of– a program solving (half-solving) a decision problem– a problem being solvable (half-solvable)

• Be precise: with the above two statements!

68

Module 5

• Topics– Proof of the existence of unsolvable problems

• Proof Technique– There are more problems/languages than there are

programs/algorithms

– Countable and uncountable infinities

69

Overview

• We will show that there are more problems than programs– Actually more problems than programs in any

computational model (programming language)

• Implication– Some problems are not solvable

70

Preliminaries

Define set of problems

Observation about programs

71

Define set of problems

• We will restrict the set of problems to be the set of language recognition problems over the alphabet {a}.

• That is– Universe: {a}*– Yes Inputs: Some language L subset of {a}*– No Inputs: {a}* - L

72

Set of Problems *• The number of distinct problems is given by the

number of languages L subset of {a}*– 2{a}* is our shorthand for this set of subset languages

• Examples of languages L subset of {a}*– 0 elements: { }

– 1 element: {/\}, {a}, {aa}, {aaa}, {aaaa}, …

– 2 elements: {/\, a}, {/\, aa}, {a, aa}, …

– Infinite # of elements: {an | n is even}, {an | n is prime}, {an | n is a perfect square}

73

Infinity and {a}*

• All strings in {a}* have finite length

• The number of strings in {a}* is infinite

• The number of languages L in 2{a}* is infinite

• The number of strings in a language L in 2{a}* may be finite or infinite

74

Define set of programs

• The set of programs we will consider are the set of legal C++ programs as defined in earlier lectures

• Key Observation– Each C++ program can be thought of as a finite

length string over alphabet P

• P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

75

Example *

int main(int A[], int n){ {26 characters including newline}

int i, max; {13 characters including initial tab}

{1 character: newline}

if (n < 1) {12 characters}

return (“Illegal Input”); {28 characters including 2 tabs}

max = A[0]; {13 characters}

for (i = 1; i < n; i++) {25 characters}

if (A[i] > max) {18 characters}max = A[i]; {15 characters}

return (max); {15 characters}

} {2 characters including newline}

76

Number of programs

• The set of legal C++ programs is clearly infinite

• It is also no more than |P*|

– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

77

Goal

• Show that the number of languages L in 2{a}* is greater than the number of strings in P*

– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

• Problem– Both are infinite

78

How do we compare the relative sizes of infinite sets?

Bijection (yes)

Proper subset (no)

79

Bijections

• Two sets have EQUAL size if there exists a bijection between them– bijection is a 1-1 and onto function between

two sets

• Examples– Set {1, 2, 3} and Set {A, B, C}– Positive even numbers and positive integers

80

Bijection Example

• Positive Integers Positive Even Integers 1 2

2 4

3 6

... ...

i 2i

… ...

81

Proper subset

• Finite sets– S1 proper subset of S2 implies S2 is strictly

bigger than S1• Example

– women proper subset of people

– number of women less than number of people

• Infinite sets– Counterexample

• even numbers and integers

82

Two sizes of infinity

Countable

Uncountable

83

Countably infinite set S *

• Definition 1– S is equal in size (bijection) to N

• N is the set of natural numbers {1, 2, 3, …}

• Definition 2 (Key property)– There exists a way to list all the elements of set

S (enumerate S) such that the following is true• Every element appears at a finite position in the

infinite list

84

Uncountable infinity *

• Any set which is not countably infinite

• Examples– Set of real numbers– 2{a}*, the set of all languages L which are a

subset of {a}*

• Further gradations within this set, but we ignore them

85

Proof

86

(1) The set of all legal C++ programs is countably infinite

• Every C++ program is a finite string

• Thus, the set of all legal C++ programs is a language LC

• This language LC is a subset of P*

87

For any alphabet , * is countably infinite

• Enumeration ordering– All length 0 strings

• ||0 = 1 string:

– All length 1 strings• || strings

– All length 2 strings• ||2 strings

– …

• Thus, P* is countably infinite

88

Example with alphabet {a,b} *

• Length 0 strings– 0 and

• Length 1 strings– 1 and a, 2 and b

• Length 2 strings– 3 and aa, 4 and ab, 5 and ba, 6 and bb, ...

• Question– write a program that takes a number as input and

computes the corresponding string as output

89

(2) The set of languages in 2{a}* is uncountably infinite

• Diagonalization proof technique – “Algorithmic” proof– Typically presented as a proof by contradiction

90

Algorithm Overview *

• To prove this set is uncountably infinite, we construct an algorithm D that behaves as follows:– Input

• A countably infinite list of languages L[] subset of {a}*

– Output• A language D(L[]) which is a subset of {a}* that is not

on list L[]

91

Visualizing D

List L[]

L[0]

L[1]

L[2]

L[3]

...

LanguageD(L[]) not inlist L[]

Algorithm D

92

Why existence of D implies result

• If the number of languages in 2{a}* is countably infinite, there exists a list L[] s.t.– L[] is complete

• it contains every language in 2{a}*

– L[] is countably infinite

• The existence of algorithm D implies that no list of languages in 2{a}* is both complete and countably infinite– Specifically, the existence of D shows that any

countably infinite list of languages is not complete

93

Visualizing One Possible L[ ] *

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

•#Rows is countably infinite

•Given

•#Cols is countably infinite

• {a}* is countably infinite

• Consider each string to be a feature– A set contains or does not contain each string

94

Constructing D(L[ ]) *• We construct D(L[]) by using a unique feature

(string) to differentiate D(L[]) from L[i]– Typically use ith string for language L[i]– Thus the name diagonalization

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

OUT

IN

IN

IN

OUT

D(L[])

95

D(L[]) cannot be any language L[i]• D(L[]) cannot be language L[0] because D(L[])

differs from L[0] with respect to string a0 = /\.– If L[0] contains /\, then D(L[]) does not, and vice versa.

• D(L[]) cannot be language L[1] because D(L[]) differs from L[1] with respect to string a1 = a.– If L[1] contains a, then D(L[]) does not, and vice versa.

• D(L[]) cannot be language L[2] because D(L[]) differs from L[2] with respect to string a2 = aa.– If L[2] contains aa, then D(L[]) does not, and vice versa.

• …

96

Questions

• Do we need to use the diagonal?– Every other column and every row?– Every other row and every column?

• What properties are needed to construct D(L[])?

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

97

Visualization

Solvable Problems

The set of solvable problems is a proper subset of the set of all problems.

All problems

98

Summary

• Equal size infinite sets: bijections– Countable and uncountable infinities

• More languages than algorithms– Number of algorithms countably infinite– Number of languages uncountably infinite– Diagonalization technique

• Construct D(L[]) using infinite set of features

• The set of solvable problems is a proper subset of the set of all problems

99

Module 6• Topics

– Program behavior problems• Input of problem is a program/algorithm

• Definition of type program

– Program correctness• Testing versus Proving

100

Number Theory Problems

• These are problems where we investigate properties of numbers– Primality

• Input: Positive integer n

• Yes/No Question: Is n a prime number?

– Divisor• Input: Integers m,n

• Yes/No question: Is m a divisor of n?

101

Graph Theory Problems

• These are problems where we investigate properties of graphs– Connected

• Input: Graph G

• Yes/No Question: Is G a connected graph?

– Subgraph• Input: Graphs G1 and G2

• Yes/No question: Is G1 a subgraph of G2?

102

Program Behavior Problems

• These are problems where we investigate properties of programs and how they behave

• Give an example problem with one input program P

• Give an example problem with two input programs P1 and P2

103

Program Representation• Program variables

– Abstractly, we define the type “program”• graph G, program P

– More concretely, we define type program to be a string over the program alphabet P = {a, …, z, A, …, Z, 0, …, 9, punctuation, white space}

• Note, many strings over P are not legal programs

• We consider them to be programs that always crash

• Possible declaration of main procedure– bool main(program P)

104

Program correctness

• How do we determine whether or not a program P we have written is correct?

• What are some weaknesses of this approach?

• What might be a better approach?

105

Testing versus Analyzing

ProgramP

Test Inputsx1

x2

x3

...

OutputsP(x1)P(x2)P(x3)

...

AnalyzerProgram P Analysis ofProgram P

106

2 Program Behavior Problems *• Correctness

– Input• Program P

– Yes/No Question• Does P correctly solve the primality problem?

• Functional Equivalence– Input

• Programs P1, P2

– Yes/No Question• Is program P1 functionally equivalent to program P2

107

Module 7

• Halting Problem– Fundamental program behavior problem– A specific unsolvable problem– Diagonalization technique revisited

• Proof more complex

108

Definition• Input

– Program P • Assume the input to program P is a single nonnegative integer

– This assumption is not necessary, but it simplifies the following unsolvability proof

– To see the full generality of the halting problem, remove this assumption

– Nonnegative integer x, an input for program P

• Yes/No Question– Does P halt when run on x?

• Notation– Use H as shorthand for halting problem when space is a

constraint

109

Example Input *• Program with one input of type unsigned

bool main(unsigned Q) {

int i=2;

if ((Q = = 0) || (Q= = 1)) return false;

while (i<Q) {

if (Q%i = = 0) return (false);

i++;

}

return (true);

}

• Input x4

110

Three key definitions

111

Definition of list L *

• P* is countably infinite where P = {characters, digits, white

space, punctuation}• Type program will be type string with P as the alphabet• Define L to be the strings in P

* listed in enumeration order– length 0 strings first– length 1 strings next– …

• Every program is a string in P – For simplicity, consider only programs that have

• one input• the type of this input is an unsigned

• Consider strings in P* that are not legal programs to be

programs that always crash (and thus halt on all inputs)

112

Definition of PH *

• If H is solvable, some program must solve H

• Let PH be a procedure which solves H

– We declare it as a procedure because we will use PH as a subroutine

• Declaration of PH

– bool PH(program P, unsigned x)• In general, the type of x should be the type of the input to P

• Comments– We do not know how PH works

– However, if H is solvable, we can build programs which call PH as a subroutine

113

bool main(unsigned y) /* main for program D */{ program P = generate(y);

if (PH(P,y)) while (1>0); else return (yes); }

/* generate the yth string in P* in enumeration order */

program generate(unsigned y) /* code for program of slide 21 from module 5 did this for

{a,b}* */

bool PH(program P, unsigned x)/* how PH solves H is unknown */

Definition of program D

114

Generating Py from y *• We won’t go into this in detail here

– This was the basis of the question at the bottom of slide 21 of lecture 5 (alphabet for that problem was {a,b} instead of P).

– This is the main place where our assumption about the input type for program P is important

• for other input types, how to do this would vary

• Specification– 0 maps to program – 1 maps to program a– 2 maps to program b– 3 maps to program c– …– 26 maps to program z– 27 maps to program A– …

115

Proof that H is not solvable

116

Argument Overview *

H is solvable

D is NOT on list L

PH exists

Definition ofSolvability

D exists

D’s code

D is on list L

L is list ofall programs

D does NOT exist

PH does NOT exist

H is NOT solvable

p → q is logically equivalent to (not q) → (not p)

117

Proving D is not on list L

• Use list L to specify a program behavior B that is distinct from all real program behaviors (for programs with one input of type unsigned)– Diagonalization argument similar to the one for proving the

number of languages over {a} is uncountably infinite

– No program P exists that exhibits program behavior B

• Argue that D exhibits program behavior B– Thus D cannot exist and thus is not on list L

118

Non-existent program behavior B

119

Visualizing List L *

P0

P1

P2

P3

P4...

1 2 3 4 ...

H HHHH

NHH H HNH

NH NH NH NH NH

H H H H H

HHNH NHNH

•#Rows is countably infinite

• p* is countably infinite

•#Cols is countably infinite

• Set of nnnegative integers is countably infinite

• Consider each number to be a feature– A program halts or doesn’t halt on each integer– We have a fixed L this time

120

Diagonalization to specify B *

P0

P1

P2

P3

P4...

1 2 3 4 ...

H HHHH

NHH H HNH

NH NH NH NH NH

H H H H H

HHNH NHNH

•We specify a non-existent program behavior B by using a unique feature (number) to differentiate B from Pi

NH

H

H

H

NH

B

121

Arguing D exhibits program behavior B

122

bool main(unsigned y) /* main for program D */{ program P = generate(y);

if (PH(P,y)) while (1>0); else return (yes); }

/* generate the yth string in P* in enumeration order */

program generate(unsigned y) /* code for extra credit program of slide 21 from lecture 5 did

this for {a,b}* */

bool PH(program P, unsigned x)/* how PH solves H is unknown */

Code for D

123

Visualization of D in action on input y

• Program D with input y– (type for y: unsigned)

Given input y, generate the program (string) Py

Run PH on Py and y

• Guaranteed to halt since PH solves H

IF (PH(Py,y)) while (1>0); else return (yes);

P0

P1

P2

Py...

1 2 ...

HH

H H

NH NH

H

NH

NH

D ... y

...

HNH

124

D cannot be any program on list L• D cannot be program P0 because D behaves

differently on 0 than P0 does on 0.– If P0 halts on 0, then D does not, and vice versa.

• D cannot be program P1 because D behaves differently on 1 than P1 does on 1.– If P1 halts on 1, then D does not, and vice versa.

• D cannot be program P2 because D behaves differently on 2 than P2 does on 2.– If P2 halts on 2, then D does not, and vice versa.

• …

125

Alternate Proof

126

Alternate Proof Overview

• For every program Py, there is a number y that we associate with it

• The number we use to distinguish program Py from D is this number y

• Using this idea, we can arrive at a contradiction without explicitly using the table L– The diagonalization is hidden

127

H is not solvable, proof II

• Assume H is solvable– Let PH be the program which solves H

– Use PH to construct program D which cannot exist

• Contradiction– This means program PH cannot exist.

– This implies H is not solvable

• D is the same as before

128

Arguing D cannot exist

• If D is a program, it must have an associated number y

• What does D do on this number y?

• 2 cases– D halts on y

• This means PH(D,y) = NO

– Definition of D

• This means D does not halt on y

– PH solves H

• Contradiction

• This case is not possible

129

Continued

– D does not halt on this number y

• This means PH(D,y) = YES

– Definition of D

• This means D halts on y– PH solves H

• Contradiction

• This case is not possible

– Both cases are not possible, but one must be for D to exist

– Thus D cannot exist

130

Placing the Halting Problem

Solvable

ll Problems

H

131

Implications *

• The Halting Problem is one of the simplest problems we can formulate about program behavior

• We can use the fact that it is unsolvable to show that other problems about program behavior are also unsolvable

• This has important implications restricting what we can do in the field of software engineering– In particular, “perfect” debuggers/testers do not exist

– We are forced to “test” programs for correctness even though this approach has many flaws

132

Summary

• Halting Problem definition– Basic problem about program behavior

• Halting Problem is unsolvable– We have identified a specific unsolvable

problem– Diagonalization technique

• Proof more complicated because we actually need to construct D, not just give a specification B

133

Module 8

• Halting Problem revisited– Universal Turing machine half-solves halting

problem– A universal Turing machine is an operating

system/general purpose computer

134

Half-solving the Halting Problem

• State the Halting Problem

• Give an input instance of the Halting Problem

• We saw last time that the Halting Problem is not solvable. How might we half-solve the Halting Problem?

135

Example InputProgram P


int i=2;


while (i<Q) {


i++;

}

return (true);

}

Nonnegative integer

4

136

OrganizationUniversal Turing machine’s Memory

Program P

Program P’s Memory

Program P


int i=2;


while (i<Q) {


i++;

}

return (true);

}

Program Counter

int i,Q;4

Line 1

137

Description of universal Turing machine *

• Basic Loop– Find current line of program P– Execute current line of program P

• Update program P’s memory

– Update program counter– Return to Top of Loop

Program P

Program P’s Memory

Program Counter

138

Past, Present, Future• Turing came up with the concept of a universal Turing machine in

the 1930’s– This is well before the invention of the general purpose computer

– People were still thinking of computing devices as special-purpose devices (calculators, etc.)

– Turing helped move people beyond this narrow perspective

• Turing/Von Neumann perspective– Computers are general purpose/universal algorithms

• Focused on computation• Stand-alone

• Today, we are moving beyond this view– Internet, world-wide web

– However, results in Turing perspective still relevant

139

Debuggers

• How are debugger’s like gdb or ddd related to universal Turing machines?

• How do debuggers simplify the debugging process?

140

Placing the Halting Problem

Solvable

Half-solvable

ll Problems

H

141

Summary

• Universal Turing machines– 1930’s, Turing– Introduces general purpose computing concept– Not a super intelligent program, merely a

precise follower of instructions

• Halting Problem half-solvable but not solvable

142

Module 9

• Closure Properties– Definition– Language class definition

• set of languages

– Closure properties and first-order logic statements

• For all, there exists

143

Closure Properties

• A set is closed under an operation if applying the operation to elements of the set produces another element of the set

• Example/Counterexample– set of integers and addition– set of integers and division

144

Integers and Addition

Integers

2

5

7

145

Integers and Division

Integers

2

5

.4

146

Language Classes

– We will be interested in closure properties of language classes

• A language class is a set of languages

• Thus, the elements of a language class (set of languages) are languages which are sets themselves

– Crucial Observation• When we say that a language class is closed under

some set operation, we apply the set operation to the languages (elements of the language classes) rather than the language classes themselves

147

Example Language Classes *

• In all these examples, we do not explicitly state what the underlying alphabet is

• Solvable languages (REC)– Languages whose language recognition problem can be solved

• Half-solvable languages (RE)– Languages whose language recognition problems can be half-

solved

• Finite languages– Languages with a finite number of strings

• CARD-3– Languages with at most 3 strings

148

Finite Sets and Set Union *

Finite Languages

{a,b,aa}

{a,b,bb}

{a,b,aa,bb}

All Languages

149

All Languages

CARD-3 and Set Union

CARD-3

{a,b,aa}

{a,b,bb}

{a,b,aa,bb}

150

All Languages

Finite Languages and Set Complement

Finite Languages

{a,b,ab}

{/\,aa,ba,bb,aaa,...}

151

Infinite Number of Facts

• A closure property often represents an infinite number of facts

• Example: The set of finite languages is closed under the set union operation– {} union {} is a finite language– {} union {} is a finite language– {} union {0} is a finite language– ...– {} union {} is a finite language– ...

152

First-order logic and closure properties *

• A way to formally write (not prove) a closure property– L1, ...,Lk in LC, op (L1, ... Lk) in LC

– Only one expression is needed because of the for all quantifier

• Number of languages k is determined by arity of the operation op

153

Example F-O logic statements *

• L1,L2 in FINITE, L1 union L2 in FINITE

• L1,L2 in CARD-3, L1 union L2 in CARD-3

• L in FINITE, Lc in FINITE

• L in CARD-3, Lc in CARD-3

154

Stating a closure property is false

• What is true if a set is not closed under some k-ary operator?– There exist k elements of that set which, when

combined together under the given operator, produce an element not in the set

– L1, ...,Lk in LC, op (L1, …, Lk) not in LC

• Example– Finite sets and set complement

155

Complementing a F-O logic statement

• Complement “L1,L2 in CARD-3, L1 union L2 in CARD-3”

– not (L1,L2 in CARD-3, L1 union L2 in CARD-3)

– L1,L2 in CARD-3, not (L1 union L2 in CARD-3)

– L1,L2 in CARD-3, L1 union L2 not in CARD-3

156

Proving/Disproving

• Which is easier and why?– Proving a closure property is true– Proving a closure property is false

157

Module 10

• Recursive and r.e. language classes– representing solvable and half-solvable problems

• Proofs of closure properties – for the set of recursive (solvable) languages– for the set of r.e. (half-solvable) languages

• Generic Element proof technique

158

RE and REC language classes

• REC– A solvable language is commonly referred to as

a recursive language for historical reasons– REC is defined to be the set of solvable or

recursive languages

• RE– A half-solvable language is commonly referred

to as a recursively enumerable or r.e. language– RE is defined to be the set of r.e. or half-

solvable languages

159

Closure Properties of REC *

• We now prove REC is closed under two set operations– Set Complement– Set Intersection

• In these proofs, we try to highlight intuition and common sense

160

REC and Set Complement

RECEVEN

All Languages

ODD• Even: set of even length strings

• Is Even solvable (recursive)?

• Give a program P that solves it.

• Complement of Even?– Odd: set of odd length strings

• Is Odd recursive (solvable)?

• Does this prove REC is closed under set complement?

• How is the program P’ that solves Odd related to the program P that solves Even?

161

P’ Illustration

PInput x Yes/No

P’

No/Yes

162

Code for P’

bool main(string y)

{

if (P (y)) return no; else return yes;

}

bool P (string y)

/* details deleted; key fact is P is guaranteed to halt on all inputs */

163

Set Complement Lemma

• If L is a solvable language, then L complement is a solvable language

• Proof– Let L be an arbitrary solvable language

• First line comes from For all L in REC

– Let P be the C++ program which solves L• P exists by definition of REC

164

– Modify P to form P’ as follows• Identical except at very end

• Complement answer – Yes → No– No → Yes

– Program P’ solves L complement• Halts on all inputs

• Answers correctly

– Thus L complement is solvable• Definition of solvable

proof continued

165

REC Closed Under Set Union

All Languages

L1

L2

L1 U L2

REC

• If L1 and L2 are solvable languages, then L1 U L2 is a solvable language

• Proof– Let L1 and L2 be arbitrary

solvable languages

– Let P1 and P2 be programs which solve L1 and L2, respectively

166

REC Closed Under Set Union

All Languages

L1

L2

L1 U L2

REC

– Construct program P3 from P1 and P2 as follows

– P3 solves L1 U L2 • Halts on all inputs• Answers correctly

– L1 U L2 is solvable

167

Yes/No

Yes/No

P3 Illustration

P1

P2

ORYes/No

P3

168

Code for P3

bool main(string y) {

if (P1(y)) return yes;

else if (P2(y)) return yes;

else return no;

}bool P1(string y) /* details deleted; key fact is P1 always halts. */

bool P2(string y) /* details deleted; key fact is P2 always halts. */

169

Other Closure Properties

• Unary Operations– Language Reversal– Kleene Star

• Binary Operations– Set Intersection– Set Difference– Symmetric Difference– Concatenation

170

Closure Properties of RE *

• We now try to prove RE is closed under the same two set operations– Set Union – Set Complement

• In these proofs– We define a more formal proof methodology– We gain more intuition about the differences

between solvable and half-solvable problems

171

RE Closed Under Set Union

• Expressing this closure property as an infinite set of facts– Let Li denote the ith r.e. language

• L1 intersect L1 is in RE


• ...


• ...

172

Generic Element or Template Proofs

• Since there are an infinite number of facts to prove, we cannot prove them all individually

• Instead, we create a single proof that proves each fact simultaneously

• I like to call these proofs generic element or template proofs

173

Basic Proof Ideas• Name your generic objects

– For example, L or L1 or L2

• Only use facts which apply to any relevant objects– For example, there must exist a program P1 that half-

solves L1

• Work from both ends of the proof– The first and last lines are often obvious, and we can

often work our way in

174

RE Closed Under Set UnionL1

L2

L1 U L2

• Let L1 and L2 be arbitrary r.e. languages

• L1 U L2 is an r.e. language

• There exist P1 and P2 s.t. Y(P1)=L1 and Y(P2)=L2

• There exists a program P that half-solves L1 U L2

• Construct program P3 from P1 and P2

• Prove Program P3 half-solves L1 U L2

175

• What code did we use for P3 when we worked with solvable languages?

Constructing Program P3

L1

L2

L1 U L2




else return no;}bool P1(string y) /* details deleted; key fact is P1 always halts. */

bool P2(string y) /* details deleted; key fact is P2 always halts. */

• Will this code work for half-solvable languages?

176

Proving P3 Is Correct

• 2 steps to showing P3 half-solves L1 U L2

– For all x in L1 U L2, must show P3

– For all x not in L1 U L2, must show P3

177

P3 works correctly?

– Let x be an arbitrary string in L1 U L2

• Note, this subproof is a generic element proof

– What are the two possibilities for x?• x is in L1

• x is in L2

– What does P3 do if x is in L1?• Does it matter if x is in or not in L2?


– Does P3 work correctly on such x?• If not, what strings cause P3 a problem?




else return no;

}

178

P3 works correctly?

– Let x be an arbitrary string NOT in L1 U L2


– x is not in L1 AND x is not in L2

– What does P3 do on x in this case?

• What does P1 do on x?

• What does P2 do on x?

– Does P3 work correctly on such x?




else return no;

}

179

Code for Correct P3

bool main(string y){if ((P1(y) || P2(y)) return yes; /* P1 and P2 run in parallel (alternating execution) */

else return no;}bool P1(string y)

/* key fact is P1 only guaranteed to halt on yes input instances */

bool P2(string y) /* key fact is P2 only guaranteed to halt on yes input instances */

180

P3 works correctly?

– Let x be an arbitrary string in L1 U L2


– What are the two possibilities for x?• x is in L1

• x is in L2



– Does P3 work correctly on such x?• If not, what strings cause P3 a problem?

bool main(string y){ if ((P1(y) || P2(y)) return yes; else return no;}/* P1 and P2 run in parallel */

181

• First-order logic formulation for statement– RE is closed under set complement

• What this really means– Let Li denote the ith r.e. language

• L1 complement is in RE

• L2 complement is in RE

• ...

RE and Set complement L

LcRE

182

RE and Set complement

• Let L be an arbitrary r.e. language

• L complement is an r.e. language

• There exists P s.t. Y(P)=L

• There exists a program P’ which half-solves L complement

• Construct program P’ from P• Prove Program P’ half-solves L complement

LLc

RE

183

Constructing P’• What did we do in recursive case?

– Run P and then just complement answer at end• Accept → Reject• Reject → Accept

• Does this work in this case?– No. Why not?

• Does this prove that RE is not closed under set complement?

184

Other closure properties

• Unary Operations– Language reversal– Kleene Closure

• Binary operations– set intersection– concatenation

• Not closed– Set difference (on practice hw)

185

Pseudo Closure Property

• Lemma: If L and Lc are half-solvable, then L is solvable.

• Question: What about Lc?

186

High Level Proof

– Let L be an arbitrary language where L and Lc are both half-solvable

– Let P1 and P2 be the programs which half-solve L and Lc, respectively

– Construct program P3 from P1 and P2

– Argue P3 solves L

– L is solvable

187

Constructing P3

• Problem– Both P1 and P2 may loop on some input strings,

and we need P3 to halt on all input strings

• Key Observation– On all input strings, one of P1 and P2 is

guaranteed to halt. Why?

188

Illustration

*

L

P1 halts

Lc

P2 halts

189

Construction and Proof

• P3’s Operation

– Run P1 and P2 in parallel on the input string x until one accepts x

• Guaranteed to occur given previous argument

• Also, only one program will accept any string x

– IF P1 is the accepting machine THEN yes ELSE no

190

P3 Illustration

P1

P2

Yes

Yes

P3

Input

Yes

No

191

Code for P3 *

bool main(string y)

{

parallel-execute(P1(y), P2(y)) until one returns yes;


if (P2(Y)) return no;

}bool P1(string y) /* guaranteed to halt on strings in L*/

bool P2(string y) /* guaranteed to halt on strings in Lc */

192

RE and REC

REC

RE

All Languages

L Lc

193

RE and REC

REC

RE

All Languages

LLc

Lc

Lc

Are there any languages L in RE - REC?

194

RE and REC

REC

RE

All Languages

H

So where does Hc belong?

Hc

195

Closure Property Questions

• Which of the following imply L is solvable given REC is closed under set union?– L1 U L2 = L

– L1 U L = L2

– L U L2 = L1

• In all cases, L1 and L2 are known to be solvable

196

Closure Property Questions

• Which of the following imply L is NOT solvable given REC is closed under set union?– L1 U L2 = L

– L1 U L = L2

– L U L2 = L1

• In all cases, L1 is solvable and L2 is NOT solvable

197

Summary• Definition of REC and RE

• Proofs of some closure properties for both language classes– RE more complex

• Pseudo-closure Property

• RE is not closed under set complement

• Proving a language is or is not in a language class using closure properties

198

Module 11

• Proving more specific problems are not solvable

• Input transformation technique– Use subroutine theme to show that if one

problem is unsolvable, so is a second problem– Need to clearly differentiate between

• use of program as a subroutine and

• a program being an input to another program

199

Basic Idea/Technique

200

Proving a problem L is unsolvable

• Assume PL is a procedure that solves problem L– We have no idea how PL solves L

• Construct a program PH that solves H using PL as a subroutine– We use PL as a black box– (We could use any unsolvable problem in place of H)

• Argue PH solves H– But we know H is unsolvable

• Conclude that L is unsolvable– Otherwise PL would exist and then H would be solvable– L will be a problem about program behavior

201

Focusing on H

• In this module, we will typically use H, the Halting Problem, as our known unsolvable problem

• The technique generalizes to using any unsolvable problem L’ in place of H.– You would need to change the proofs to work with L’

instead of H, but in general it can be done

• The technique also can be applied to solvable problems to derive alternative consequences

• We focus on H to simplify the explanation

202

Constructing PH using PL

Answer-preserving input transformations and Program PT

203

PH has two subroutines

• There are many ways to construct PH using program PL that solves L

• We focus on one method in which PH consists of two subroutines– Procedure PL that solves L– Procedure PT which computes a function f that I

call an answer-preserving (or answer-reversing) input transformation

• More about this in a moment

204

Two Representations of PH *

PH

P,y Yes/NoPLY/NPT

PT(P,y)

bool PH (Program P, unsigned y) { return PL (PT (P,y));}

205

Answer-preserving input transformation PT

• Input– An input to H

• Output– An input to L such that

• yes inputs of H map to yes inputs of L• no inputs of H map to no inputs of L

• Note, PT must not loop when given any legal input to H

206

Why this works *

PH

PLPT

yes input to H yes input to L yes

no input to H no input to L no

We have assumed that PL solves L

bool PH (Program P, unsigned y) { return PL (PT (P,y));}

207

Answer-reversing input transformation PT

• Input– An input to H

• Output– An input to L such that

• yes inputs of H map to no inputs of L• no inputs of H map to yes inputs of L

• Note, PT must not loop when given any legal input to H

208

Why this works

PH

PLPT

yes input to H no input to L yes

no input to H yes input to L no

We have assumed that PL solves L

no

yes

bool PH (unsigned x) { return complement(PL (PT (P,y)));}

209

Yes→Yes and No→No

Domain of H

Yes inputsfor H

No inputsfor H

Yes inputsfor L

No inputsfor L

Domain of L

PLPT

PH

x PT(x) Yes/No

210

Notation and Terminology

• If there is such an answer-preserving (or answer-reversing) input transformation f (and the corresponding program PT), we say that H transforms to (many-one reduces to) L

• NotationH ≤ L

Domain of H

Yes inputs No inputs

Yes inputs No inputsDomain of L

211

Examples not involving the Halting Problem

212

Generalization

• While we focus on transforming H to other problems, the concept of transformation generalizes beyond H and beyond unsolvable program behavior problems

• We work with some solvable, language recognition problems to illustrate some aspects of the transformation process in the next few slides

213

Example 1

• L1 is the set of even length strings over {0,1}– What are the set of legal input instances and no inputs for

the L1 LRP?

• L2 is the set of odd length strings over {0,1}– Same question as above

• Tasks– Give an answer-preserving input transformation f that

shows that L1 LRP ≤ L2 LRP

– Give a corresponding program PT that computes f

Domain of L1


Yes inputs No inputsDomain of L2

214

Program PT

string main(string x)

{

return(x concatenate “0”);

}

215

Example 2

• L1 is the set of all strings over {0,1}

– What is the set of all inputs, yes inputs, no inputs for the L1 LRP?

• L2 is {0}– Same question as above

• Tasks– Give an answer-preserving input transformation f which

shows that the L1 LRP ≤ L2 LRP

– Give a corresponding program PT which computes f

Domain of L1



216

Program PT

string main(string x)

{

return( “0”);

}

217

Example 3

• L1 – Input: Java program P that takes as input an unsigned int– Yes/No Question: Does P halt on all legal inputs

• L2

– Input: C++ program P that takes as input an unsigned int– Yes/No Question: Does P halt on all legal inputs

• Tasks– Describe what an answer-preserving input transformation f

that shows that L1 ≤ L2 would be/do?

Domain of L1



218

Proving a program behavior problem L is unsolvable

219

Problem Definitions *

• Halting Problem H– Input

• Program QH that has one input of type unsigned int

• non-negative integer y that is input to program QH

– Yes/No Question• Does QH halt on y?

• Target Problem L– Input

• Program QL that has one input of type string

– Yes/No question• Does Y(QL) = the set of

even length strings?

• Assume program PL solves L

220

Construction review

PH

x Yes/No

•We are building a program PH to solve H

bool PH (unsigned x) { return}

PTPT(x)

•PH will use PT as a subroutine, and we must explicitly construct PT using specific properties of H and L

PT (P,y)

PLY/N

•PH will use PL as a subroutine

PL ( );

221

P’s and Q’s• Programs which are PART of program PH and

thus “executed” when PH executes– Program PT, an actual program we construct– Program PL, an assumed program which solves

problem L

• Programs which are INPUTS/OUTPUTS of programs PH, PL, and PT and which are not “executed” when PH executes– Programs QH, QL, and QYL

• code for QYL is available to PT

222

Two inputs for L *


• Program Q that has one input of type string

– Yes/No question• Does Y(Q) = the set of


• Program PL

– Solves L– We don’t know how

• Consider the following program Q1

bool main(string z)

{while (1>0) ;}

– What does PL output when given Q1 as input?

• Consider the following program Q2

bool main(string z)

{ if ((z.length %2) = = 0) return (yes)

else return (no); }

– What does PL output when given Q2 as input?

223

Another input for L *


• Program Q that has one input of type string

– Yes/No question• Does Y(Q) = the set of


• Program PL

– Solves L– We don’t know how

• Consider the following program QL with 2 procedures Q1 and QYL

bool main(string z) {

Q1(5); /* ignore return value */

return(QYL(z));}

bool Q1(unsigned x) {if (x > 3) return (no); else loop;

}

bool QYL(string y) {if ((y.length( ) % 2) = = 0) return (yes);

else return(no);}

• What does PL output when given QL as input?

224

Input and Output of PT *

• Input of PT (and H)

– Program QH

• one input of type unsigned int

– Non-negative integer y

• Program QL that is the output of PT

and input of Lbool main(string z) {

QH(y); /* QH and y come left-hand side */

/* ignore return value */

return(QYL(z));

}

bool QH(unsigned x) {

/* comes from left-hand side

}

bool QYL(string y) {

if ((y.length( ) % 2) = = 0) return (yes); else return(no);

}

QH,y PT QL

225

Example 1 *

Input to PT

Program QH

bool main(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}

Non-negative integer y5

Output of PT

Program QL

bool QH(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}bool QYL(string z) { if ((z.length % 2) == 0) return (yes) else return (no);}bool main(string z) { unsigned y = 5; QH(y); return (QYL(z));}

QH,y PT QL

226

Example 2

Input to PT

Program QH

bool main(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}

Non-negative integer y3

Output of PT

Program QL

bool QH(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}bool QYL(string z) { if ((z.length % 2) == 0) return (yes) else return (no);}bool main(string z) { unsigned y = 3; QH(y); return (QYL(z));}

QH,y PT QL

227

PT in more detail

228

Declaration of PT

• What is the return type of PT?– Type program1 (with one input of type string)

• What are the input parameters of PT?– The same as the input parameters to H; in this case,

• type program2 (with one input of type unsigned int)• unsigned int (input type to program2)

program1 main(program2 QH, unsigned y)

PLPT

PH

QH,y QL Yes/No

229

program1 main(program2 QH, unsigned y) {/* Will be viewing types program1 and program2 as STRINGS over the

program alphabet P */

program1 QL = replace-main-with-QH(QH);/* Insert line break */

QL += “\n”;

/* Insert QYL */

QL += “bool QYL(string z) {\n \t if ((z.length % 2) == 0) return (yes) else return (no);\n }”;

/* Add main routine of QL */

QL += “bool main(string z) {\n\t”; /* determined by L */

QL += “unsigned y =”

QL += convert-to-string(y);

QL += “;\n\t QH(y)\n\t return(QYL(z));\n}”;

return(QL);}

program1 replace-main-with-QH(program2 P) /* Details hidden */string convert-to-string(unsigned y) /* Details hidden */

Code for PT PLPT

PH

QH,y QL Yes/No

230

PT in action

PT

code for QYL

Program QHbool main(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}

Input y5

Program QLbool QH(unsigned y) { if (y ==5) return yes; else if (y ==4) return no; else while (1>0) {};}bool QYL(string z) { if ((z.length % 2) == 0) return (yes) else return (no);}bool main(string z) { unsigned y = 5; QH(y); return (QYL(z));}

PLPT

PH

QH,y QL Yes/No

231

Constructing QL (and thus PT)

How to choose QYL or QNL

232

Start with no input for H

• If QH, y is a no input to the Halting problem

• Program QL bool main(string z) {

QH(y); /* ignore return value */

return(Q?L(z)); /* yes or no? */}

bool QH(unsigned x) {/* comes from left-hand side

}

bool Q?L(string y) {

}

– Thus Y(QL) = {}

– QH loops on y

– Determine if this makes QL a no or yes input instance to L

233

Answer-preserving input transformation


– Thus Y(QL) = {}

– QH loops on y




return(QYL(z)); /* yes */}


}

bool QYL(string y) {

}

– Now choose a QYL (or QNL) that is a yes (or no) input instance to L

234

Make yes for H map to yes for L


– Thus Y(QL) = {}

– QH loops on y







}

bool QYL(string y) {if ((y.length( ) % 2) = = 0) return (yes);else return (no);

}

235

Possible shortcutProgram QL bool main(string z) {

QH(y); /* ignore return value */if ((z.length( ) % 2) = = 0) return (yes);else return (no);

}


}

Program QL bool main(string z) {




}


}

236

Another Example

237

Problem Definitions

• Halting Problem H– Input

• Program QH that has one input of type unsigned int

• non-negative integer y that is input to program QH

– Yes/No Question• Does QH halt on y?


• Program QL that has one input of type string

– Yes/No question• Is Y(QL) finite?

• Assume program PL solves L

238

Start with no input for H




return(Q?L(z)); /* yes or no? */}


}

bool Q?L(string y) {

}

– Thus Y(QL) = {}

– QH loops on y


239

Answer-reversing input transformation




return(QNL(z)); /* no */}


}

bool QNL(string y) {

}

– Thus Y(QL) = {}

– QH loops on y



240

Make yes for H map to no for L




return(QNL(z)); /* no */}


}

bool QNL(string y) {if ((y.length( ) % 2) = = 0) return(yes);else return(no);

}

– Thus Y(QL) = {}

– QH loops on y



241

Analyzing proposed transformations

4 possibilities

242

Problem Setup• Input of Transformation

• Program QH, unsigned x• Output of Transformation



return(QYL(z)); /* yes or no */}

bool QH(unsigned x) {}


}

• Problem L• Input: Program P• Yes/No Question: Is Y(P) =

{aa}?

• Question: Is the transformation on the left an answer-preserving or answer-reversing input transformation from H to problem L?

243

Key Step• Input of Transformation

• Program QH, unsigned x• Output of Transformation






}

• Problem L• Input: Program P• Yes/No Question: Is Y(P) =

{aa}?

• The output of the transformation is the input to the problem.• Plug QL in for

program P above• Is Y(QL) = {aa}?

244

Is Y(QL) = {aa}?• Problem L

• Input: Program P• Yes/No Question: Is Y(P) =

{aa}?• Analysis

• If QH loops on x, Y(QL)={}• No input to H creates a QL that

is a no input for L• If QH halts on x, Y(QL) =

{even length strings}• Yes input to H creates a QL

that is a no input for L• Transformation does not work

• All inputs map to no inputs

• Input of Transformation• Program QH, unsigned x

• Output of Transformation• Program QL bool main(string z) {





}

245

Three other problems• Problem L1

• Input: Program P• Yes/No Question: Is Y(P)

infinite?

• Problem L2• Input: Program P• Yes/No Question: Is Y(P)

finite?

• Problem L3• Input: Program P• Yes/No Question: Is Y(P) =

{} or is Y(P) infinite?







}

246

Is Y(QL) infinite?• Problem L1


infinite?• Analysis

• If QH loops on x, Y(QL)={}• No input to H creates a QL

that is a no input for L• If QH halts on x, Y(QL) =


that is a yes input for L• Transformation works

• Answer-preserving







}

247

Is Y(QL) finite?• Problem L2


finite?• Analysis


that is a yes input for L• If QH halts on x, Y(QL) =


that is a no input for L• Transformation works

• Answer-reversing







}

248

Is Y(QL) = {} or is Y(QL) infinite?• Problem L3

• Input: Program P• Yes/No Question: Is Y(P) =

{} or is Y(P) infinite?• Analysis


that is a yes input for L• If QH halts on x, Y(QL) =


that is a yes input for L• Transformation does not work

• All inputs map to yes inputs







}

Documents

Module 1: Course Overview