What is an Algorithm

1

Algorithmic FoundationsCOMP108

COMP108Algorithmic Foundations

Introduction + Mathematical Induction

Prudence Wong

http://www.csc.liv.ac.uk/~pwong/teaching/comp108/200708


Module information …


3

(Intro + MI)

Teaching Team

Dr Prudence Wong

Demonstrator

Mr Matt Webster

Rm 3.18 Ashton Building

[email protected]

Office hours: Monday 3-5pm


4

(Intro + MI)

Module Structure

Lectures

Tutorials Help channels

Assignments& Class Tests

Examination

2


5

(Intro + MI)

~36 lectures(Mon 2-3pm, Thu 10-11am, Fri 10-11am)

12 tutorials, 1 hr per week

Office hours, email, appointment (by email)

2 Assignments & 2 Class Tests(continuous assessment)

A written examination(80% of final mark)


6

(Intro + MI)

Main ReferenceIntroduction to the Designand Analysis of AlgorithmsAnany V. LevitinVillanova UniversityAddison Wesley

Additional ReadingIntroduction to AlgorithmsThomas H. Cormen,Charles E. Leiserson,Ronald L. RivestThe MIT Press


7

(Intro + MI)

More about tutorials

Tutorials start this week

Goto http://www.csc.liv.ac.uk/~pwong/teaching/comp108/200708

to register for a tutorial session

Tutorials are either held in Rm 310 Ashton Building and Lab 4 George Holt Building

Check the website to see the location for each tutorial

The first tutorial will be held inRm 310 Ashton Building


8

(Intro + MI)

More about assignments

� plagiarism is the practise of presenting the thoughts or writings of another or others as original

� consequence?

� What happen if you are ill when there is a deadline for assignment?

� Get medical proof and ask for extension

� This is not an excuse to commit plagiarism

declaration form on Plagiarism and Collusion

3


9

(Intro + MI)

More about Class Tests

There will be two class tests

� Friday 15 February 10-11am

� Friday 18 April 10-11am

Scope will be announced closer to the time

Incorrect dates in handout


10

(Intro + MI)

Why COMP108?

Pre-requisite for: COMP202, COMP209(COMP202 is pre-requisite for COMP308, COMP309)

Algorithm design is a foundation for efficient and effective programs

"Year 1 modules do not count towards honour classification…"

(Message from Career Services:

Employers DO consider year 1 module results)


11

(Intro + MI)

Aims� To give an overview of the study of algorithms in terms of

their efficiency.

� To introduce the standard algorithmic design paradigmsemployed in the development of efficient algorithmic solutions.

� To describe the analysis of algorithms in terms of the use of formal models of Time and Space.

� To give a brief introduction to the subject of computational complexity theory and its use in classifying computational problems.

What do we mean by good?

How to achieve?

Can we prove?

Can all problems be solved efficiently?


Ready to start …

4


13

(Intro + MI)

Learning outcomes

� Able to tell what an algorithm is and have some understanding why we study algorithms

� Able to use pseudo code to describe algorithm

� Able to prove by Mathematical Induction


14

(Intro + MI)

Learning outcomes





15

(Intro + MI)

What is an algorithm?

A sequence of precise and concise instructions that guide you (or a computer) to solve a specific problem

Daily life examples: cooking recipe, furniture assembly manual(What are input / output in each case?)

Input Algorithm Output


16

(Intro + MI)

Why do we study algorithms?

Example: We are given a map with n cities and the traveling cost between the cities. What is the cheapest way to go from city A to city B?

The obvious solution to a problem may not be efficient

Simple solution� Compute the cost of each

route from A to B� Choose the cheapest one

108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

A

B

5


17

(Intro + MI)

Shortest path to go from A to B

How many routes between A & B?

involving 1 intermediate city?




A

B2


18

(Intro + MI)



involving 1 intermediate city? 3 cities?




A

B2

Number of routes= 2


19

(Intro + MI)







A

B

3

Number of routes= 2Number of routes

= 2 + 3


20

(Intro + MI)







A

B

Number of routes= 2 + 3

6


21

(Intro + MI)







A

B

6

Number of routes= 2 + 3 + 6= 11

Number of routes= 2 + 3


22

(Intro + MI)



involving 1 intermediate city? 3? 8?




A

B

For large n, it is impossible to check all routes!We need more sophisticated solutions

TOO MANY!!


23

(Intro + MI)


A

B

There is an algorithm, called Dijkstra's algorithm, that can compute this shortest path efficiently.


24

(Intro + MI)

Idea of Dijkstra's algorithm

108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

Go one step from A, label the visited city with the cost.

A

B10

0

7


25

(Intro + MI)


108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

Choose city with the smallest cost and go one step from it similarly.

A

B10

0

5


26

(Intro + MI)


108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

Again ...

A

B10

5

7

0


27

(Intro + MI)


108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

Choose city with the smallest cost. All neighbours of A have labels already.Choose the next smallest one.

A

B

5

7

0

10


28

(Intro + MI)

108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

108

Idea of Dijkstra's algorithmAn alternative cheaper route is found!Record this route and erase the more expensive one.

A

B

5

7

0

This path must NOT be used because we find a cheaper alternative path

8


29

(Intro + MI)


108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

Repeat and repeat ...

A

B

5

7

0

8

17


30

(Intro + MI)


Finally, the cheapest route from A to every other city will be discovered.

A

B?

We will further discuss Dijkstra’s algorithm later.

108

5

79

6

11

95

2

123

65

4

5

7

3

1

5

3

5

7

0

8

??

?

??


31

(Intro + MI)

Lesson to learn

The most straightforward (or brute force) algorithm

for a problem may run slowly. We need more sophisticated solutions.


32

(Intro + MI)

Learning outcomes




9


How to represent algorithms …


34

(Intro + MI)

Algorithm vs Program

Algorithms are free from grammatical rules� Content is more important than form

� Acceptable as long as it tells people how to perform a task

Programs must follow some syntax rules� Form is important

� Even if the idea is correct, it is still not acceptable if there is syntax error

Still remember? An algorithm is a sequence of precise and concise instructions that guide a computer to solve a specific problem


35

(Intro + MI)

Computing the n-th power

Input: a number x & a non-negative integer n

Output: the n-th power of x

Algorithm:

1. Set a temporary variable p to 1.

2. Repeat the multiplication p = p * x for n times.

3. Output the result p.


36

(Intro + MI)

Pseudo Codep = 1

i = 1

while i <= n do

begin

p = p * x

i = i+1

end

output p

p = 1

for i = 1 to n do

p = p * x

output p

Another way to describe algorithm is by pseudo code

similar to programming language

more like English Combination of both

10


37

(Intro + MI)

Pseudo Code: statement

Assignment statementvariable = expression

e.g., assign the expression 3 * 4 to the variable resultresult = 3 * 4

compound statementsbegin

statement1

statement2

end


38

(Intro + MI)

Pseudo Code: conditional

Conditional statementif condition then

statement

if condition thenstatement

elsestatement

if a > 0 then

abs = a

else

abs = -a

output abs

if a < 0 then

a = -a

abs = a

output abs

What do these two algorithms compute?


39

(Intro + MI)

Pseudo Code: iterative (loop)

Iterative statementfor var = start_value to end_value dostatement

while condition do

statement

repeatstatement

until condition

condition to CONTINUE the loop

condition to STOP the loop

condition for while loop is NEGATION of

condition for repeat-until loop


40

(Intro + MI)

the loop is executed for n times

for loopfor var = start_value to end_value dostatement

Computing sum of the first n numbers:

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

1. var is first assigned the value start_value.

2. If var ≤ end_value, execute the statement.

3. var is incremented by 1 and then go back to step 2.

11


41

(Intro + MI)


i=1

i <= n?

sum = sum+i

No

Yes



input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum


42

(Intro + MI)




input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

iteration i sum

before 0

1 1 1

2 2 3

3 3 6

4 4 10

end 5

suppose n = 4


43

(Intro + MI)

while loopwhile condition do

statement


input n

sum = 0

i = 1

while i <= n do

begin

sum = sum + i

i = i + 1

end

output sum

1. if condition is true, execute the statement

2. else stop

3. go back to step 1

this while-loop performs the same task as the for-loop in the previous slides

condition to CONTINUE the loop


44

(Intro + MI)

Computing sum of all input numbers:

sum = 0

while (user wants to continue) do

begin

ask for a number

sum = sum + number

end

output sum

while loop – example 2this loop is executed for undeterminednumber of times

continue?

ask for number

sum = sum+number

No

Yes

12


45

(Intro + MI)

repeat-untilrepeatstatement

until condition


sum = 0

repeat

ask for a number

sum = sum + number

until (user wants to stop)

output sum

1. execute the statement

2. if condition is true, stop

3. go back to step 1

� this loop is also executed for undeterminednumber of times

�How it differsfrom while-loop?

condition

to STOP the loop


46

(Intro + MI)

repeat-untilrepeatstatement

until condition


sum = 0

repeat

ask for a number

sum = sum + number

until (user wants to stop)

output sum

� this loop is also executed for undeterminednumber of times

�How it differsfrom while-loop?

Stop?

ask for number

sum = sum+number

Yes

No

condition

to STOP the loop


47

(Intro + MI)

Pseudo Code: exercise

Write for-loops for the followings

1.Find the product of all integers in the interval [x, y], assuming that x and y are both integers

� e.g., if x and y are 2 and 5, then the output should be 2*3*4*5 = 120

2.List all factors of a given positive integer x

� e.g., if x is 60, then the output should be 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, 30, 60


48

(Intro + MI)

Hint (1)

Find the product of all integers in the interval [x, y], assuming that x and y are both integers

product = 1

for i = x to y do

begin

product = product * i

end

output product

13


49

(Intro + MI)

you may use the operator '%'.a%b means the remainder ofa divided b

Hint (2)

List all factors of a given positive integer x

for i = 1 to x do

begin

if (x%i == 0) then

output i

end


50

(Intro + MI)

Convert to while loopsFind the product of all integers in the interval

[x, y], assuming that x and y are both integers

product = 1

i = x

while i <= y do

begin


i = i+1

end

output product


51

(Intro + MI)

Convert to while loops (2)


i = 1

while i <= x do

begin

if x%i == 0 then

output i

i = i+1

end


52

(Intro + MI)

Convert to repeat loopsFind the product of all integers in the interval

[x, y], assuming that x and y are both integers

product = 1

if x <= y then begin

i = x

repeat


i = i+1

until i > y

output product

end

14


53

(Intro + MI)

Convert to repeat loops (2)


i = 1

repeat

if x%i == 0 then

output i

i = i+1

until i > x


54

(Intro + MI)

Learning outcomes





Mathematical Induction …


56

(Intro + MI)

Analysis of Algorithms

After designing an algorithm, we analyze it.

� Proof of correctness: show that the algorithm gives the desired result

� Time complexity analysis: find out how fast the algorithm runs

� Space complexity analysis: decide how much memory space the algorithm requires

� Look for improvement: determine whether the algorithm is best possible; can we improve it to run faster or use less memory?

15


57

(Intro + MI)

A typical analysis technique - MIMathematical Induction (MI)

� technique to prove that a given statement is true for all natural numbers (or is true for all members of an infinite sequence)

E.g., To prove 1+2+…+n = n(n+1)/2 ∀∀∀∀ +ve integers n

� when n is 1, L.H.S = 1, R.H.S = 1*2/2 = 1 OK!� when n is 2, L.H.S = 1+2 = 3, R.H.S = 2*3/2 = 3 OK!� when n is 3, L.H.S = 1+2+3 = 6, R.H.S = 3*4/2 = 6 OK!

However, none of these constitute a proof and we cannot enumerate over all possible numbers.

=> Mathematical Induction

for all


58

(Intro + MI)

Mathematical Induction

Two steps

� Basic step / Base case: show that the statement is true for some base case, usually true for 0 or 1

� Induction step: show that if the statement is true for some integer k, then it is also true for some integer k+1


59

(Intro + MI)

Example

� To prove: 1+2+…+n = n(n+1)/2 ∀∀∀∀ +ve integers n

� Base case: When n=1, L.H.S = 1, R.H.S = 1*2/2=1. Therefore, the statement is true for n=1.

� Induction hypothesis: Assume that statement is true when n=k for some integer k ≥ 1.

� i.e., assume that 1+2+…+k = k(k+1)/2

� Induction step: When n=k+1,

� L.H.S = 1+2+...+k+(k+1) = ......

� R.H.S = (k+1)((k+1)+1)/2 = ......


60

(Intro + MI)

Example - Induction step

� L.H.S = 1+2+...+k+(k+1)

= k(k+1)/2 + (k+1) ←←←← by hypothesis

= ((k2+k) + 2(k+1)) / 2

= (k2+3k+2)/2

� R.H.S = (k+1)((k+1)+1)/2

= (k+1)(k+2)/2

= (k2+3k+2)/2

= L.H.S

�So, the statement is also true for n=k+1

Induction hypothesis1+2+…+k = k(k+1)/2

Target: to prove1+2+…+k+(k+1) = (k+1)((k+1+1))/2

16


61

(Intro + MI)

Example - ConclusionWe have proved

1. statement true for n=1

2. if statement is true for n=k, then also true for n=k+1

In other words,

� true for n=1 implies true for n=2 (induction step)



� and so on ......

Conclusion: true for all +ve integers n


62

(Intro + MI)

To prove n3+2n is divisible by 3 ∀∀∀∀ integers n≥0

�Base case: When n=0, n3+2n=0, divisible by 3. So statement true for n=0.

�Induction hypothesis: Assume that statement is true for k, i.e., assume k3+2k is divisible by 3

�Induction step: When n=k+1,

� (k+1)3+2(k+1) = (k3+3k2+3k+1) + (2k+2)

= (k3+2k) + 3(k2+k+1)

�Conclusion: statement true ∀∀∀∀ integers n≥≥≥≥0

by hypothesis, divisible by 3

Example 2

divisible by 3

statement true for k+1

Target: to prove(k+1)3+2(k+1)is divisible by 3


63

(Intro + MI)

NoteThe induction step means that if statement is

true for some natural number k, thenstatement is also true for k+1.

It does NOT mean that the statement must be true for k nor for k+1.

Therefore, we MUST prove that statement is true for some starting natural number n0, which is the base case.

Missing the base case will make the proof fail.


64

(Intro + MI)

More example� To prove 2n < n! ∀∀∀∀ +ve integers n ≥ 4.

� Base case: When n=4, L.H.S = 24 = 16, R.H.S = 4! = 4*3*2*1 = 24, L.H.S < R.H.S. So, statement true for n=4

� Induction hypothesis: Assume that statement is true for some integer k ≥ 4, i.e., assume 2k < k!

� Induction step: When n=k+1

� L.H.S = 2k+1 = 2*2k < 2*k! ←←←← by hypothesis

� R.H.S = (k+1)! = (k+1)*k! > 2*k! > L.H.S ←←←← because k+1>2

� So, statement true for k+1

� Conclusion: statement true ∀∀∀∀ +ve integers n ≥ 4.

n! = n(n-1)(n-2) … *2*1

Target: to prove2k+1<(k+1)!

17


65

(Intro + MI)

More example� To prove 2n < n! ∀∀∀∀ +ve integers n ≥ 4.

� Base case: When n=4, L.H.S = 24 = 16, R.H.S = 4! = 4*3*2*1 = 24, L.H.S < R.H.S. So, statement true for n=4

� Induction hypothesis: Assume that statement is true for some integer k ≥ 4, i.e., assume 2k < k!

� Induction step: When n=k+1

� L.H.S = 2k+1 = 2*2k < 2*k! ← by hypothesis

� R.H.S = (k+1)! = (k+1)*k! > 2*k! > L.H.S ← because k+1>2

� So, statement true for k+1

� Conclusion: statement true ∀ +ve integers n ≥ 4.

Why base case is n=4?When n=1, L.H.S=21=2, R.H.S=1!=1When n=2, L.H.S=22=4, R.H.S=2!=2When n=3, L.H.S=23=8, R.H.S=3!=6Statement is not true for n=1, 2, 3


66

(Intro + MI)

ExerciseTo prove 12+22+32+...+n2 = n(n+1)(2n+1)/6

Base case: when n=1,L.H.S = 1, R.H.S = 1*2*3/6=1=L.H.S

Induction hypothesis: Assume statement is true for n=k, i.e., assume that 12+22+32+...+k2 = k(k+1)(2k+1)/6

Induction step: When n=k+1,L.H.S = …R.H.S = … = L.H.S

Then statement is true for n=k+1

By principle of MI, statement is true for all +ve integers

Target: to prove 12+22+…+k2+(k+1)2=(k+1)((k+1)+1)(2(k+1)+1)/6


67

(Intro + MI)

Exercise (cont’d)Induction Step: When n = k+1

L.H.S. = 12+22+32+...+k2+(k+1)2

= k(k+1)(2k+1)/6 + (k+1)2 ←←←← by hypothesis

= (k2+k)(2k+1)/6 + (k2+2k+1)

= ((2k3+ k2+2k2+k) + (6k2+12k+6))/6

= (2k3+ 9k2+13k+6)/6

R.H.S. = (k+1)((k+1)+1)(2(k+1)+1)/6

= (k+1)(k+2)(2k+3)/6

= (k2+3k+2)(2k+3)/6

= (2k3+3k2+6k2+9k+4k+6)/6

= (2k3+ 9k2+13k+6)/6 = L.H.S.

Therefore, the statement is true for n=k+1.By the principle of MI, the statement is true ∀∀∀∀ +ve integers


68

(Intro + MI)

ExerciseProve that 1 + 3 + 5 + … + (2n-1) = n2 for all +ve integers

>= 1

Base case: When n=1, L.H.S.=2*1-1=1, R.H.S.=12=1

Induction hypothesis: Assume statement is true for some integer k, i.e., assume 1+3+5+…+(2k-1)=k2

Induction step: When n=k+1

L.H.S. = 1+3+5+…+(2k-1)+(2(k+1)-1)

= k2+2k+1 = (k+1)2 = R.H.S.

Therefore, statement is true for n=k+1

Target: to prove1+3+5+…+(2k-1)+(2(k+1)-1))=(k+1)2

By principle of MI, statement is true for all +ve integers

18


69

(Intro + MI)

Learning outcomes




Documents

What is an Algorithm