ALGORITHMS

Introduction

Definition

Algorithm:Any well-defined computational procedure that takes some value or set of values as input and produces some value or set of values as output.

Properties of Algorithms• Input– an algorithm has input values from a specified set

• Output– for each set of input values, an algorithm produces

output values from a specified set

• Definiteness– the steps of the algorithm must be defined precisely

Properties of Algorithms (cont)• Finiteness– an algorithm should produce the desired output after a

finite number of steps for any input in the set

• Effectiveness– it must be possible to perform each step of an

algorithm exactly in a finite amount of time

• Generality– the algorithm should be applicable for all problems of

the desired form, not just a set of input values

Question 1

• How do you show that an algorithm is incorrect?

Question 2

• How do you show that an algorithm is correct ?

Computational Model for Analysis

• One processor random-access machine (RAM) – instructions are executed one at a time– no concurrent operations

Choosing the “Best” Algorithm

• Empirical approach– try competing algorithms on several different

problem instances• Theoretical approach– determine mathematically the quantity of

resources (execution time and memory) needed by the algorithm as a function of the size of the instance considered

What is “Input Size”?

• Problem dependent• Identification of objects in problem processed

by algorithm– Number of items in the input is often used (e.g.

length of an array)– Sometimes use number of bits– May use more than one measure (number of

edges and vertices in a graph)

What is the “Running Time”

• Number of primitive operations or “steps” executed.

• Want this definition to be machine independent

• Initial view taken by text – constant amount of time for each line of pseudocode

Insertion Sort Example

• Commonly used sorting algorithm• Sorting of playing cards often used as an

analogy

5 2 4 6 1 3 2 6

5 4 6 1 3 2 6

Initially sorted list

key 2

A

5 5 4 6 1 3 2 6

5 4 6 1 3 2 6

key 2

A

2 5 4 6 1 3 2 6

2 5 4 6 1 3 2 6

A

2 5 4 6 1 3 2 6

2 5 6 1 3 2 6

key 4

A

2 5 5 6 1 3 2 6

2 5 6 1 3 2 6

key 4

A

2 4 5 6 1 3 2 6

2 4 5 6 1 3 2 6

A

INSERTION-SORT (A) cost time

1 for j 2 to length[A] do c1 n

2 key A[j] c2 n-1

3 ;Insert A[j] into the sorted

; sequence A[1 .. j -1] 0 n-1

4 i j - 1 c4 n-1

5 while i > 0 and A[i] > key do c5

6 A[i+1] A[i] c6

7 i i- 1 c7

8 A[i+1] key c8 n-1

t jj2n

)1(2

n

j jt

)1(2

n

j jt

Running Time for Insertion Sort

)1()1()1(

)1()1()(

82726

25421

nctctc

tcncncncnT

nj j

nj j

nj j

Why Worst Case Analysis? • Gives an upper bound on the running time

for any input• For some algorithms, the worst case occurs

fairly often• Difficult to identify the average case• Difficult to analyze the average case• The average case is often roughly as bad as

the worst case.

Order of Growth

• Consider only the highest order terms in formulas

• Ignore the leading term’s constant coefficient

• For insertion sort the worst case running time is

(n2)

Designing Algorithms

• A number of design paradigms for algorithms that have proven useful for many types of problems

• Insertion sort – incremental approach• Other examples of design approaches– divide and conquer– greedy– dynamic programming

Divide and Conquer

• Good divide and conquer algorithm generally implies easy recursive version of the algorithm

• Three steps– divide– conquer– combine

Merge Sort

• Dividedivide and n-element sequence into two n/2 element sequences

• Conquerif the resulting list is of length 1 it is sortedelse call the merge sort recursively

• Combinemerge the two sorted sequences

MERGE-SORT (A,p,r)

1 if p < r

2 then q(p+r)/2

3 MERGE-SORT(A,p,q)

4 MERGE-SORT(A,q+1,r)

5 MERGE(A,p,q,r)

To sort A[1..n], invoke MERGE-SORT with

MERGE-SORT(A,1,length(A))

Merge

1 3

Merge

4 6

1 2 2 3 4 5 6 6

Merge

2 4 5 6

Merge

2 5

Merge

2 6

Merge

initial sequence

sorted sequence

4 6 1 25 2 3 6

1 2 3 6

Merge

Recurrence for Divide and Conquer Algorithms

)()()/(

for )1()(

nCnDbnaT

cnnT

Recurrence for Merge Sort

1 if)()2/(2

1 = if)1()(

nnnT

nnT