Algorithms

1. Notion of an algorithm2. Properties of an algorithm3. The GCD algorithm4. Correctness of the GCD algorithm5. Termination of the GCD algorithm6. Performance7. Data structures

1-Notion of an algorithm

DefinitionAn algorithm is a clearly specified set of instructions describing the solution to a specific problem.

An algorithm• takes the input and transforms it into an adequate

output,• must be independent from any programming language,• is written in a level of detail that allows to reproduce it in

any programming language,• has to be designed so it can be reused and understood

by others.

2-Properties of an algorithm

Some properties must be satisfied by an algorithm in order to allow a successful execution of the corresponding program:

1. Correctness: if the input conditions are satisfied and the algorithm instructions executed, then the correct output is produced.

2. Termination: the algorithm must terminate after a finite number of steps. Thus, it has to be composed by a finite number of steps. This can be ensured if the algorithm avoids an infinite loop.

3. Performance: Quantification of the space and time complexities.

3-The GCD algorithm:

Given two positive integers m and n, find the greatest common divisor, gcd(m,n).

A view of the problem:All numbers from 1 up to the smallest of m and n, say n.

Naive algorithm:Go through search space (from 1 to n)Keep track of largest number that divides both m and n.

Is there a more efficient way of doing this?

Yes, the Euclidean Algorithm (Euclid – c.350 B.C.E.)

Pseudocode of the GCD algorithm :

Algorithm GreatestCommonDivisor

Input: Two positive integers, m and n

Ouptut: The gcd of m and n

repeatr m mod nm nn r

until (r == 0)Output m and STOP.

Example:

m = 24, n = 9r 24 mod 9 = 6m 9n 6r 9 mod 6 = 3m 6n 3r 6 mod 3 = 0m 3n 0Output 3STOP

How does the algorithm works?

Given m and n, how many times does “repeat” execute?

First, we need the following result:

Theorem 1:

If m > n, then m mod n < m/2

Proof:

If n m/2, then the claim follows since:r = m mod n < n

Ex. 5 11/2 r = 11 mod 5 = 1 < 5 11/2

If n > m/2, then r = m mod n = m – n < m/2Ex. 7 > 11/2 r = 11 mod 7 = 4 < 11/2 q.e.d.

Consider sequence of remainders:

r0 = m mod n < m/2r1 = n mod r0 < n/2r2 = r0 mod r1 < r0/2r3 = r1 mod r2 < r1/2r4 = r2 mod r3 < r2/2r5 = r3 mod r4 < r3/2 …..

Second iteration: r1 < n/2Third iteration: r2 < r0/2 < m/4Fourth iteration: r3 < r1/2 < n/4Fifth iteration: r4 < r2/2 < r0/4 < m/8Sixth iteration: r5 < r3/2 < r1/4 < n/8

Thus, ri will become 0 in at most 2 log n iterations

Example: m = 1989 and n = 1590

Remainder sequence:399 = 1989 mod 1590 399 < 1989/2 393 = 1590 mod 399 393 < 1590/2 6 = 399 mod 393 6 < 399/2 < 1989/43 = 393 mod 6 3 < 393/2 < 1590/40 = 6 mod 3 0 < 6/2 < 399/4 < 1989/8

We stop here, but if we continue: < 1590/8

< 1989/16

< 1590/16

Number of Steps:

In 2 * 3 steps, ri reduced by factor of 8 2 * 4 ri 16 2 * 5 ri 32 2 * 6 ri 64 …… 2 * log n ri n

Thus, ri becomes 0 in at most 2 log n iterations.

4-Correctness of the GCD algorithm.4-Correctness of the GCD algorithm.

• Correctness is an important issue in algorithm design. This means Correctness is an important issue in algorithm design. This means proving that the algorithm works for all legal inputs. proving that the algorithm works for all legal inputs.

It is analogous to proving the correctness of a theorem in It is analogous to proving the correctness of a theorem in mathematics!mathematics!

• The correctness of the GCD algorithm depends on the following loop The correctness of the GCD algorithm depends on the following loop invariant:invariant:

gcd(m,n)=gcd(n,r) where r=m%ngcd(m,n)=gcd(n,r) where r=m%n

Claim 1:

gcd(m,n) = gcd(n,r)

Proof:

The gcd of the new pair is equal to the gcd of the previous pair.

How is this correct?

Let us write:m = q * n + rwhere 0 r n.

This implies that:a common divisor of m and n is alsoa common divisor of n and r, and vice versa. q.e.d.

5-Termination of the GCD algorithm.5-Termination of the GCD algorithm.

• Show that algorithm terminates in a finite number of steps.Show that algorithm terminates in a finite number of steps.• This must be true for every valid input.This must be true for every valid input.• Can we show this for Algorithm GCD?Can we show this for Algorithm GCD?• We must show that We must show that riri goes to 0 in a finite number of steps. goes to 0 in a finite number of steps.

Observe:Observe:

• The sequence of remainders strictly decreases.The sequence of remainders strictly decreases.• They are all non-negative.They are all non-negative.• Thus Thus riri will become 0 in at most 2 log will become 0 in at most 2 log nn steps. steps.

6-Performance of the GCD algorithm.6-Performance of the GCD algorithm.

• Quantification of performance of the algorithm.Quantification of performance of the algorithm.• Crucial parameters: time and space.Crucial parameters: time and space.• Called time and space complexity of the algorithm.Called time and space complexity of the algorithm.• Will be discussed latter in the course.Will be discussed latter in the course.

For example:For example:

Time complexity of GCD: Time complexity of GCD:

Takes at most 2 log Takes at most 2 log nn steps, where steps, where nn < < mmThus, worst-case time complexity: Thus, worst-case time complexity: OO(log (log nn))

7-Data structures7-Data structures

• The study of different ways of organizing data.The study of different ways of organizing data.

• Why?Why?

• Efficiency of algorithm depends on how data is organized.Efficiency of algorithm depends on how data is organized.

• Reason for studying data structures and algorithms together.Reason for studying data structures and algorithms together.

• Organic connection between the two areas.Organic connection between the two areas.

• In 60-212 the programming language JAVA is studied.In 60-212 the programming language JAVA is studied.

• Here: An algorithmic perspective.Here: An algorithmic perspective.

• Independent of programming language (eg. C, C++, JAVA).Independent of programming language (eg. C, C++, JAVA).

For example:

• The median of a list of n numbers is a number m such that:

n/2 numbers in the list are m, and n/2 … are m.

• Many definitions of median, we take:

If n is even:Two medians:

lower median and upper median,Then, median is average of lower and upper medians.

If n is odd: Both medians (lower and upper) are the same.

• Consider this problem:

Given a sorted list of n numbers, find the median.

• A crucial question:

How should we store the list?

• We store it in an array, A, thenThe median is found in constant time, O(1) !!

• Median = (A[5]+A[6])/2 = (19+22)/2 = 20.5

2 5 9 16 19 22 26 27 30 31

1 2 3 4 5 6 7 8 9 10

• Whereas in a linked list:

Traverse half of the list in n/2 steps, which is O(n) !

• Quite simple stated:

The way in which data is organized is crucial in complexity.

More examples like this will be seen later.

2 5 9 16 19 22 ….

first

Example of problems:

• Design an efficient algorithm to determine if a list has repeated elements.

• Given a list of n elements find their minimum (or maximum).

• Given n points in the plane, find the pair(s) of points which are closest to each other.

• Given n points in the plane determine if any three are contained in a straight line.