Discrete Structures

Lecture 11: AlgorithmsMiss, Yanyan,Ji

United International College

Thanks to Professor Michael Hvidsten

• Objective: Construct and analysis algorithm

• Outline: Algorithm definition and propertiesUsing Pseudocode describe sorting and search

algorithm Algorithm analysis: Big oh, Big Theta, Big Omega

Objective and Outline

Algorithms• Definition: An algorithm is a finite set of precise instructions

for performing a computation or for solving a problem. • Example: Find the largest (maximum) number in a list.• Algorithm: (in Pseudocde) procedure max(a1, a2 , a3 , …, an: integers)

max = a1

for i = 2 to n if max < ai then max = ai

return max

Algorithm Properties• All algorithms share these properties:

– Input: The set of input value(s) for the algorithm– Output: Value(s) that form the solution to the problem– Well-Defined: The steps of the algorithm must be precise– Correctness: An algorithm should produce the correct

input for a given set of input values– Finiteness: An algorithm should produce an output after a

finite number of steps– Effectiveness and Generality: The algorithm should be

applicable for all possible input values.

Algorithms• Example: Find the maximum number in a list. procedure max(a1, a2 , a3 , …, an: integers)

max = a1

for i = 2 to n if max < ai then max = ai

return max• Does this algorithm have all of the properties listed

above?

Solution• Input :a sequence of integers• Output: the largest integer in the sequence• Definiteness: only assignments, a finite loop, and condition

statements occur.• Correctness: initial value of max is the first term of the sequence,

as successive terms of the sequence are examined. max is updated to the value of a term if the term exceeds the maximum of the terms previously examined.

• Finiteness: it terminates after all the integers in the sequence have been examined.

• Effectiveness: the algorithm can be carried out in a finite amount of time since each step is either a comparison or an assignment.

• Generality: it can be used to find the maximum of any finite sequence of integers.

Algorithm Properties• Note: An algorithm can be considered as a function that takes

an input and returns an output.

• Pseudocode: Provides an intermediate step between an English language description and a implementation in a programming language. We want to use instructions resembling those of a programming language. We don’t use any particular language (like C or Java) to describe the steps in an algorithm, since we want to be able to implement the algorithm in every language.

Search Algorithms• General Problem: Locate an element x in a list of

elements a1, a2 , a3 , …, an.

• Example: Locate a word in a dictionary. • Linear Search Algorithm:

Compare x with a1. If x = a1, return 1. Otherwise, compare x with a2. If x = a2, return 2. Continue in this process until we either find x, then the solution will be the location of x or we exhaust the list (in which case we return 0).

Linear Search• Linear Search Algorithm: (Pseudocode) procedure linear search (x: integer, a1,a2,a3,…, an: distinct integers)

i = 1 while (i ≤ n and x ≠ ai )

i = i+1 if i ≤ n then return i else return 0

Binary Search• Binary Search Algorithm:

If the terms in the list are in order of increasing size, a1 ≤ a2 ≤ a3 ≤ … ≤ an we can use the following procedure:

Compare x to the middle term of the list am where m = (n+1)/2 . If x > ⌊ ⌋ am, continue searching in the second half

of the list. Otherwise, continue searching in the first half of the list.

In either case, we now have a new list to search of half the size as the original. Use the same procedure on this new list, on so on.

Binary Search• Binary Search Algorithm:

i = left endpoint of search interval j = right endpoint of search interval Start: i = 1, j = n, x = (term to find)While i<j do the following: Compute middle index of list, m = (i+j)/2 ⌊ ⌋ If x > am, search in second half of list (am+1, am+2 , … aj)

and set i = m+1 Else search in first half of list (ai, ai+1 , … am) and set j = m

Binary SearchExample: Find index of 13 in (1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16)

Start: i=1, j=12, x = 13(1) m = (i+j)/2 =13/2 = 6 and ⌊ ⌋ am = 7. Since x > 7, we set i =

m+1 = 7 and continue searching in (8 10 12 13 15 16). [i=7, j=12]

(2) m = (i+j)/2 = 9 and ⌊ ⌋ am = 12. Since x > 12, we set i = m+1 = 10 and continue searching in (13 15 16). [i=10, j=12]

(3) m = 22/2 = 11 and ⌊ ⌋ am = 15. Since x < 15, set j= m = 11 and continue searching in (13 15). [i=10, j=11]

(4) m = 21/2 =10 and ⌊ ⌋ am = 13. Since x = 13, set j=m=10. We are done, since i=j=10.

Return 10.

Binary Search• Binary Search Algorithm: (Pseudocode) procedure Binary search(x: integer, a1,a2,a3,…, an: increasing integers)

i = 1 (left endpoint of search interval)j = 1 (right endpoint of search interval)while (i < j)

begin

m = (i+j)/2⌊ ⌋ if x > am then i = m+1

else j = m end if x = ai return i

else return 0

public class BinarySearch { public static int search(int x, int[] list) {

int i=0, j = list.length-1, m =0; while (i < j) {

m = (int)Math.floor( ((double)(i+j)) / 2.0); System.out.println("i = " + i + ", j = " + j + ", m = " + m); System.out.println(" x = " + x + ", list[m]= " + list[m]);

if(x > list[m]) i = m+1; else j = m; }

System.out.println("last step: i = " + i + ", j = " + j + ", list[i] = " + list[i]); if(x == list[i]) return i; else return 0; }

public static void main(String arg[]) { int list[] = {1,2,3,5,6,7,8,10,12,13,15,16}; int x = 13;

int m = BinarySearch.search(x,list); System.out.println("index of x in list = " + m);

}}

• Third, we would type “javac BinarySearch.java” to compile our java program. If successful, a file called “BinarySearch.class” is created. This is the bytecode for our program.

• Lastly, to run the program type “java BinarySearch”

Java compiling

Java Compiling

Question

Class exercise: List all the steps used to search for 9 in the sequence

1,3,4,5,6,8,9,11 using(a) a linear search(b) a binary searchAnswer:(a) i=1,i=2,i=3,i=4,i=5,i=6,i=7 So Location=7(b) i=1,j=8,m=4,i=5,j=8,m=6,i=7,j=8,m=7,j=7,location=7

Question

• Input sequence <a1,a2,….,an>• Output sequence<a1’,a2’,…,an’> such that: a1’<=a2’<=a3’…<=an’ or a1’>=a2’>=a3’…>=an’

Sorting

Sorting• Bubble sort is one of the simplest sorting algorithms,

but not one of the most efficient. The smaller elements “bubble” to the top ,the larger elements “sink” to the bottom.

• The Bubble sort algorithm: Procedure bubblesort(a1,…,an :real numbers with

n>=2) for i:=1 to n-1 for j=1 to n-I if aj > aj+1 then interchange aj and aj+1

{a1,…,an is in increasing order}

Sorting

• Use the bubble sort to put 3,2,4,1,5 into increasing order.

• Insertion sort is a simple sorting algorithm, a comparison sort in which the sorted array is built one entry at a time.

• The insertion sort algorithm:Procedure insertion sort(A[1:n]) for j:=2 to n do key=A [j] i=j-1; while i>0 and A [i]>key do A [i+1]:=A [i] i:=i-1 end A [i+1]:=keyEnd {A [n] are sorted}

Sorting

keykeyi

j

• Insertion Example: 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 2 4 8 9 3 6 2 3 4 8 9 6 2 3 4 6 8 9 done

Sorting

• Class Exercise:Sorting the sequence numbers: 6,2,3,1,5,4 using(a) Bubble sort(b) Insertion sort

Question

Algorithm Analysis

• Why need algorithm analysis ? Writing a working program is not good enough The program may be inefficient! If the program is run on a large data set, then the

running time becomes an issue

• Without worry about the hardware and software used to implement an algorithm.

• Assume the different operations used in an algorithm take the same time, which simplifier the analysis.

• Determine whether it is practical to use a particular algorithm to solve a problem as the size of the input increase

• Compare two algorithms to determine which is more efficient as the size of input grows.

The growth of functions

• Big-O notation is used extensively to estimate the number of operations an algorithm as its input grows.

• Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.

• c , n0 > 0 such that f(N) c g(N) when N n0

• g(N) is an upper bound on f(N)• f(x) is big –O of g(x)

Big-O Notation

• F(x)=x2+2x+1 is O(x2)• We observe that we can readily estimate of f(x) when

x>1 because x<x2 and 1<x2 when x>1.• x2<=x2+2x+1<=x2+2x2+x2=4x2

• Consequently, we can take c=4 and n0=1

Big-O Notation

• Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.

• c , n0 > 0 such that f(N) c g(N) when N n0

• g(N) is an lower bound on f(N)• f(x) is big – of g(x)

Big-Omega

• Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers.

• f(N) = (g(N)) iff f(N) = O(g(N)) and f(N) = (g(N))• g(N) is both an upper and a lower bound on the size of f(N)

Big-Theta

• Show that 3x2+8x log x is (x2)

0 8x log x 8 x2, it follows that 3x2+8x log x

11 x2 for x>1. Consequently, 3x2+8x log x is O(X2). Clearly. X2 is O(3x2+8x log x ).So, 3x2+8x log x is (x2)

Big-Theta

Class Exercise:Using the definition of Big-O notation to proof:(1) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)).

Then (f1+f2)(x) is O(max(|g1(x)|,|g2(x)|).

(2) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)). Then (f1f2)(x) is O(g1(x)g2(x)).

Exercise

• Best Case– constraints on the input, other than size, resulting

in the fastest possible running time.• Worst Case

– constraints on the input, other than size, resulting in the slowest possible running time.

• Average Case– average running time over every possible type of

input (usually involve probabilities of different types of input).

Complexity of Algorithm

Three Analyses of Insertion SortingBest Case

A[1] ≤ A[2] ≤ A[3] ≤ ··· ≤ A[n]The number of comparisons needed is equal to

Worst CaseA[1] ≥ A[2] ≥ A[3] ≥ ··· ≥ A[n]The number of comparisons needed is equal to

Average CaseΘ(n2) assuming that each of the n! instances are equally likely.

Complexity of Algorithm

Commonly Used Terminology for the Complexity of AlgorithmComplexity TerminologyΘ(1) Constant ComplexityΘ(log n) Logarithmic ComplexityΘ(n) Linear ComplexityΘ(n log n) N log N complexityΘ(n b) Polynomial ComplexityΘ(b n) Exponential ComplexityΘ(n!) Factorial Complexity