Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Chapter 1
Introduction
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-1
What is an algorithm?
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
“computer”
problem
algorithm
input output
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-2
Historical Perspective
Euclid’s algorithm for finding the greatest common divisor
Muhammad ibn Musa al-Khwarizmi – 9th century mathematician www.lib.virginia.edu/science/parshall/khwariz.html
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-3
Example of computational problem: sorting
Statement of problem:• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering of the input sequence <a 1, a 2, …, a n> so that a i≤ a j whenever i < j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:• Selection sort• Insertion sort• Merge sort• (many others)
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-4
Selection Sort
Input: array a[1],…,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i=1 to nswap a[i] with smallest of a[i],…a[n]
• see also pseudocode, section 3.1
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-5
Some Well-known Computational Problems
SortingSearchingShortest paths in a graphMinimum spanning treePrimality testingTraveling salesman problemKnapsack problemChessTowers of HanoiProgram termination
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-6
Basic Issues Related to Algorithms
How to design algorithms
How to express algorithms
Proving correctness
Efficiency• Theoretical analysis
• Empirical analysis
Optimality
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-7
Analysis of Algorithms
How good is the algorithm?• Correctness• Time efficiency• Space efficiency
Does there exist a better algorithm?• Lower bounds• Optimality
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-8
What is an algorithm?
Recipe, process, method, technique, procedure, routine,… with following requirements:
1. Finitenessterminates after a finite number of steps
2. Definitenessrigorously and unambiguously specified
3. Inputvalid inputs are clearly specified
4. Outputcan be proved to produce the correct output given a valid input
5. Effectivenesssteps are sufficiently simple and basic
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-9
Why study algorithms?
Theoretical importance
• the core of computer science
Practical importance
• A practitioner’s toolkit of known algorithms
• Framework for designing and analyzing algorithms for new problems
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-10
Euclid’s Algorithm
Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of equalitygcd(m,n) = gcd(n, m mod n)
until the second number becomes 0, which makes the problemtrivial.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-11
Two descriptions of Euclid’s algorithm
Step 1 If n = 0, return m and stop; otherwise go to Step 2Step 2 Divide m by n and assign the value of the remainder to rStep 3 Assign the value of n to m and the value of r to n. Go to
Step 1.
while n ≠ 0 dor ← m mod nm← n n ← r
return m
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-12
Other methods for computing gcd(m,n)
Consecutive integer checking algorithmStep 1 Assign the value of min{m,n} to tStep 2 Divide m by t. If the remainder is 0, go to Step 3;
otherwise, go to Step 4Step 3 Divide n by t. If the remainder is 0, return t and stop;
otherwise, go to Step 4Step 4 Decrease t by 1 and go to Step 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-13
Other methods for gcd(m,n) [cont.]
Middle-school procedureStep 1 Find the prime factorization of mStep 2 Find the prime factorization of nStep 3 Find all the common prime factorsStep 4 Compute the product of all the common prime factors
and return it as gcd(m,n)
Is this an algorithm?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-14
Sieve of EratosthenesInput: Integer n ≥ 2Output: List of primes less than or equal to nfor p ← 2 to n do A[p] ← p
for p ← 2 to ⎣n⎦ doif A[p] ≠ 0 //p hasn’t been previously eliminated from the list
j ← p* pwhile j ≤ n do
A[j] ← 0 //mark element as eliminated
j ← j + p
Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-15
Two main issues related to algorithms
How to design algorithms
How to analyze algorithm efficiency
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-16
Algorithm design techniques/strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Space and time tradeoffs
Greedy approach
Dynamic programming
Iterative improvement
Backtracking
Branch and bound
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-17
Important problem types
sorting
searching
string processing
graph problems
combinatorial problems
geometric problems
numerical problems
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-18
Fundamental data structures
list
• array
• linked list
• string
stack
queue
priority queue
graph
tree
set and dictionary
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-19
Instrumenting an Algorithm
We want to count the number of operations performed when executing an algorithm
We will increment the counter for each check of a condition in an if statement or a while statement and we will increment the counter for each assignment statement or return
We do not include the counting operations themselvescount++ // count the first test to enter the while loopwhile n ≠ 0 do
r ← m mod nm← n n ← rcount ← count + 4 // three assignments and next loop entry
count++ // count the return that appears nextreturn m
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Refining our Instrumentation
1-20
Operation Count Comment Example
Assignment 1 Assign single value or one operation
x = y + z;
Assignment 2 Assign a complex expression
x = y * y + z * z;
Condition 1 Only a single operation number > 0
Condition 2 Complex expression (number > 0) && (number < 10)
Method call 10 For each method call sort(array)
Return 1 or 2 Depends on complexity of expression (see assign)
return number (Note: increment before the instruction)
Each array access
1 or 2 Depends on complexity of subscript (see assign)
Arr[i] * Arr[j] (Note: three total, two accesses and one *)
Each field access
1 Assumes fields are local, not using getter
data. number
Instrumentation itself
0 Never include the instrumentation
Count++; // instrumentation// don’t count it
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-21
Instrumenting a Recursive Algorithm
Here is a recursive solution to the gcd problemgcd(m, n)
if n == 0 return melse return gcd(n, m % n)
Here is an instrumented recursive solution to the gcd problemgcd(m, n)
if n == 0 {count += 2; // one for condition, one for returnreturn m
} else {count += 13; // 1 for cond., 1 for %, 10 for method, 1 for retreturn gcd(n, m % n)
}
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
MicroLab01 and Program01
Both of these activities will involve different versions of implementing Euclid’s algorithm to calculate the GCDMicroLab01 (ML01)• Implement and test GCD01 – iterative version using mod (%)• Implement and test GCD02 – recursive version using mod (%)• Instrument both GCD01 and GCD02 and collect data
Program01 (P01)• Implement and test GCD03 – iterative version using consecutive
integer checking (see slide 12)• Implement and test GCD04 – iterative version using subtraction, as
described on assignment sheet• Instrument both GCD03 and GCD04 and collect data• Write a brief report of your results
1-22