2
Online
• The Concept of an Algorithm
• Algorithm Representation
• Algorithm Discovery
• Search and Sorting
• Recursive Structures
• Efficiency and Correctness
4
Efficiency of Algorithm
• The efficiency is an important issue of good algorithm.– How long does it take to find an entry by using
search algorithm?– Example: university registrar
• 30000 sorted records in the database, search one record by the student identification number
• Sequential search: average 15000 comparisons/search
• Binary search: maximum 15 comparisons/search
5
Analysis of Algorithm
• The purpose of algorithm analysis is to predict the behavior, specially the running time, of an algorithm without implementing it on a specific computer.
• We can compare different algorithms to determine the best one for our purposes.
• We wants a good way to measure speed which is independent of computer and programming language.
6
Time Complexity (1/2)
• The complexity of an algorithm is an estimate of the time taken, as a function of the amount n of information provided.
• There are too many influencing factors for the execution. Thus, we focus on the major steps only.– Example: For sorting algorithms, we count the
number of comparison and ignore the rest.
7
Time Complexity (2/2)
• We are interesting in the worst case, i.e. the upper bound of the running time.– We chose a special input to make the running time
is largest than those of other inputs. This is called as the worst case.
• Analysis consideration– Best case analysis– Worst case analysis– Average case analysis
8
Space Complexity
• The space complexity indicates the amount of temporary storage required for running the algorithm.
• The storage required for the input or for the output is ignore.
9
Steps in a for-loop
• The input size is n, then the number of this algorithm is n2+n.
• We only care the number of print (i,j) is n2.
for (j=0; j<n; j++) {
for (i=0; i<m; i++)
cout << i << “ and ” << j << “\n”;
cout << “***********\n”;
}
1 step for
n2 times1 step for n times
10
Steps in Sequential Search
• If List has n entries, then the number of this algorithm is 3n+4.
2 steps
1 step
1 step
3 steps for n times
1 step
11
Methodology of Complexity
• We need an methodology to denote the time (or space) complexity.
• The Big O notation is used to approximate the exact running time.– Example: sequential search has time
complexity O(n), where n is the size of list.– It means that “n steps is enough to find the
target among n items”.
The Big O Notation
•g(n) has the complexity of f(n).“g(n) has order of magnitude f(n)”
g(n) = O(f(n))
g(n) < c × f(n) for all large enough n
f(n) is an upper bound of g(n).– Example:
g(n)=5n2 + 10n + 25 = O(f(n)=n2) = O(n2)
•Define: g(n)=O(f(n)) iff c,N: constant g(n) cf(n), nN.
Why Approximate ?
•We want to ignore startup overheads.– 5n2+15 the running time is approximately n2
– If something takes 5n+15 steps, the overhead of 15 steps becomes insignificant for large n.
•We want to ignore constant factors. – 10n the running time is approximately n– Something which takes 10n steps on one computer might
only take 5n steps on another computer.
•We want to ignore the smaller one.– If something takes 2n2 + 5n steps, we ignore the 5n part,
because it is swamped by the 2n2 part for large n.
14
Approximate Approach
• Only concentrate on the size of input (n).
• We assume that n is large and ignore the redundancy.
• The complexity of an algorithm can often be estimated directly, rather than counting the number of steps and then simplifying.
• The analysis is only an approximation.
15
Example: g(n)=3n+4
• Find f(n)=n, c=4, N=4
g(n)=3n+4 4n, n4.
g(n)=O(f(n))=O(n)• The number of major steps in the sequential
search is bound by g(n)=3n+4.• The time complexity of the sequential search
is O(n). That is, we only care the number of comparisons in n.
16
Example: g(n)=5n2+15
• Let f(n)=n2, c=6, N=4
g(n)=5n2+15 6n2, n4.
g(n)=O(n2)
• Let f(n)=n3, c=1, N=6
g(n)=5n2+15 n3, n6.
g(n)=O(n3)
• Usually, we interested in the upper bound which is the closest to g(n).
17
Lemma
• f(n)=O(s(n)), g(n)=O(r(n))
f(n)+g(n)=O(s(n)+r(n))
• f(n)=O(s(n)), g(n)=O(r(n))
f(n)•g(n)=O(s(n)•r(n))
18
Example of Comparison
• Which algorithm is faster?– O(n) v.s. O(n1.8) v.s. O(n2)
• The numbers of three algorithms are 100n, 100n1.8 and 2n2+50, respectively.– n=5, 100n=500 > 2n2+50=100
However, n50, 100n < 2n2+50– n=1, 100n1.8=100 > 2n2+50=52
However, n3*108, 100n1.8 < 2n2+50
19
Notes of O Notation
• The approximate is valid if n is large enough.
• We ignore constant factors and concentrate on the behavior of the algorithm as the size of input goes to infinity.
• We say that the time complexity of an algorithm, or the the running time, is O(f(n)), if the number of major steps is bound by f(n).
20
Running Times (in Seconds)
• n=1000, the machine speed 1000steps/sec.
Running time Execution timelog2n 0.010
n log2n 1
n1.5 10
n2 32
n3 106
1.1n 1039
21
The and Notation
• Define: g(n)=Ω(f(n)) iff c,N: constant g(n) cf(n) nN.– f(n) is a lower bound of g(n).
• Define: g(n)=(f(n)) iff g(n)=O(f(n)) and g(n)=Ω(f(n)).– f(n) is a lower bound and an upper bound of
g(n).
24
Figure 4.18Applying the insertion sort in a worst-case
situation O(n2)
• The number of comparisons is 1+2+3+...+(n-1)=n(n-1)/2.
26
Software Verification
• Any software needs verification.– Example: Gold chain with minimum cut
• We want a formal techniques to prove the correctness of a program.
29
Hardware Verification
• The verification of circuit deigns and machine construction are important.– Mark I contained wiring errors.– The floating-point portion of the early Pentium
microprocessors.