View
1
Download
0
Category
Preview:
Citation preview
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Adam Smith
Algorithm Design and Analysis
LECTURE 3 • Asymptotic Notation • Basic Data Structures
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Asymptotic notation
We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.
O-notation (upper bounds):
EXAMPLE: 2n2 = O(n3)
functions, not values
(c = 1, n0 = 2) funny, “one-way” equality
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Set Definition
O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }
EXAMPLE: 2n2 ∈ O(n3) (Logicians: λn.2n2 ∈ O(λn.n3), but it’s convenient to be sloppy, as long as we understand what’s really going on.)
Examples
• 106 n3+ 2n2 -n +10 = O(n3)
• n½ + log(n) = O(n½)
• n (log(n) + n½) = O(n3/2)
• n = O(n2)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Ω-notation (lower bounds)
Ω(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
EXAMPLE: (c = 1, n0 = 16)
O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Ω-notation (lower bounds) • Be careful: “Any comparison-based sorting algorithm requires at least O(n log n) comparisons.”
– Meaningless! – Use Ω for lower bounds.
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Θ-notation (tight bounds)
Θ(g(n)) = O(g(n)) ∩ Ω(g(n))
EXAMPLE:
Polynomials are simple: ad nd + ad–1nd–1 + + a1n + a0 = Θ(nd)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
o-notation and ω-notation
o(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 }
EXAMPLE: (n0 = 2/c)
O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.
2n2 = o(n3)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
o-notation and ω-notation
ω(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 }
EXAMPLE: (n0 = 1+1/c)
O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Common Functions: Asymptotic Bounds
• Polynomials. a0 + a1n + … + adnd is Θ(nd) if ad > 0. • Polynomial time. Running time is O(nd) for some constant d independent of the input size n. • Logarithms. log a n = Θ(log b n) for all constants a, b > 0.
For every x > 0, log n = O(nx).
• Exponentials. For all r >1 and all d > 0, nd = O(rn). • Factorial.
grows faster than every exponential
can avoid specifying the base log grows slower than every polynomial
Every exponential grows faster than every polynomial
n! = (!
2!n)!n
e
"n(1 + o(1)) = 2!(n log n)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Sort by asymptotic order of growth
a) n log(n) b) c) log(n) d) n2 e) 2n f) n g) n! h) n1,000,000
i) n1/log(n) j) log(n!)
1. n1/log(n) 2. log(n) 3. 4. n 5. n log(n) = Θ(log(n!)) 6. 7. n2 8. n1,000,000
9. 2n 10. n!
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Conventions for formulas
Convention: A set in a formula represents an anonymous function in the set.
f(n) = n3 + O(n2) means f(n) = n3 + h(n) for some h(n) ∈ O(n2) .
EXAMPLE: (right-hand side)
9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson
Convention for formulas
Convention: A set in a formula represents an anonymous function in the set.
n2 + O(n) = O(n2) means for any f(n) ∈ O(n): n2 + f(n) = h(n) for some h(n) ∈ O(n2) .
EXAMPLE: (left-hand side)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Properties
• Transitivity. – If f = O(g) and g = O(h) then f = O(h). – If f = Ω(g) and g = Ω(h) then f = Ω(h). – If f = Θ(g) and g = Θ(h) then f = Θ(h).
• Additivity. – If f = O(h) and g = O(h) then f + g = O(h). – If f = Ω(h) and g = Ω(h) then f + g = Ω(h). – If f = Θ(h) and g = O(h) then f + g = Θ(h).
Exercise: Show that log(n!) = Θ(n log n)
• Stirling’s formula:
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
n! = (!
2!n)!n
e
"n(1 + o(1))
log(n!) = log(!
2!n) + n log(n)" n log(e) + log(1 + o(1))! "# $=log(1)+o(1)
since log is continuous
= n%
log(n)" log(e) +log(2!n)
n! "# $constant+o(1)
&+ o(1)
= n(log(n)"O(1)) + o(1)= n log n(1"O( 1
log n )) + o(1)
= n log n(1± o(1)) = !(n log n)
8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne
Overview Notation … means … Think… E.g. Lim f(n)/g(n)
f(n)=O(n) 9 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)
Upper bound
100n2 = O(n3)
If it exists, it is < 1
f(n)=Ω(g(n)) 9 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)
Lower bound
n100 = Ω(2n)
If it exists, it is > 0
f(n)=Θ(g(n)) both of the above: f=Ω(g) and f = O(g)
Tight bound log(n!) = Θ(n log n)
If it exists, it is > 0 and < 1
f(n)=o(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)
Strict upper bound
n2 = o(2n) Limit exists, =0
f(n)=ω(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)
Strict lower bound
n2 = ω(log n)
Limit exists, =1
Recommended