Big O: A Review

Pat MorinCOMP2402/2002

Carleton University

Pat Morin COMP2402/2002 Big O: A Review

Big O: Definition

O(g(n)) = {f (n) : there exists positive constants c and n0such that f (n) ≤ cg(n) for all n ≥ n0}

I Notice: O(g(n)) is a set of functions

I When we say f (n) = O(g(n)) we really mean f (n) ∈ O(g(n))I E.g., n2 + 42n + 7 = O(n2) means:

I The function f (n) = n2 + 42n + 7 is in the set O(n2)

Pat Morin COMP2402/2002 Big O: A Review

Big O: Definition

O(g(n)) = {f (n) : there exists positive constants c and n0such that f (n) ≤ cg(n) for all n ≥ n0}

I Notice: O(g(n)) is a set of functions

I When we say f (n) = O(g(n)) we really mean f (n) ∈ O(g(n))I E.g., n2 + 42n + 7 = O(n2) means:

I The function f (n) = n2 + 42n + 7 is in the set O(n2)

Pat Morin COMP2402/2002 Big O: A Review

Big O: Definition

O(g(n)) = {f (n) : there exists positive constants c and n0such that f (n) ≤ cg(n) for all n ≥ n0}

I Notice: O(g(n)) is a set of functionsI When we say f (n) = O(g(n)) we really mean f (n) ∈ O(g(n))

I E.g., n2 + 42n + 7 = O(n2) means:

I The function f (n) = n2 + 42n + 7 is in the set O(n2)

Pat Morin COMP2402/2002 Big O: A Review

Big O: Definition

O(g(n)) = {f (n) : there exists positive constants c and n0such that f (n) ≤ cg(n) for all n ≥ n0}

I Notice: O(g(n)) is a set of functionsI When we say f (n) = O(g(n)) we really mean f (n) ∈ O(g(n))I E.g., n2 + 42n + 7 = O(n2) means:

I The function f (n) = n2 + 42n + 7 is in the set O(n2)

Pat Morin COMP2402/2002 Big O: A Review

n2 + 42n + 7 = O(n2)

n2 + 42n + 7 ≤ 2n2 for all n ≥ 50

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove n2 + 42n + 7 = O(n2)

I

n2 + 42n + 7 ≤ n2 + 42n2 + 7n2 for n ≥ 1

= 50n2

I So, n2 + 42n + 7 ≤ 50n2 for all n ≥ 1

I n2 + 42n2 + 7n2 = O(n2) [ c = 50, n0 = 1 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove n2 + 42n + 7 = O(n2)

I

n2 + 42n + 7 ≤ n2 + 42n2 + 7n2 for n ≥ 1

= 50n2

I So, n2 + 42n + 7 ≤ 50n2 for all n ≥ 1

I n2 + 42n2 + 7n2 = O(n2) [ c = 50, n0 = 1 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove n2 + 42n + 7 = O(n2)

I

n2 + 42n + 7 ≤ n2 + 42n2 + 7n2 for n ≥ 1

= 50n2

I So, n2 + 42n + 7 ≤ 50n2 for all n ≥ 1

I n2 + 42n2 + 7n2 = O(n2) [ c = 50, n0 = 1 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove n2 + 42n + 7 = O(n2)

I

n2 + 42n + 7 ≤ n2 + 42n2 + 7n2 for n ≥ 1

= 50n2

I So, n2 + 42n + 7 ≤ 50n2 for all n ≥ 1

I n2 + 42n2 + 7n2 = O(n2) [ c = 50, n0 = 1 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove 5n log2 n + 8n − 200 = O(n log2 n)

5n log2 n + 8n − 200 ≤ 5n log2 n + 8n

≤ 5n log2 n + 8n log2 n for n ≥ 2 (log2 n ≥ 1)

≤ 13n log2 n

I 5n log2 n + 8n − 200 ≤ 13n log2 n for all n ≥ 2

I 5n log2 n + 8n − 200 = O(n log2 n) [ c = 13, n0 = 2 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove 5n log2 n + 8n − 200 = O(n log2 n)

5n log2 n + 8n − 200 ≤ 5n log2 n + 8n

≤ 5n log2 n + 8n log2 n for n ≥ 2 (log2 n ≥ 1)

≤ 13n log2 n

I 5n log2 n + 8n − 200 ≤ 13n log2 n for all n ≥ 2

I 5n log2 n + 8n − 200 = O(n log2 n) [ c = 13, n0 = 2 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove 5n log2 n + 8n − 200 = O(n log2 n)

5n log2 n + 8n − 200 ≤ 5n log2 n + 8n

≤ 5n log2 n + 8n log2 n for n ≥ 2 (log2 n ≥ 1)

≤ 13n log2 n

I 5n log2 n + 8n − 200 ≤ 13n log2 n for all n ≥ 2

I 5n log2 n + 8n − 200 = O(n log2 n) [ c = 13, n0 = 2 ]

Pat Morin COMP2402/2002 Big O: A Review

Example

I Prove 5n log2 n + 8n − 200 = O(n log2 n)

5n log2 n + 8n − 200 ≤ 5n log2 n + 8n

≤ 5n log2 n + 8n log2 n for n ≥ 2 (log2 n ≥ 1)

≤ 13n log2 n

I 5n log2 n + 8n − 200 ≤ 13n log2 n for all n ≥ 2

I 5n log2 n + 8n − 200 = O(n log2 n) [ c = 13, n0 = 2 ]

Pat Morin COMP2402/2002 Big O: A Review

Some common relations

I O(nc1) ⊂ O(nc2) for any c1 < c2I For any constants a, b, c > 0,

O(a) ⊂ O(log n) ⊂ O(nb) ⊂ O(cn)

I These make things faster

2 log2 n + 2 = O(log n)

n + 2 = O(n)

2n + 15n1/2 = O(n)

I We can multiply these to learn about other functions,

O(an) = O(n) ⊂ O(n log n) ⊂ O(n1+b) ⊂ O(ncn)

I Examples: O(n1.5) ⊆ O(n1.5 log n)

Pat Morin COMP2402/2002 Big O: A Review

Some common relations

I O(nc1) ⊂ O(nc2) for any c1 < c2I For any constants a, b, c > 0,

O(a) ⊂ O(log n) ⊂ O(nb) ⊂ O(cn)

I These make things faster

2 log2 n + 2 = O(log n)

n + 2 = O(n)

2n + 15n1/2 = O(n)

I We can multiply these to learn about other functions,

O(an) = O(n) ⊂ O(n log n) ⊂ O(n1+b) ⊂ O(ncn)

I Examples: O(n1.5) ⊆ O(n1.5 log n)

Pat Morin COMP2402/2002 Big O: A Review

Some common relations

I O(nc1) ⊂ O(nc2) for any c1 < c2I For any constants a, b, c > 0,

O(a) ⊂ O(log n) ⊂ O(nb) ⊂ O(cn)

I These make things faster

2 log2 n + 2 = O(log n)

n + 2 = O(n)

2n + 15n1/2 = O(n)

I We can multiply these to learn about other functions,

O(an) = O(n) ⊂ O(n log n) ⊂ O(n1+b) ⊂ O(ncn)

I Examples: O(n1.5) ⊆ O(n1.5 log n)

Pat Morin COMP2402/2002 Big O: A Review

Some common relations

I O(nc1) ⊂ O(nc2) for any c1 < c2I For any constants a, b, c > 0,

O(a) ⊂ O(log n) ⊂ O(nb) ⊂ O(cn)

I These make things faster

2 log2 n + 2 = O(log n)

n + 2 = O(n)

2n + 15n1/2 = O(n)

I We can multiply these to learn about other functions,

O(an) = O(n) ⊂ O(n log n) ⊂ O(n1+b) ⊂ O(ncn)

I Examples: O(n1.5) ⊆ O(n1.5 log n)

Pat Morin COMP2402/2002 Big O: A Review

An indulgence

I In this course, we have seen expressions like O(n − i)I Two argument function g(n, i) = n − iI For the purposes of this course, we will take O(g(n, i)) to be

O(g(n, i)) = {f (n, i) : there exists positive constants c and n0such that f (n, i) ≤ cg(n, i) for all n ≥ n0and all valid arguments i}

I For example (Lists) valid values of i are {0, . . . , n − 1} or(sometimes) {0, . . . , n}

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time is

I 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time is

I 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)

I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)

I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)

I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])

I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)

I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e areconstants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review

Why Use big-O Notation?

I Consider the following (simple) code:

for (int i = 0; i < n; i++) {

a[i] = i;

}

I The running time isI 1 assignment (int i = 0)I n+1 comparisons (i < n)I n increments (i++)I n array offset calculations (a[i])I n indirect assignments (a[i] = i)I = a + b(n + 1) + cn + dn + en, where a, b, c , d , and e are

constants that depend on the machine running the code

I Easier just to say O(n) (constant-time) operations

Pat Morin COMP2402/2002 Big O: A Review