COMPSCI 720S1C, 2006 - Aucklandmcw/Teaching/720/lectures.pdf · Outline Background Introduction Generating Functions Generating functions and recurrences Combinatorial and Algorithmic

COMPSCI 720S1C, 2006

Mark C. Wilson

May 16, 2007

Outline Background Introduction Generating Functions Generating functions and recurrences Combinatorial and Algorithmic Applications Randomized algorithms

1 BackgroundProbabilityAlgorithm Analysis

2 IntroductionFirst example - quicksort

3 Generating FunctionsGF definitionsExtra details on power seriesOther types of generating functions

4 Generating functions and recurrencesExtra details on recurrencesExtras on the symbolic methodCoefficient extraction from GFs

5 Combinatorial and Algorithmic ApplicationsTreesStringsTries

6 Randomized algorithmsMonte Carlo AlgorithmsLas Vegas Algorithms


Probability

Probability basics I

A probability space is a set X with a probability measure Prdefined on a σ-algebra S.

A σ-algebra is a collection of subsets of X that contains ∅and is closed under complement, unions.

A probability measure is a function Pr : S → [0, 1] such thatPr(∅) = 0; Pr(∪iAi) =

∑i PrAi if the Ai are pairwise

disjoint.

For us, usually X is finite of size n and S contains all the 2n

subsets of X. The space is discrete.

A random variable is a (measurable, real-valued) function onX. For us, the “measurable” can safely be ignored.


Probability

Probability basics II

The mean of a discrete random variable T isµ := E[T ] :=

∑x∈X T (x) Pr(x) =

∑y y Pr(T (x) = y).

The variance of T is σ2 := E[T 2]− E[T ]2.Chebyshev’s inequality says that Pr(|T − µ| > cσ) < c−2 foreach c > 0. In practice this probability is often exponentiallysmall in c.

The measure is uniform if the probability of a subset dependsonly on its size. The probability of a k-element subset of ann-element space is then k/n. In this case all the abovequantities can be computed via counting.


Algorithm Analysis

Recalling ideas from previous courses

We aim to compare resource use of algorithms for a givencomputational problem. A mathematical model is needed.

We measure asymptotic running time for large inputs. Smallinputs are not a challenge. We don’t care about constantfactor speedups due to faster machine, better programming,etc.

We identify elementary operations and measure running timein terms of these (e.g. comparisons, swaps for sorting).

Every comparison-based sorting method must use Ω(n log n)comparisons in the worst case for a file of size n.


Algorithm Analysis

Asymptotic notation

Let f, g : N → N. We say that f ∈ O(g) if there is C > 0such that f(n) ≤ Cg(n) for all sufficiently large n.

f ∈ Ω(g) ⇔ g ∈ O(f).f ∈ Θ(g) ⇔ f ∈ O(g) and f ∈ Ω(g).In particular if limn→∞ f(n)/g(n) exists and equals L, then

f ∈ Θ(g) if 0 < L < ∞;f ∈ Ω(f) if L = ∞;f ∈ O(g) if L = 0.


Algorithm Analysis

Our basic framework

Let A be an algorithm for a given problem, and let I be theset of legal inputs. The size of input ι ∈ I is denoted |ι|; welet In be the set of inputs of size n. The running time of Aon input ι is the number of elementary operations T (ι).Worst-case analysis studies W (n) := maxι∈In T (ι). Example:for sorting integers, In is the set of all sequences of integersof length n. For quicksort, W (n) ∈ Θ(n2) but for mergesort,W (n) ∈ Θ(n log n).Here we prefer to study the distribution of T (ι) over In, sincethis is often more relevant in practice. This approach requiressome probability model on the inputs, and then T is a randomvariable, whose mean, variance, etc, can be studied.

We concentrate on systematic and powerful mathematicaltools, avoiding special tricks.


Organizational matters

Lecturer: Dr Mark Wilson; office 303.588;www.cs.auckland.ac.nz/˜mcw. Office hours by appointmentand whenever my door is open - email is preferred.

Recommended reading:

Flajolet and Sedgewick, An Introduction to the Analysis ofAlgorithms (on reserve in library);Wilf, generatingfunctionology (in library, also freely availableonline from my webpage);Graham, Knuth, Patashnik, Concrete Mathematics (onreserve).Lecture slides available online, but are continually beingupdated and corrected, so don’t rely on them until the end ofthe course.

One assignment worth 20% of course marks will be given.


First example - quicksort

Quicksort algorithm

A recursive algorithm for sorting an array of n elements froma totally ordered set.

If n = 0 or n = 1, do nothing. Otherwise, choose a pivotelement x and partition the array so that if y < x then y is tothe left of x and if z > x then z is to the right of x. Then callthe algorithm recursively on the left and right parts.

Note that we have not specified what happens if x = y, butwe must do so in any implementation.

The partitioning step requires at least n− 1 comparisons, plussome swaps.



Quicksort recurrence

Let C(F ) be the number of comparisons done by quicksort onan n-element file F of distinct elements. Let F1, F2 be thesubfiles consisting of elements less than, greater than thepivot. Then

C(F ) =

0 if n = 0;

n− 1 + C(F1) + C(F2) if n > 0.

If the pivot is always the smallest element then F1 is empty,so iterating this recursion gives W (n) ∈ Θ(n2). Quicksort hasa bad worst case.Choose an input file (permutation of 1, . . . , n) uniformly atrandom. After pivoting, the pivot is equally likely to be in anyof the n positions, and each subfile is uniformly distributed, so

E[Cn] = n− 1 +1n

n∑p=1

(E[Cp−1] + E[Cn−p]).





C(F ) =

0 if n = 0;

n− 1 + C(F1) + C(F2) if n > 0.

If the pivot is always the smallest element then F1 is empty,so iterating this recursion gives W (n) ∈ Θ(n2). Quicksort hasa bad worst case.

Choose an input file (permutation of 1, . . . , n) uniformly atrandom. After pivoting, the pivot is equally likely to be in anyof the n positions, and each subfile is uniformly distributed, so

E[Cn] = n− 1 +1n

n∑p=1

(E[Cp−1] + E[Cn−p]).





C(F ) =

0 if n = 0;

n− 1 + C(F1) + C(F2) if n > 0.

If the pivot is always the smallest element then F1 is empty,so iterating this recursion gives W (n) ∈ Θ(n2). Quicksort hasa bad worst case.Choose an input file (permutation of 1, . . . , n) uniformly atrandom. After pivoting, the pivot is equally likely to be in anyof the n positions, and each subfile is uniformly distributed, so

E[Cn] = n− 1 +1n

n∑p=1

(E[Cp−1] + E[Cn−p]).



Quicksort recurrence solution

Let an = E[Cn] so thatan = n− 1 + 1

n

∑1≤p≤n(ap−1 + an−p). Collect common

terms in the sums to obtain

an = n− 1 +2n

n−1∑j=0

aj .

This full history recurrence can be solved by eliminating thehistory, (a general form of telescoping).

The solution is an = 2(n + 1)Hn − 4n ≈ 2n log n, where

Hn :=∑

1≤j≤n

1/j, the nth harmonic number.

Quicksort is of optimal order on average.

What about the variance?





n



an = n− 1 +2n

n−1∑j=0

aj .



Hn :=∑

1≤j≤n








n



an = n− 1 +2n

n−1∑j=0

aj .



Hn :=∑

1≤j≤n








n



an = n− 1 +2n

n−1∑j=0

aj .



Hn :=∑

1≤j≤n








n



an = n− 1 +2n

n−1∑j=0

aj .



Hn :=∑

1≤j≤n






Quicksort recurrence solution details

We have for n > 1

nan − (n− 1)an−1 = n(n− 1) + 2∑

0≤j<n

aj

− (n− 1)(n− 2)− 2∑

0≤j<n−1

aj

= 2(n− 1) + 2an−1

Thus nan = 2(n− 1) + (n + 1)an−1, so that

an

n + 1=

2(n− 1)n(n + 1)

+an−1

n=

an−1

n+ 2(n− 1).

We can iterate to obtain

an

n + 1=

a1

2+

n∑j=2

2j + 1

+1j− 1

j + 1.




We have for n > 1

nan − (n− 1)an−1 = n(n− 1) + 2∑

0≤j<n

aj

− (n− 1)(n− 2)− 2∑

0≤j<n−1

aj

= 2(n− 1) + 2an−1


an

n + 1=

2(n− 1)n(n + 1)

+an−1

n=

an−1

n+ 2(n− 1).


an

n + 1=

a1

2+

n∑j=2

2j + 1

+1j− 1

j + 1.




We have for n > 1

nan − (n− 1)an−1 = n(n− 1) + 2∑

0≤j<n

aj

− (n− 1)(n− 2)− 2∑

0≤j<n−1

aj

= 2(n− 1) + 2an−1


an

n + 1=

2(n− 1)n(n + 1)

+an−1

n=

an−1

n+ 2(n− 1).


an

n + 1=

a1

2+

n∑j=2

2j + 1

+1j− 1

j + 1.



Issues arising from our analysis

How do we analyse suggested improvements, such as using acutoff for small files, or median-of-3 pivoting, or differentpartitioning method?

The method we used for solving the recurrences wassomewhat specialized. How to do it more generally?

How do we get more information about the full distribution ofthe number of comparisons?

How to derive asymptotics for the sums occurring?

The above analysis relies on the fact that the subfiles arethemselves uniformly distributed. What to do if this is not thecase?

How to analyse input where keys can be equal?

We answer points 1–4 in this course.




















































GF definitions

Ordinary generating functions — OGFs

The OGF associated to a sequence a0, a1, . . . , is the formalpower series F (z) =

∑n anzn. Examples: 1, 1, 1, . . . , has

OGF 1 + z + z2 + · · · = 1/(1− z). See handout for moreexamples.

Basic operations on sequences (sum, convolution, . . . )correspond to those on OGFs (sum, product, . . . ). Seehandout for more operations.

The equality∑

n zn = 1/(1− z) is purely formal at this stagebut also makes sense for |z| < 1. So far OGFs are just aconvenient short way to describe sequences, but soon they willbe a powerful machine.

Given an OGF, how to extract its sequence if not availablefrom above list? Taylor series definition always works, but isusually not computationally useful. We mainly use tablelookup here.


GF definitions






The equality∑




GF definitions






The equality∑




GF definitions






The equality∑




GF definitions

Summary - power series

Basic principle: if an identity between power series is true atthe level of analytic functions, then it is true for formal series,provided all operations concerned are formally valid(computation of each coefficient can be carried out in B).

Thus we may freely use all our algebra and calculusknowledge, with appropriate caution about composition.

An example of what is not allowed formally:∑n(z + 1)n/n! = e

∑n zn/n!. This is true at the function

level for every z ∈ C and says exp(z + 1) = exp(1) exp(z),but the constant term of the left side can’t be computed in B.


GF definitions








GF definitions








Extra details on power series

Formal power series

A sequence is a function f : N → C. We sometimes write(f(0), f(1), . . . , ) instead of just f . The special sequences1 := (1, 0, 0, . . . , ) and z := (0, 1, 0, 0 . . . ) will be useful.

The set of all sequences we call A. We define operations +and · on A as follows.

The sum f + g of sequences is the sequence h such thath(n) = f(n) + g(n) for each n ∈ N.The product f · g is the sequence h such that for each n,h(n) =

∑nk=0 f(k)g(n− k).

With these operations and the obvious multiplication bycomplex numbers, A becomes a commutative associativealgebra. It contains a subalgebra B (consisting of allsequences with finitely many nonzero terms) isomorphic to thealgebra of polynomials in one indeterminate.



Formal power series




∑nk=0 f(k)g(n− k).




Formal power series



The sum f + g of sequences is the sequence h such thath(n) = f(n) + g(n) for each n ∈ N.

The product f · g is the sequence h such that for each n,h(n) =

∑nk=0 f(k)g(n− k).




Formal power series




∑nk=0 f(k)g(n− k).




Operations in the power series algebra

We can define the derivative of f as the sequence h such thath(n) = (n + 1)f(n + 1).

The antiderivative of f is the sequence h such thath(n) = f(n− 1)/n for n > 0, and h(0) = 0.

The composition of f and g is the sequence h given byh(n) =

∑nk=0 f(k)gk. This is only valid when g(0) = 0.

The inverse of f is the series h such that f · h = 1; this existsif and only if f(0) 6= 0.

All operations take place in B; that is, computing h(n) alwaysinvolves only finite algebra.




We can define the derivative of f as the sequence h such thath(n) = (n + 1)f(n + 1).The antiderivative of f is the sequence h such thath(n) = f(n− 1)/n for n > 0, and h(0) = 0.























Analytic functions and power series

Some facts from a course in complex analysis (not proved here):

For each f ∈ A there is a largest R ≤ ∞, such that for|z| < R, the infinite series

∑n f(n)zn converges absolutely

and uniformly to a limit F (z).

If R > 0, then F is analytic (has derivatives of all orders) for|z| < R, and can be integrated and differentiatedterm-by-term.Conversely, if R > 0 and F is an analytic function for |z| < R,then there is a unique sequence f so that F (z) =

∑n f(n)zn.

Here f is essentially the sequence of Taylor coefficients of Fat 0,

f(n) =1n!

(d

dz

)n

F (z)|z=0

and can also be computed by Cauchy’s integral formula

f(n) =1

2πi

∫C

F (z)zn+1

dz.







and uniformly to a limit F (z).If R > 0, then F is analytic (has derivatives of all orders) for|z| < R, and can be integrated and differentiatedterm-by-term.

Conversely, if R > 0 and F is an analytic function for |z| < R,then there is a unique sequence f so that F (z) =

∑n f(n)zn.


f(n) =1n!

(d

dz

)n

F (z)|z=0


f(n) =1

2πi

∫C

F (z)zn+1

dz.







and uniformly to a limit F (z).If R > 0, then F is analytic (has derivatives of all orders) for|z| < R, and can be integrated and differentiatedterm-by-term.Conversely, if R > 0 and F is an analytic function for |z| < R,then there is a unique sequence f so that F (z) =

∑n f(n)zn.


f(n) =1n!

(d

dz

)n

F (z)|z=0


f(n) =1

2πi

∫C

F (z)zn+1

dz.







and uniformly to a limit F (z).If R > 0, then F is analytic (has derivatives of all orders) for|z| < R, and can be integrated and differentiatedterm-by-term.Conversely, if R > 0 and F is an analytic function for |z| < R,then there is a unique sequence f so that F (z) =

∑n f(n)zn.


f(n) =1n!

(d

dz

)n

F (z)|z=0


f(n) =1

2πi

∫C

F (z)zn+1

dz.


Other types of generating functions

Exponential generating functions — EGFs

The EGF associated to a sequence a0, a1, . . . , is the formalpower series F (z) =

∑n anzn/n!. Examples: 1, 1, 1, . . . , has

EGF 1 + z + z2/2! + · · · = exp(z). See handout.

The EGF of sequence an is the OGF of an/n!.EGFs are often used for labelled constructions and forpermutations (perhaps more later).

Basic operations on sequences (sum, convolution, . . . )correspond to those on EGFs (sum, product, . . . ). Seehandout.

Sometimes OGFs are better for computation, sometimesEGFs.







The EGF of sequence an is the OGF of an/n!.

EGFs are often used for labelled constructions and forpermutations (perhaps more later).
































Probability generating functions — PGFs

The PGF associated to a random variable X taking values inN is G(z) =

∑n Pr(X = n)zn.

The mean of X is just G′(1) and the variance isG′′(1) + G′(1)−G′(1)2.The PGF of the sum of independent RVs X and Y is theproduct of the individual PGFs.

The PGF of the sequence Pr(X > n) of tail probabilities is(1−G(z))/(1− z).





∑n Pr(X = n)zn.

The mean of X is just G′(1) and the variance isG′′(1) + G′(1)−G′(1)2.

The PGF of the sum of independent RVs X and Y is theproduct of the individual PGFs.






∑n Pr(X = n)zn.







∑n Pr(X = n)zn.





Multivariate generating functions — MGFs

Occur naturally in many contexts. An important one for probabilityis as follows. Let ank be the number of objects of some type withsize n and another parameter χ equal to k. LetF (z, u) =

∑ankz

nuk, the bivariate GF. Then

cn := [zn]F (z, 1) =∑

k ank is the number of objects of size n;

µn := (1/cn)[zn]Fu(z, 1) = (1/cn)∑

k kank is the mean of χon a uniformly chosen object of size n;

σ2n := (1/cn)[zn]Fuu(z, 1) + µn − µ2

n is the variance of χ on auniformly chosen object of size n.


GFs solve recurrences

Main idea: recurrence gives an equation involving the GF. Tryto solve this, then extract the coefficients.

Example (Fibonacci): a0 = 0, a1 = 1; an = an−1 + an−2 forn ≥ 2. Multiply each side by zn, then sum on n. LetF (z) =

∑n anzn. Then we get F (z)− z = zF (z) + z2F (z).

This gives F (z) = z/(1− z − z2).To convert back to a sequence, we can use partial fractions.Get F (z) = A/(1− φz) + B/(1 + φ−1z) where φ = 1.618 · · · ,the golden ratio, A = 1/

√5, B = −1/

√5.

Thus an = (φn + (−1)nφ−n)/√

5 ∈ Θ(φn).Note: this can be done automatically by a computer algebrasystem!






This gives F (z) = z/(1− z − z2).

To convert back to a sequence, we can use partial fractions.Get F (z) = A/(1− φz) + B/(1 + φ−1z) where φ = 1.618 · · · ,the golden ratio, A = 1/

√5, B = −1/

√5.









√5, B = −1/

√5.









√5, B = −1/

√5.


5 ∈ Θ(φn).

Note: this can be done automatically by a computer algebrasystem!







√5, B = −1/

√5.




Quicksort recurrence with GFs

From nan = n(n− 1) + 2∑

j<n aj , a0 = 0, we obtain

zF ′(z) = 2z2/(1− z)3 + 2zF (z)/(1− z) via the dictionary.

This is a standard first order linear inhomogeneous differentialequation that can be solved (how?) to obtain

F (z) =2

(1− z)2

(log

11− z

− z

).

Thus by lookup we have an = 2(n + 1)Hn − 4n.


Quicksort recurrence with GFs

From nan = n(n− 1) + 2∑

j<n aj , a0 = 0, we obtain

zF ′(z) = 2z2/(1− z)3 + 2zF (z)/(1− z) via the dictionary.

This is a standard first order linear inhomogeneous differentialequation that can be solved (how?) to obtain

F (z) =2

(1− z)2

(log

11− z

− z

).

Thus by lookup we have an = 2(n + 1)Hn − 4n.


GFs also yield recurrences

Example: for fixed r ≥ 1, consider

F (z) = (1− z)/(1− 2z + zr+1) =∑

n

anzn.

Thus (∑

n anzn)(1− 2z + zr+1) = 1− z. Comparecoefficients to see that a0 = 1, a1 − 2a0 = −1, and

an − 2an−1 + an−r−1 = 0 for n ≥ 2.

This allows linear time computation of an.

Every rational OGF gives a linear constant coefficientrecurrence in this way, and vice versa.

In the same way, linear recurrences with polynomialcoefficients correspond to linear differential equations for theOGF.




F (z) = (1− z)/(1− 2z + zr+1) =∑

n

anzn.

Thus (∑


an − 2an−1 + an−r−1 = 0 for n ≥ 2.







F (z) = (1− z)/(1− 2z + zr+1) =∑

n

anzn.

Thus (∑


an − 2an−1 + an−r−1 = 0 for n ≥ 2.





Extra details on recurrences

GFs sometimes yield better recurrences

The counting GF of binary trees by internal nodes satisfiesT (z) = 1 + zT (z)2 (later). This is equivalent to the quadraticrecurrence a0 = 1, an =

∑k<n akan−1−k for the number an

of binary trees with n nodes.

There is an algorithm (Comtet 1964) which finds a lineardifferential equation with polynomial coefficients for eachalgebraic GF.In this case the answer turns out to be

(4z2 − z)T ′(z) + (2z − 1)T (z) + 1 = 0

which is equivalent to the recurrence

(n + 1)an = (4n− 2)an−1 a0 = 1.

This allows for much faster computation and makes it plainthat an involves a quotient of factorials ( in factan = 1

n+1

(2nn

), the nth Catalan number).






of binary trees with n nodes.There is an algorithm (Comtet 1964) which finds a lineardifferential equation with polynomial coefficients for eachalgebraic GF.

In this case the answer turns out to be

(4z2 − z)T ′(z) + (2z − 1)T (z) + 1 = 0


(n + 1)an = (4n− 2)an−1 a0 = 1.


n+1

(2nn







of binary trees with n nodes.There is an algorithm (Comtet 1964) which finds a lineardifferential equation with polynomial coefficients for eachalgebraic GF.In this case the answer turns out to be

(4z2 − z)T ′(z) + (2z − 1)T (z) + 1 = 0


(n + 1)an = (4n− 2)an−1 a0 = 1.


n+1

(2nn




Comtet’s algorithm outline

Suppose P (z, F (z)) = 0 where P (z, y) ∈ C[z, y] isirreducible. Differentiate and solve for F ′ to obtain

F ′(z) =A(z, y)B(z, y)

for relatively prime polynomials A,B ∈ C[z, y].

Note that B and P are relatively prime.

The extended Euclidean algorithm yields polynomialsu(z, y), v(z, y), g(z) such that uB + vP = g. Define C = Aumod P to get F ′(z) = C(y, z)/g(z). Repeat as required.

In above binary tree example,P = zy2 − y + 1, B = 1− 2zy, A = y2, u =(2zy−1)/(4z−1), C = (2zy−y−z)/(z−4z2), g = 1/(4z−1).Algorithm terminates in one step.





F ′(z) =A(z, y)B(z, y)









F ′(z) =A(z, y)B(z, y)







OGFs and counting I

Can often set up a recursion and solve as above. Example:how many binary trees Tn with n (internal) nodes? Binarytree is a recursive object: either a single external node, or aninternal node connected to an ordered pair of binary trees.Thus T0 = 1 and for n > 0,

Tn =∑

1≤k≤n

Tk−1Tn−k =∑

0≤k≤n−1

TkTn−1−k.

Hence OGF satisfies T (z) = zT (z)2 + 1. ThusT (z) = (1−

√1− 4z)/2z (how to choose square root?).

Can extract coefficients using binomial theorem: get

Tn =1

n + 1

(2n

n

)(Catalan number).

Asymptotics can be derived (later) and we will getTn ∼ 4n/

√πn3.



OGFs and counting I


Tn =∑

1≤k≤n

Tk−1Tn−k =∑

0≤k≤n−1

TkTn−1−k.




Tn =1

n + 1

(2n

n

)(Catalan number).


√πn3.



OGFs and counting I


Tn =∑

1≤k≤n

Tk−1Tn−k =∑

0≤k≤n−1

TkTn−1−k.




Tn =1

n + 1

(2n

n

)(Catalan number).


√πn3.



OGFs and counting I


Tn =∑

1≤k≤n

Tk−1Tn−k =∑

0≤k≤n−1

TkTn−1−k.




Tn =1

n + 1

(2n

n

)(Catalan number).


√πn3.



OGFs and counting II

A nicer way to get the OGF is as follows. Let T be the set ofall binary trees, |t| the size of tree t, Tn = t ∈ T : |t| = n.Then

T (z) =∑

n

∑t∈Tn

zn =∑t∈T

z|t| = 1 +∑

t∈T \T0

z|tl|+|tr|+1

= 1 +∑tl∈T

∑tr∈T

z|tl|+|tr|+1 = 1 +∑tl∈T

z|tl|∑tr∈T

z|tr|

= 1 + zT (z)2

where tr, tl are the right and left subtrees of t.

Study this example carefully - it leads to the symbolic methodfor enumeration.



OGFs and counting II

A nicer way to get the OGF is as follows. Let T be the set ofall binary trees, |t| the size of tree t, Tn = t ∈ T : |t| = n.Then

T (z) =∑

n

∑t∈Tn

zn =∑t∈T

z|t| = 1 +∑

t∈T \T0

z|tl|+|tr|+1

= 1 +∑tl∈T

∑tr∈T

z|tl|+|tr|+1 = 1 +∑tl∈T

z|tl|∑tr∈T

z|tr|

= 1 + zT (z)2

where tr, tl are the right and left subtrees of t.

Study this example carefully - it leads to the symbolic methodfor enumeration.



The symbolic method I

Avoids explicit recurrences, goes straight from recursivedescription of structure to the counting GF. A greattime-saver, and also more amenable to computer algebraimplementation.

A combinatorial class is a set X which is the disjoint union offinite sets Xn. The size |x| of an element x is the value of nfor which x ∈ Xn.

Let A be a combinatorial class, with |An| = an. The countingOGF for A is A(z) =

∑n anzn =

∑a∈A z|a|.

Set operations such as disjoint union, cartesian product,sequence correspond to GF operations sum, product,quasi-inverse.

Main examples: plane trees, compositions, regular languages.







∑n anzn =

∑a∈A z|a|.









∑n anzn =

∑a∈A z|a|.









∑n anzn =

∑a∈A z|a|.









∑n anzn =

∑a∈A z|a|.





The symbolic method II

Let A,B be combinatorial classes with counting OGFsA(z), B(z). The size of an ordered pair of objects (α, β) isdefined to be |α|+ |β|.

The counting OGF of A× B is then∑γ∈A×B

z|γ| =∑α∈A

∑β∈B

z|α|+|β| = A(z)B(z).

Also the counting OGF for A∪B is A(z) + B(z) if the classesare disjoint. Thus the OGF for the set of sequences ofelements of A is 1 + A(z) + A(z)2 + · · · = (1−A(z))−1.

If a combinatorial class is constructed from atoms using onlydisjoint union, product and sequence constructions, then itscounting OGF is rational.




Let A,B be combinatorial classes with counting OGFsA(z), B(z). The size of an ordered pair of objects (α, β) isdefined to be |α|+ |β|.The counting OGF of A× B is then∑

γ∈A×Bz|γ| =

∑α∈A

∑β∈B

z|α|+|β| = A(z)B(z).







γ∈A×Bz|γ| =

∑α∈A

∑β∈B

z|α|+|β| = A(z)B(z).







γ∈A×Bz|γ| =

∑α∈A

∑β∈B

z|α|+|β| = A(z)B(z).




Extras on the symbolic method

The symbolic method for EGFs

EGFS are often used for labelled constructions. A labelledcombinatorial object is one where each atom carries a positiveinteger label and all labels are distinct. The labelling is properif the label set is 1, . . . , n.

Example: a permutation is a properly labelled sequence ofatoms.When forming the sum or product of labelled classes, we needto relabel so that a proper labelling is obtained and thestructure of the components is preserved. If a has size k and bsize n− k, then the number of ways to properly label theordered pair (a, b) is

(nk

).

Thus it makes sense to consider EGFs since(∑n

anzn/n!

)(∑n

bnzn/n!

)=∑

n

(∑k

(n

k

)akbn−k

)zn/n!.

Similarly, the EGF for sets is1 + A(z)/1! + A(z)2/2! + · · · = exp(A(z)).




EGFS are often used for labelled constructions. A labelledcombinatorial object is one where each atom carries a positiveinteger label and all labels are distinct. The labelling is properif the label set is 1, . . . , n.Example: a permutation is a properly labelled sequence ofatoms.

When forming the sum or product of labelled classes, we needto relabel so that a proper labelling is obtained and thestructure of the components is preserved. If a has size k and bsize n− k, then the number of ways to properly label theordered pair (a, b) is

(nk

).


anzn/n!

)(∑n

bnzn/n!

)=∑

n

(∑k

(n

k

)akbn−k

)zn/n!.





EGFS are often used for labelled constructions. A labelledcombinatorial object is one where each atom carries a positiveinteger label and all labels are distinct. The labelling is properif the label set is 1, . . . , n.Example: a permutation is a properly labelled sequence ofatoms.When forming the sum or product of labelled classes, we needto relabel so that a proper labelling is obtained and thestructure of the components is preserved. If a has size k and bsize n− k, then the number of ways to properly label theordered pair (a, b) is

(nk

).


anzn/n!

)(∑n

bnzn/n!

)=∑

n

(∑k

(n

k

)akbn−k

)zn/n!.






(nk

).


anzn/n!

)(∑n

bnzn/n!

)=∑

n

(∑k

(n

k

)akbn−k

)zn/n!.






(nk

).


anzn/n!

)(∑n

bnzn/n!

)=∑

n

(∑k

(n

k

)akbn−k

)zn/n!.




The symbolic method - binary tree example

A binary tree is either a single external node or an internalnode connected to a pair of binary trees. Let T be the classof binary trees:

T = ext ∪ int × T × T .

In terms of a formal grammar

< tree >=< ext >|< int >< tree >< tree > .

Give < ext > weight a and < int > weight b to obtainT (z) = za + zbT (z)2. Special cases: a = 0, b = 1 counts treesby internal nodes; a = 1, b = 0 by external nodes; a = b = 1by total nodes.



The symbolic method - binary tree example

A binary tree is either a single external node or an internalnode connected to a pair of binary trees. Let T be the classof binary trees:

T = ext ∪ int × T × T .

In terms of a formal grammar

< tree >=< ext >|< int >< tree >< tree > .

Give < ext > weight a and < int > weight b to obtainT (z) = za + zbT (z)2. Special cases: a = 0, b = 1 counts treesby internal nodes; a = 1, b = 0 by external nodes; a = b = 1by total nodes.



The symbolic method - composition examples

An integer composition is a sequence of positive integers. Thesize is the sum of the sequence. The counting OGF for thepositive integers is I(z) = z/(1− z), so the counting OGF forcompositions by size is (1− z/(1− z))−1 = (1− z)/(1− 2z)and there are 2n−1 compositions of n.

Let S be any subset of positive integers and letI(z) =

∑n∈S zn be its counting OGF. Then (1− I(z))−1

enumerates compositions with parts restricted to S. Example:S = 1, 2, . . . , r gives (1− z)/(1− 2z + zr+1).Similarly we can count the number of terms in the sequencewith another variable. The bivariate GF for compositions is

11− uz/(1− z)

=1− z

1− z − uz=

1− z

1− (1 + u)z.







enumerates compositions with parts restricted to S. Example:S = 1, 2, . . . , r gives (1− z)/(1− 2z + zr+1).

Similarly we can count the number of terms in the sequencewith another variable. The bivariate GF for compositions is

11− uz/(1− z)

=1− z

1− z − uz=

1− z

1− (1 + u)z.







enumerates compositions with parts restricted to S. Example:S = 1, 2, . . . , r gives (1− z)/(1− 2z + zr+1).Similarly we can count the number of terms in the sequencewith another variable. The bivariate GF for compositions is

11− uz/(1− z)

=1− z

1− z − uz=

1− z

1− (1 + u)z.



The symbolic method - labelled tree example

A properly labelled (unordered) tree is a connected acyclicgraph with n vertices, each with one of the numbers 1, . . . , n.

By symbolic method, the set T of rooted labelled unorderedtrees satisfies T = • × set(T ) and so T (z) = z exp(T (z)).Lagrange inversion gives

an =n!n

[yn−1] exp(ny) =n!n

[yn−1]∑

k

(ny)k/k! = nn−1.

Hence the number of labelled trees is nn−2.





By symbolic method, the set T of rooted labelled unorderedtrees satisfies T = • × set(T ) and so T (z) = z exp(T (z)).

Lagrange inversion gives

an =n!n


[yn−1]∑

k

(ny)k/k! = nn−1.







an =n!n


[yn−1]∑

k

(ny)k/k! = nn−1.







an =n!n


[yn−1]∑

k

(ny)k/k! = nn−1.



Coefficient extraction from GFs

Coefficient extraction formulae (no proof)

Some basic results (best proved using the Cauchy integralformula):

n! ≈√

2πn(n/e)n (Stirling).

Hence1

n + 1

(2n

n

)≈ 4n

√πn

.

Suppose F (z) =∑

n anzn = G(z)/H(z) is rational and Hhas a unique root ρ of smallest modulus. Then

an ≈ Cρ−nnp−1

where p is the order of ρ and C is easily computable.If F (z) = zφ(F (z)) with φ(0) 6= 0, then

[zn]F (z) =1n

[un−1]φ(u)n (Lagrange inversion formula).





n! ≈√


Hence1

n + 1

(2n

n

)≈ 4n

√πn

.

Suppose F (z) =∑



where p is the order of ρ and C is easily computable.

If F (z) = zφ(F (z)) with φ(0) 6= 0, then

[zn]F (z) =1n






n! ≈√


Hence1

n + 1

(2n

n

)≈ 4n

√πn

.

Suppose F (z) =∑



where p is the order of ρ and C is easily computable.If F (z) = zφ(F (z)) with φ(0) 6= 0, then

[zn]F (z) =1n




Lagrange inversion example: degree-restricted trees

Let 0 ∈ Ω ⊆ N. We consider the class TΩ of all ordered planetrees such that the outdegree of each node is restricted tobelong to Ω.

Examples: Ω = 0, 1 gives paths; Ω = 0, 2 gives binarytrees; Ω = 0, t gives t-ary trees; Ω = N gives generalordered trees.

Let TΩ(z) be the enumerating GF of this class. The symbolicmethod immediately gives the equation

TΩ(z) = zφ(TΩ(z))

where φ(x) =∑

ω∈Ω xω.

Lagrange inversion is tailor-made for this situation. We have

[zn]TΩ(z) =1n

[un−1]φ(u)n.








where φ(x) =∑

ω∈Ω xω.


[zn]TΩ(z) =1n

[un−1]φ(u)n.








where φ(x) =∑

ω∈Ω xω.


[zn]TΩ(z) =1n

[un−1]φ(u)n.








where φ(x) =∑

ω∈Ω xω.


[zn]TΩ(z) =1n

[un−1]φ(u)n.



Lagrange inversion example II - ternary trees

Let Ω = 0, 3. Then the counting (by external nodes) OGFT (z) satisfies T (z) = z(1 + T (z)3).

By Lagrange inversion we get

an = [zn]T (z) =1n

[un−1](1 + u3

)n.

By lookup we obtain

an =

1n

(nk

)if n = 3k + 1

0 otherwise.

Asymptotics are easily derived using Stirling’s approximation.




Let Ω = 0, 3. Then the counting (by external nodes) OGFT (z) satisfies T (z) = z(1 + T (z)3).By Lagrange inversion we get

an = [zn]T (z) =1n

[un−1](1 + u3

)n.

By lookup we obtain

an =

1n

(nk

)if n = 3k + 1

0 otherwise.






an = [zn]T (z) =1n

[un−1](1 + u3

)n.

By lookup we obtain

an =

1n

(nk

)if n = 3k + 1

0 otherwise.






an = [zn]T (z) =1n

[un−1](1 + u3

)n.

By lookup we obtain

an =

1n

(nk

)if n = 3k + 1

0 otherwise.



Trees

Types of trees I

A free tree is a connected graph with no cycles.

A rooted tree is a free tree with a distinguished node calledthe root.

An ordered tree is a rooted tree where the order of subtrees isimportant; recursively, a root connected to a sequence ofordered trees.

A m-ary tree is an ordered tree where every node has 0 or mchildren.

A labelled tree is a tree with n nodes such that each node islabelled by an element of [n] and all labels are distinct.

Synonyms in literature: plane = ordered; oriented = rooted.


Trees

Some trees in analysis of algorithms

Binary search tree: a binary tree with each internal nodehaving a key, such that the key of each node n is ≤ all keys inRn and ≥ all keys in Ln. Applications: database forcomparable data, model for quicksort.

Heap-ordered tree: a binary tree such that the key of eachnode is ≥ the key of anything in its subtree. Applications:priority queue.

Trie: an m-ary tree where each external node may containdata; children of leaves must be nonempty. Applications:database for string data, model for radix exchange sort, leaderelection in distributed computing.


Trees

Tree attributes

The size is the number of (external, internal, or just plain)nodes.

The depth of a node in a rooted tree is the distance to theroot.

The maximum depth is the height. The sum of all depths ofinternal (external) nodes is the internal (external) path length.


Trees

Path length in binary trees (uniform model)

The bivariate generating function F (z, u) enumerating binarytrees by number of nodes and internal path length satisfies theequation

F (z, u) = 1 + zF (zu, u)2.

The mean and variance are given by a standard computation.Note that

Fu(z, u) = 2zF (zu, u)[Fu(zu, u) + zFz(zu, u)]

and so Fu(z, 1) = 2zF (z, 1)[Fu(z, 1) + zFz(z, 1)]. Thus

µn :=[zn] zFz(z,1)

1−2zF (z,1)

[zn]F (z, 1)

The mean µn is asymptotic to√

πn3/2, so the mean level of anode is of order

√n. The variance is also of order n3/2.


Trees

Path length in binary search trees

Suppose we insert n distinct keys into an initially empty BST.The uniform distribution on permutations of size n inducesthe quicksort distribution on BSTs of size n.

The internal path length equals the construction cost of abinary search tree of size n; dividing by n gives the expectedcost of a successful search.

Let F (z, u) =∑ z|π|

|π|! u`(π) be the BGF of BSTs by size and

internal path length. Note that [zn]F (z, u) is the PGF ofinternal path length on BSTs with n nodes.Then

Fz(z, u) = F (zu, u)2 F (0, u) = 1.

Moments of the distribution are easily obtained as for theuniform model. The mean is ∼ 2n log n and variance is inΘ(n2). Quite a different shape.


Strings

String basics

Let A be a finite set called the alphabet. A string over A is afinite sequence of elements of A. The set of all strings over Ais written A∗.

A subset of A∗ is a language.

If A = 0, 1 the strings are called bitstrings.

Basic algorithmic questions: string matching (find a pattern ina given string); search for a word in a dictionary; compress astring. Many applications in computational biology, computersecurity, etc.


Strings

Hidden pattern occurrences

The set of occurrences of the subsequence (hidden pattern)−a1 − a2 − · · · − ak− in a string of length n corresponds toA∗a1A∗a2A∗a3 . . .A∗akA∗.

The counting OGF of A∗ is 1/(1−mz) so the OGF for allpattern occurrences is P (z) = zk/(1−mz)k+1 wherem = |A|.The expected number of occurrences in a random “word” oflength n is [zn]P (z)/(mn) = m−k

(nk

).

The OGF for total occurrences of the substring a1a2 . . . ak iszk/(1−mz)2 and a similar analysis applies.

Note relevance to various conspiracy theories. We shouldexpect a sufficiently long random text to contain any givenhidden message.


Strings

Pattern avoidance

We have already counted total occurrences of a givensubstring or pattern. Now we want to count number of wordsan not containing a given pattern (a harder problem).

A nice trick: let T be the position of the end of the firstoccurrence of the pattern, Xn the event that the first n bitsof a random bitstring do not contain the pattern. ThenS(z) =

∑n≥0 anzn implies that

S(1/2) =∑n≥0

an/2n =∑n≥0

Pr(Xn) =∑n≥0

Pr(T > n) = E[T ].


Strings

Pattern avoidance - simple example

Given substring σ = 00 · · · 0 of length k, let S(z) be thecounting OGF for bitstrings without σ as substring.Recursion/symbolic method gives

S(z) =

(∑i<k

zi

)(1 + zS(z))

so

S(z) =1 + z + z2 + · · ·+ zk−1

1− z − z2 − · · · − zk=

1− zk

1− 2z + zk+1.

Asymptotics: an ≈ Cρ−n where ρ is smallest modulus root ofdenominator.Note that ρ = 1/2 + ρk+1/2 and 0 < ρ < 1. Thus1/2 < ρ < 1/2 + 1/2k, etc, and we can compute ρ quickly byiteration.Note S(1/2) = 2k+1 − 2.


Strings

Substring patterns - autocorrelation polynomial I

Consider an arbitrary binary string σ = σ0σ1 · · ·σk−1 of lengthk.

For 0 ≤ j ≤ 1, shift σ right j places. Define cj = 1 if theoverlap matches the tail σ(j) of σ, cj = 0 otherwise. Theautocorrelation polynomial is c(z) =

∑j cjz

j .

Let S, (resp. T ) be the set of bitstrings not containing p(resp. containing it once at the end). Then

S ∪ T ∼= ε ∪ S × 0, 1S × σ ∼= T × ∪j:cj 6=0σ

(j)

and the symbolic method gives S(z) + T (z) = 1 + 2zS(z)and S(z)zk = T (z)c(z).


Strings




∑j cjz

j .


S ∪ T ∼= ε ∪ S × 0, 1S × σ ∼= T × ∪j:cj 6=0σ

(j)



Strings




∑j cjz

j .


S ∪ T ∼= ε ∪ S × 0, 1S × σ ∼= T × ∪j:cj 6=0σ

(j)



Strings

Substring patterns - autocorrelation polynomial II

Thus

S(z) =c(z)

zk + (1− 2z)c(z).

Note [zn]S(z/2) = Pr(Xn).Looking at the poles of S(z/2) we see that Pr(Xn) → 1exponentially fast as n →∞.

Thus every fixed substring is contained with high probabilityin a sufficiently long string.


Strings


Thus

S(z) =c(z)

zk + (1− 2z)c(z).

Note [zn]S(z/2) = Pr(Xn).

Looking at the poles of S(z/2) we see that Pr(Xn) → 1exponentially fast as n →∞.



Strings


Thus

S(z) =c(z)

zk + (1− 2z)c(z).




Strings


Thus

S(z) =c(z)

zk + (1− 2z)c(z).




Strings

Regular languages

Rational GFs always arise from the transfer matrix method.

Special case: the counting GF of an unambiguous regularlanguage is rational (Chomsky-Schutzenberger, 1963).

Recall that every regular language can be defined by anunambiguous regular expression.

Thus if we construct a combinatorial class iteratively usingonly disjoint union, cartesian product, and sequence, thecounting GF is rational.


Strings

Regular expression example

Consider language (over alphabet a, b) defined by(bb | a(bb)∗aa | a(bb)∗(ab | ba)(bb)∗(ab | ba))∗ (number of b’sis even, number of a’s divisible by 3).

The symbolic method gives

S(z) =(1− z2)2

1− 3z2 − z3 + 3z4 − 3z5 + z6.

Hence an ≈ CAn, A ∼= 1.7998.

Need to check that the expression is unambiguous.


Strings

Patterns in strings: summary

The generating function for any regular expression is arational function and can be computed algorithmically.

In certain special cases more efficient methods (such asautocorrelation polynomial) exist.

We have really only considered the case where all lettersappear independently and with equal probability. In realapplications, letter probabilities vary and letters are notindependent. The methods above can be extended to copewith this.


Tries

Tries

Each binary tree corresponds to a set of binary strings (0encodes left branch, 1 encodes right branch, string is given bylabels on path to external node). This set of strings isprefix-free.

Conversely a finite prefix-free set of strings corresponds to aunique binary tree, a full trie.

More generally, we may stop branching as soon as the stringsare all distinguished. This gives a trie, a binary tree such thatall children of leaves are nonempty. Each string is stored in anexternal node but not all external nodes have strings. Can bedescribed by symbolic method.

A Patricia trie saves space, by collapsing one-way branches toa single node.

Relevant parameters: number of internal nodes In; externalpath length Ln; length of rightmost branch Rn.


Tries

Trie recurrences

We assume that a trie is built from n infinite random bitstrings.Each bit of each string is independently either 0 or 1. We have

Ln = n +12n

∑k

(n

k

)(Lk + Ln−k)

In = 1 +12n

∑k

(n

k

)(Ik + In−k)

Rn = 1 +12n

∑k

(n

k

)Rk.

Let L(z) =∑

n Lnzn/n!, etc. Then

L(z) = 2L(z/2)ez/2 + zez − z;

I(z) = 2I(z/2)ez/2 + ez − z − 1;

R(z) = R(z/2)ez/2 + ez − z − 1;

P (z) = P (z/2)ez/2.


Tries

Solving the trie recurrences, I

If φ(z) = 2ez/2φ(z/2) + a(z), then by iteration we obtain

φ(z) =∑j≥0

2jez(1−2−j)a(2−jz).

Thus we obtain

Ln = n∑j≥0

(1−

(1− 2−j

)n−1)

In =∑j≥0

2j[1−

(1− 2−j

)n − n

2j

(1− 2−j

)n−1].

How to derive an asymptotic approximation? See Flaj-Sedgp211, p402 for elementary arguments. Answers:Ln ≈ n lg n, In ≈ n/ lg 2. More precise answers are obtainedby complex methods (Mellin transform).


Tries

Solving the trie recurrences, II

Define φ(z) = e−zφ(z), etc (this is the Poisson transform).Then we have

φ(z) = 2φ(z/2) + a(z).

Iteration yields

φ(z) =∑j≥0

2j a(2−jz).

This gives, on inverting the transform,

Ln =∑k≥2

(−1)k

(n

k

)k2k−1

2k−1 − 1.

Asymptotics for such alternating sums can be obtained byRice’s method.


Tries

Summary: tries

A useful data structure for dictionary and pattern matching.Also a mathematical model for many algorithms.

Asymptotically optimal (lg n) expected search cost.

Space wastage: about 44% extra nodes (1/ lg 2− 1).

Recurrences under the infinite random bitstring model yieldGF equations that are tricky. Solution involves infinite sums offunctions.

Explicit formulae for solutions are infinite sums. Mellintransforms or Rice’s integrals give precise asymptotics;elementary methods can also be used.


Randomized Algorithms

Incorporating random choices into an algorithm can improveits expected performance. The key idea is that no adversarycan force us into a bad case, because our choices are unknown.

There are philosophical problems concerning random numbergeneration on a deterministic computer, or even what“random” means. Pseudorandom generators are used inpractice; their output passes most statistical tests for“randomness”.

We can also think of a randomized algorithm as an elementrandomly chosen from a set of algorithms.

There are two main types: Monte Carlo (answer may bewrong, running time is deterministic) and Las Vegas (answeris correct, running time is random).


Monte Carlo Algorithms

Monte Carlo algorithms

If the runtime on a given instance is deterministic and theanswer is correct with bounded probability, we have a MonteCarlo algorithm.

A MC algorithm for a decision problem is biased if one of theanswers (yes/no) is always correct when given. In otherwords, for example, P (Y |N) ≡ P(says Y given answer is N)= 0, P (N |N) = 1, P (Y |Y ) = p, P (N |Y ) = 1− p.

Examples: fingerprinting (verification of identities); primalitytesting.



Fingerprinting

Suppose we have a set U and want to determine whetherelements u, v are equal. It is often easier to choose afingerprinting function f and compute whether f(u) = f(v),in which case we return yes. Such methods are always biased:P (N |Y ) = 0.

Example: X, Y, Z are n× n matrices, and we want to knowwhether XY = Z. Choose a random n bit vector r andcompute XY r and Zr. Can show P (Y |N) ≤ 1/2.

Example: M is a symbolic matrix in variables x1, . . . xn; wewant to know whether det(M) = 0. Choose a finite subset Sof C and choose r1, . . . rn independently and uniformly fromS. Substitute xi = ri for all i and compute the determinant.Can show P (Y |N) ≤ m/|S| where m is the total degree ofthe polynomial det(M).There are many specific applications of the above examples.



Improving biased Monte Carlo algorithms

Let 0 < p < 1. Say a MC algorithm is p-correct if its errorprobability is at most 1− p on every instance (sometimes pdepends on the size, but never on the instance itself).

This means P (Y |N) ≤ 1− p, P (N |Y ) ≤ 1− p.

If, say, NO is always right then we can improve our confidencein the answer by repeating the algorithm n times on the sameinstance. This is amplification of the stochastic advantage. Ifwe ever get NO, report NO. Else report YES. Probability oferror is at most (1− p)n. To reduce this to ε requires numberof trials proportional to lg(1/ε) and to − lg(1− p).



Improving unbiased Monte Carlo algorithms

We need p ≥ 1/2. Repeat n (odd) times and return the morefrequent answer. Analysis is more complicated.

Let Xi = 1 if ith run gives correct answer, 0 otherwise. ThenXi is a Bernoulli random variable and X =

∑ni=1 Xi is

binomial with parameter p. Probability of error of repeatedalgorithm is P (X < n/2). This is just∑

j<n/2

(nj

)pj(1− p)n−j .

Simplifying this could be done, but it is easier to use thenormal approximation: X is approximately normal with meannp and variance np(1− p) for n large enough. Use table ofnormal distribution to work out size of n for given ε. Answeris proportional to lg(1/ε) and (p− 1/2)−2.



Primality testing

We wish to know whether a given positive integer n is prime(has no proper factor other than 1).

Fermat (1640) proved that if n is prime, and 1 ≤ a ≤ n− 1,then an−1 ≡ 1 mod n. However if n is composite (notprime) then this equality may or may not hold.

An obvious idea is: given n, choose a randomly and computean−1 mod n (recall that we can do this quickly, using adivide-and conquer approach). If the answer is not 1, wereport NO (such an a is called a witness).

This gives a biased Monte Carlo algorithm withP (N |N) = 1, P (Y |N) = 0. What about P (N |Y ), P (Y |Y )?Unfortunately P (N |Y ) can be made arbitrarily close to 1(some n have a lot of false witnesses). So this algorithm isnot p-correct for any p > 0.



Improving the primality testing algorithm

There is a more complicated test (Miller-Rabin, 1976) basedon Fermat’s result that gives P (N |Y ) ≤ 1/4, and hence a3/4-correct algorithm.

There is an even more complicated test(Agrawal-Kayal-Saxena 2002) that is in fact always correct,and this gives the first worst-case polynomial-time algorithmfor primality. But it is not as fast as the randomized algorithmabove.


Las Vegas Algorithms

Las Vegas algorithms

Nice properties: can find more than one solution even on sameinput; breaks the link between input and worse-case runtime.

Every Las Vegas algorithm can be converted to a Monte Carloalgorithm: just report a random answer if the running timegets too long.

Examples:

randomized quicksort and quickselect;randomized greedy algorithm for n queens problem;integer factorization;universal hashing;linear time MST.


Las Vegas Algorithms

Improving Las Vegas algorithms

If a particular run takes too long, we can just stop and runagain on the same input. If the expected runtime is fast, it isvery unlikely that many iterations will fail to run fast.

To quantify this: let p be probability of success, s theexpected time to find a solution, and f the expected timewhen failure occurs. Then expected time t until we find asolution is given by t = ps + (1− p)(f + t), sot = s + (1− p)f/p.

We can use this to optimize the repeated algorithm.

Documents

COMPSCI 720S1C, 2006 - Aucklandmcw/Teaching/720/lectures.pdf · Outline Background Introduction Generating Functions Generating functions and recurrences Combinatorial and Algorithmic