Chap 5: Decrease & conquer

Objectives

To introduce the decrease-and-conquer mind setTo show a variety of decrease-and-conquer solutions:

Depth-First Graph TraversalBreadth-First Graph TraversalFake-Coin ProblemInterpolation Search

To discuss the strengths and weaknesses of a decrease-and-conquer strategy

Decrease and Conquer

1. Reduce problem instance to smaller instance of the same problem and extend solution

2. Solve smaller instance3. Extend solution of smaller

instance to obtain solution to original problemAlso called inductive or incremental

SUBPROBLEM OF SIZE n-1

A SOLUTION TO THEORIGINAL PROBLEM

A SOLUTION TO SUBPROBLEM

A PROBLEM OF SIZE n

Divide-and-Conquer Illustrated

SUBPROBLEM 2 OF SIZE n/2

SUBPROBLEM 1 OF SIZE n/2

A SOLUTION TO SUBPROBLEM 1

A SOLUTION TO THEORIGINAL PROBLEM

A SOLUTION TO SUBPROBLEM 2

A PROBLEM OF SIZE n

Flavours of Decrease and Conquer

Decrease by a constant (usually 1): instance is reduced by the same constant on each iteration

Insertion sortGraph Searching: DFS, BFSGenerating combinatorials

Decrease by a constant factor (usually 2): instance is reduced by same multiple on each iteration

Binary search Fake-coin problem

Variable-size decrease: size reduction pattern varies from one iteration to the next

Euclid’s algorithmInterpolation Search

Flashback: Knapsack by Exhaustive Search

Efficiency: Ω(2n)

Subset Total Wgt

Total Value

1 2 kg R200

2 5 kg R300

3 10 kg R500

4 5 kg R100

1,2 7 kg R500

1,3 12 kg R700

1,4 7 kg R300

2,3 15 kg R800

Subset Total Wgt

Total Value

2,4 10 kg R400

3,4 15 kg R600

1,2,3

17 kg n/a

1,2,4

12 kg R600

1,3,4

17 kg n/a

2,3,4

20 kg n/a

1,2,3,4

22 kg n/a

Flashback:Generating Subsets

Combinatorics uses Decrease (by one) and Conquer AlgorithmsGenerate all 2n subsets of A = a1, …, an

Divide into subsets of a1, …, an-1 that contain an and those that don’tSneaky Solution: establish a correspondence between bit strings and subsets. Bit n denotes presence (1) or absence (0) of element an

Generate numbers from 0 to 2n-1 convert to bit strings interpret as subsetsExamples: 000 = Ø , 010 = a2 , 110 = a1, a2

Flashback:Generating Permutations

Generate all n! reorderings of 1, …, nGenerate all (n-1)! permutations of 1, …, n-1Insert n into each possible position (starting from the right or left, alternately)Implemented by the Johnson-Trotter algorithm

Satisfies Minimal-Change requirementNext permutation obtained by swapping two elements of previousUseful for updating style algorithms

Example:Start: 1Insert 2: 12 21Insert 3: 123 132 312 321 231 213

Decrease and Conquer

1. Reduce problem instance to smaller instance of the same problem and extend solution

2. Solve smaller instance3. Extend solution of smaller

instance to obtain solution to original problemAlso called inductive or incremental

SUBPROBLEM OF SIZE n-1

A SOLUTION TO THEORIGINAL PROBLEM

A SOLUTION TO SUBPROBLEM

A PROBLEM OF SIZE n

Exercise: Spot the Difference

Problem: Derive algorithms for computing an using:1. Brute Force2. Divide and conquer3. Decrease by one4. Decrease by constant factor (halve the

problem size)

Hint: each can be described in a single line

Exercise: Spot the Difference

Problem: Derive algorithms for computing an using:1. Brute Force : an= a*a*a*a*...*a n times2. Divide and conquer : an= (an/2 ) * (an/2 )3. Decrease by one : an= ( an-1 )* a 4. Decrease by constant factor (halve the

problem size) : an= (an/2) 2

Graph Traversal

Many problems require processing all graph vertices in a systematic fashionData Structures Reminder:

Graph traversal strategies:Depth-first search (traversal for the Brave)Breadth-first search (traversal for the Cautious)

a b

c d

a b c d

a 0 1 1 1

b 1 0 0 1

c 1 0 0 1

d 1 1 1 0

a

b

c

d

a

b c d

d

a d

a b c

Depth-First Search

Explore graph always moving away from last visited vertexSimilar to preorder tree traversal DFS(G): G = (V, E)

count 0mark each vertex as 0for each vertex v∈ V

if v is marked as 0

dfs(v)

dfs(v):count count + 1mark v with countfor each vertex w adjacent to v DOif w is marked as 0dfs(w)

Example: DFSa b

e f

c d

g h

Traversal Stack: (pre = push, post = pop)

1a8 2b7 3f2 4e1

5g6 6c5 7d4 8h3

Push order: a b f e g c d h

Pop order:e f h d c g b a

DFS Forest

DFS Forest: A graph representing the traversal structure Types of Edges:

Tree edges: edge to next vertex in traversal Back edges: edge in graph to ancestor nodesForward edges: edge in graph to descendantsCross edges: none of the above

a b

e f

c d

g h

a b ef

c dg h

Backedge

Notes on Depth-First Search

Applicable to different graph structures:Adjacency matrices: (V2)Adjacency linked lists: (V+E)

Yields two orderings:preorder: as vertices are 1st encountered (pushed)postorder: as vertices become dead-ends (popped)

Applications:Checking connectivity, finding connected componentsChecking a-cyclicitySearching state-space of problems for solution (AI)

Breadth-First Search

Move across to all neighbours of the last visited vertexSimilar to level-by-level tree traversals Instead of a stack, breadth-first uses a queue

BFS(G): G = (V, E)count 0mark each vertex as 0for each vertex v∈ V if v is marked as 0 bfs(v)

bfs(v):count count + 1mark v with countinitialize queue with vwhile queue not empty DO

a front of queuefor each vertex w adjacent to a

if w is marked as 0 count count +

1 mark w with

count add w to queue

remove a from queue

Example: BFSa b

e f

c d

g h

Traversal Queue:

a1 b2 e3 f4

g5

c6 h7

d8

Order: a b e f g c h d a

b e f

cd

g

hCrossedge

Notes on Breadth First Search

BFS has same efficiency as DFS and can be implemented with:

Adjacency matrices: (V2)

Adjacency linked lists: (V+E)

Yields single ordering of vertices Applications:

Same as DFS, but can also find paths from a vertex to all other vertices with the smallest number of edges

Decrease and Conquer

1. Reduce problem instance to smaller instance of the same problem and extend solution

2. Solve smaller instance3. Extend solution of smaller

instance to obtain solution to original problemAlso called inductive or incremental

SUBPROBLEM of smaller size

A SOLUTION TO THEORIGINAL PROBLEM

A SOLUTION TO SUBPROBLEM

A PROBLEM OF SIZE n

BFS(G): G = (V, E)count 0mark each vertex as 0for each vertex v∈ V if v is marked as 0 bfs(v)

bfs(v):count count + 1mark v with countinitialize queue with vwhile queue not empty DO

a front of queuefor each vertex w adjacent to a

if w is marked as 0 count count +

1 mark w with

count add w to queue

remove a from queue

DFS(G): G = (V, E)count 0mark each vertex as 0for each vertex v∈ V

if v is marked as 0

dfs(v)

bfs(v):count count + 1mark v with countinitialize queue with vwhile queue not empty DO

a front of queuefor each vertex w adjacent to a

if w is marked as 0 count count +

1 mark w with

count add w to queue

remove a from queue

dfs(v):count count + 1mark v with countfor each vertex w adjacent to v DOif w is marked as 0dfs(w)

Example: BFSa b

e f

c d

g h

Traversal Queue:

a1 b2 e3 f4

g5

c6 h7

d8

Order: a b e f g c h d a

b e f

cd

g

hCrossedge

Example: BFSa b

e f

c d

g h

Traversal Queue:

a1 b2 e3 f4

g5

c6 h7

d8

Order: a b e f g c h d a

b e f

cd

g

hCrossedge

DFS Forest

DFS Forest: A graph representing the traversal structure Types of Edges:

Tree edges: edge to next vertex in traversal Back edges: edge in graph to ancestor nodesForward edges: edge in graph to descendantsCross edges: none of the above

a b

e f

c d

g h

a b ef

c dg h

Backedge

Objectives

To introduce the decrease-and-conquer mind setTo show a variety of decrease-and-conquer solutions:

Depth-First Graph TraversalBreadth-First Graph TraversalFake-Coin ProblemInterpolation Search

To discuss the strengths and weaknesses of a decrease-and-conquer strategy

The Fake-Coin Problem: Decrease by a Constant

FactorProblem:

Among n identical looking coins, one is a fake (and weighs less) We have a balance scale which can compare any two sets of coins

Algorithm:Divide into two size n/2 piles (keeping a coin aside if n is odd)If they weigh the same then the extra coin is fakeOtherwise proceed recursively with the lighter pile

Efficiency: W(n) = W( n/2 ) + 1 for n > 1W(n) = log2 n = (log2 n)

But there is a better (log3 n) algorithm

Euclid’s GCD: Variable-Size Decrease

Problem:Greatest Common Divisor of two integers m and n is the largest integer that divides both exactly

Alternative Solutions: Consecutive integer checking (brute force)Identify common prime factors (transform and conquer)

Euclid’s Solution:gcd(m, n) = gcd(n, m mod n)gcd(m, 0) = mRight-side args are smaller by neither a constant size nor factor

Example:gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

Interpolation Search: Variable-Size Decrease

Mimics the way humans search through a phone book (look near the beginning for ‘Brown’)Assumes that values between the leftmost (A[l]) and rightmost (A[r]) elements increase linearlyAlgorithm (key = v, find search index = i):

Binary search with floating variable at index iSetup straight line through (l, A[l]) and (r, A[r])Find point P = (x, y) on line at y = v, then i = x

Efficiency:Average = (log log n + 1)Worst = (n)

l i r

A[l]

v

A[r]

index

value

Strengths and Weaknesses of

Decrease-and-ConquerStrengths:

Can be implemented either top down (recursively) or bottom up (without recursion)Often very efficient (possibly (log n) )Leads to a powerful form of graph traversal (Breadth and Depth First Search)

Weaknesses:Less widely applicable (especially decrease by a constant factor)