Decrease and Conquer

2

Decrease and ConquerExploring the relationship between a solution to a given instance of (e.g., P(n) )a problem and a solution to a smaller instance (e.g., P(n/2) or P(n-1) )of the same problem.Use top down(recursive) or bottom up (iterative) to solve the problem.Example, n!

A top down (recursive) solution A bottom up (iterative) solution

3

Examples of Decrease and ConquerDecrease by a constant: the size of the problem is reduced by the same constanton each iteration/recursion of the algorithm.

Insertion sortGraph search algorithms:

DFSBFSTopological sorting

Algorithms for generating permutations, subsetsDecrease by a constant factor: the size of the problem is reduced by the same constant factor on each iteration/recursion of the algorithm.

Binary search Fake-coin problems

Variable-size decrease: the size reduction pattern varies from one iteration of an algorithm to another.

Euclid’s algorithm

4

A Typical Decrease by One Technique

subproblemof size n-1

a solution to thesubproblem

a solution tothe original problem

a problem of size n

e.g., n!

5

A Typical Decrease by a Constant Factor (half) Technique

subproblemof size n/2

a solution to the subproblem

a solution tothe original problem

a problem of size n

e.g., Binary search

6

A Typical Divide and Conquer Technique

subproblem 2 of size n/2

subproblem 1 of size n/2

a solution to subproblem 1

a solution tothe original problem

a solution to subproblem 2

a problem of size n

e.g., mergesort

7

What’s the Difference?Consider the problem of exponentiation: Compute an

Decrease by oneBottom-up: iterative (brute Force)

Top-down:recursiveDivide and conquer:

Decrease by a constant factor:

an= a*a*a*a*...*a

an= an-1* a if n > 1= a if n = 1

an= a ⎣ n/2 ⎦ * a ⎡ n/2 ⎤ if n > 1= a if n = 1

an = (an/2 ) 2 if n is even and positive= (a(n-1)/2 ) 2 * a if n is odd and > 1= a if n = 1

8

Insertion Sort

A decrease by one algorithmA bottom-up (iterative) solution A top-down (recursive) solution

Insertion Sort: an Iterative Solution

9

ALGORITHM InsertionSortIter(A[0..n-1])//An iterative implementation of insertion sort//Input: An array A[0..n-1] of n orderable elements//Output: Array A[0..n-1] sorted in nondecreasing order

for i 1 to n – 1 do // i: the index of the first element of the unsorted part.

key = A[i]for j i – 1 to 0 do // j: the index of the sorted part of the array

if (key< A[j]) //compare the key with each element in the sorted part

A[j+1] A[j]else

breakA[j +1] key //insert the key to the sorted part of the array

Efficiency?

Insertion Sort: a Recursive Solution

10

ALGORITHM InsertionSortRecur(A[0..n-1], n-1)//A recursive implementation of insertion sort//Input: An array A[0..n-1] of n orderable elements//Output: Array A[0..n-1] sorted in nondecreasing orderif n > 1

InsertionSortRecur(A[0..n-1], n-2)Insert(A[0..n-1], n-1) //insert A[n-1] to A[0..n-2]

ALGORITHM Insert(A[0..m], m)//Insert A[m] to the sorted subarray A[0..m-1] of A[0..n-1]//Input: A subarray A[0..m] of A[0..n-1]//Output: Array A[0..m] sorted in nondecreasing orderkey = A[m]for j m – 1 to 0 do

if (key< A[j])A[j+1] A[j]

elsebreak

A[j +1] key

Recurrence relation?

Index of the element/key to be inserted.

Index of the last element

11

ExercisesDesign a recursive decrease-by-one algorithm for

finding the position of the largest element in an array of n real numbers.

Determine the time efficiency of this algorithm and compare it with that of the brute-force algorithm for the same problem.

12

Graph TraversalMany problems require processing all graph vertices in a systematic fashion

Graph traversal algorithms:

Depth-first search

Breadth-first search

13

Some Terminologies

Vertices and EdgesUndirected and directed graphs Parents, children, and ancestors.Connected graphs

A graph is said to be connected if for every pair of its vertices u and v there is a path from u to v.

14

Graph RepresentationAdjacency matrix

n x n boolean matrix if |V| is n.The element on the ith row and jth column is 1 if there’s an edge from ith vertex to the jth vertex; otherwise 0.The adjacency matrix of an undirected graph is symmetric.It takes (comparisons) to figure out all the adjacent nodes of a certain node i.

Adjacency linked listsA collection of linked lists, one for each vertex, that contain all the vertices adjacent to the list’s vertex.

An example

|V|

15

Depth-first searchThe idea

traverse “deeper” whenever possible. On each iteration, the algorithm proceeds to an unvisited vertex that is adjacent to the one it is currently in. If there are more than more neighbors, break the tie by the alphabetic order of the vertices.When reaching a dead end ( a vertex with no adjacent unvisited vertices), the algorithm backs up one edge to the parent and tries to continue visiting unvisited vertices from there.The algorithm halts after backing up to the starting vertex, with the latter being a dead end.

Similar to preorder tree traversals It’s convenient to use a stack to trace the operation of depth-first search.

Push a vertex onto the stack when the vertex is reached for the first time.Pop a vertex off the stack when it becomes a dead end.

An example

16

Example – undirected graphsa b

e f

c d

g h

Depth-first traversal:Give the order in which the vertices were reached.

a b f e g c d h

17

Depth-first search (DFS)Pseudocode for Depth-first-search of graph G=(V,E)

dfs(v)//Use depth-first to visit a connected component starting //from vertex v.count count + 1mark v with count //visit vertex vfor each vertex w adjacent to v do

if w is marked with 0 //w has not been visited yet.dfs(w)

DFS(G) // Use depth-first to visit a graph, which might // contain multiple connected components.

count 0 //visiting sequence number

mark each vertex with 0 // (unvisited)

for each vertex v∈ V do

if v is marked with 0 //w has not been visited yet.

dfs(v)

18

Types of edgesTree edges: edges comprising forest

Back edges: edges to ancestor nodes

Forward edges: edges to descendants (digraphs only)

Cross edges: none of the above

19

Example – directed grapha b

e f

c d

g h

Depth-first traversal: Give the order in which the vertices were reached.

a b f g h e c d

20

Depth-first search: Notes

DFS can be implemented with graphs represented as:

Adjacency matrices: Θ(|V |2)Adjacency linked lists: Θ(|V |+|E| )

Applications:checking connectivityfinding connected components

21

Breadth-first search (BFS)The idea

Traverse “wider” whenever possible. Discover all vertices at distance k from s (on level k) before discovering any vertices at distance k +1 (at level k+1)

Similar to level-by-level tree traversals Instead of a stack, breadth-first uses a queue.

22

Example – undirected grapha b

e f

c d

g h

Breadth-first traversal:

a b e f g c h d

23

Breadth-first search algorithmBFS(G)count 0mark each vertex with 0for each vertex v∈ V doif v is marked with 0

bfs(v)

bfs(v)count count + 1mark v with count //visit vinitialize queue with v //enqueuewhile queue is not empty do

a front of queue //dequeuefor each vertex w adjacent to a do

if w is marked with 0 //w hasn’t been visited.count count + 1mark w with count //visit wadd w to the end of the queue //enqueue

remove a from the front of the queue

24

Example – directed grapha b

e f

c d

g h

Breadth-first traversal:

a b e f g h c d

25

Breadth-first search: NotesBFS has same efficiency as DFS and can be implemented with graphs represented as:

Adjacency matrices: Θ(|V |2)Adjacency linked lists: Θ(|V |+|E| )

Applications: checking connectivityfinding connected componentsfinding paths between two vertices with the smallest number of edges

26

Topological SortingProblem: Given a Directed Acyclic Graph(DAG) G = (V, E), find a linear ordering of all its vertices such that if G contains an edge (u, v), then u appears before v in the ordering.Example: Give an order of the courses so that the prerequisites are met.

C1 C3

C2 C5

C4

Is the problem solvable if the digraph contains a cycle?

27

The DFS-Based AlgorithmDFS-based algorithm: (2 orderings exist in the DFS)

DFS traversal noting the order in which vertices are popped off stack (the order in which the dead end vertices appear)Reverse the above order.

QuestionsCan we use the order in which vertices are pushed onto the DFS stack (instead of the order they are popped off it) to solve the topological sorting problem?Does it matter from which node we start?

Θ(|V |+|E| ) using adjacency linked lists

28

Example: Scheduling with Task Dependencies

Choose clothes 8

Eat breakfast 3

Shower 7

Make coffee 5

Leave 4

Dress 2

Wake up 1

Make toast 6

Task Number Depends on

Wake up 1 --

Dress 2 7,8

Eat breakfast

3 5, 6

Leave 4 2, 3

Make coffee

5 1

Make toast

6 1

Shower 7 1

Choose clothes

8 1

29

The Source Removal Algorithms

Source removal algorithmBased on a direct implementation of the decrease-by-one techniques.Repeatedly identify and remove a sourcevertex, ie, a vertex that has no incoming edges, and delete it along with all the edges outgoing from it.

30

Exercises

Problem 1, Exercise 5.2Problem 1, Exercise 5.3Problem 5, Exercise 5.3