Dynamic ProgrammingMin Edit Distance

Longest Increasing SubsequenceClimbing Stairs

Minimum Path Sum

Min Edit Distance• Given two words X and Y, find the minimum number of steps required

to convert X to Y.

• You have the following 3 operations permitted on a word:• a) Insert a character• b) Delete a character• c) Replace a character

Min Edit Distance• For two strings• X of length n• Y of length m

• We define D(i,j)• the minimum edit distance between X[1..i] and Y[1..j]

• i.e., the first i characters of X and the first j characters of Y• the min edit distance between X and Y is thus D(n, m)

Min Edit Distance• Initialization• D(i, 0) = i

• converting X[1..i] to Y[1..0] (empty string) requires at least i Delete steps• D(0, j) = j

• converting X[1..0] (empty string) to Y[1..j] requires at least j Insert steps

• Recurrent Relation (i,j>0)• D(i, j) = min(

• D(i – 1, j) + 1,• D(i, j – 1) + 1,• D(i – 1, j – 1) + (1 if X(i) != X(j), or 0 if X(i) == X(j))

• )

Min Edit Distance• Recurrent Relation• D(i, j) = the minimum value of the three• D(i – 1, j) + 1

• Converting X[1..i] to Y[1..j] by Delete step• D(i, j – 1) + 1

• Converting X[1..i] to Y[1..j] by Insert step• D(i – 1, j – 1) + (1 if X(i) != X(j), or 0 if X(i) == X(j))

• Converting X[1..i] to Y[1..j] by Replace step if X(i) != X(j)• Or, do nothing if X(i) == X(j)

Min Edit Distance• Initialization• Recurrent Relation• Termination• D(n, m) is the minimum edit distance

Min Edit Distance• How to Implement?• Top-down: Recursion• Bottom-up: Dynamic programming

Min Edit Distance• Top-down solution

• int MED(string X, string Y, int i, int j)• if i == 0 return j• if j == 0 return i

• min_val = MED(X, Y, i – 1, j – 1)• if X.charAt(i-1) != Y.charAt(j-1):

• min_val += 1

• A = MED(X, Y, i – 1, j) + 1• B = MED(X, Y, i, j – 1) + 1• if a < min_val: min_val = a• if b < min_val: min_val = b

• return min_val

Min Edit Distance• Top-down solution• The time complexity of this solution is exponential• In worst case, we may end up doing O(3^m) operations

• The worst case happens when none of characters of two strings match

• Worst case example• X=“abc”• Y=“xyz"

Min Edit Distance• Top-down solution• Worst case example

• X=“abc”• Y=“xyz"

Min Edit Distance• Dynamic Programming

• Bottom-up solution

• int MED(String X, String Y)• m = int[X.length + 1][Y.length + 1]• for i = 0 to X.length + 1:

• for j = 0 to Y.length + 1:• if i == 0: m[0][j] = j• else if j == 0: m[i][0] = i• else:

• min_val = m[i – 1][j – 1]• if X.charAt(i – 1) != Y.charAt(j – 1):

• min_val += 1• if m[i – 1][j] + 1 < min_val: min_val = m[i – 1][j] + 1• if m[i][j – 1] + 1 < min_val: min_val = m[i][j – 1] + 1• m[i][j] = min_val

• return m[X.length][Y.length]

Min Edit Distance• Example• X=“PARK”• Y=“SPAKE”

Min Edit Distance• Example• X=“PARK”• Y=“SPAKE”

Min Edit Distance• Question• How to trace the solution?• R(i, j) = {“R”, “B”, “R-B”}

Longest Increasing Subsequence• For example,• Given K=[10, 9, 2, 5, 3, 7, 101, 18],• The longest increasing subsequence is [2, 3, 7, 101], therefore the

length is 4. Note that there may be more than one LIS combination, it is only necessary for you to return the length.

• We define F(i)• The length of the longest increasing subsequence ending with K(i)• The length of the longest increasing subsequence thus is the maximum value

in array F

Longest Increasing Subsequence• Initialization• F(i) = 1, i=1..n

• Recurrent Relation (0<=j<i)• F(i) = max (

• F(j) + 1, if K(i)>K(j)• )

Longest Increasing Subsequence• LIT(int [] K)

• F = int[K.length]• for i = 0 to K.length:

• F[i] = 1

• for i = 1 to K.length:• for j = 0 to i:

• if K[j] < K[i] and F[j] + 1 > F[i]:• F[i] = F[j] + 1

• int max_l = 0• for i = 0 to F.length:

• if F[i] > max_l:• max_l = F[i]

• return max_l

Climbing Stairs (Fibonacci Sequence)• You are climbing a stair case. It takes n steps to reach to the top.• Each time you can either climb 1 or 2 steps. In how many distinct

ways can you climb to the top?

• We define F(i)• the number of distinct ways we can climb to the i-th step

Climbing Stairs (Fibonacci Sequence)• Initialization

• F(1) = 1• F(2) = 2

• Recurrent Relation (i>2)• F(i) = F(i – 1) + F(i – 2)

• int climbStairs(int n)• if n == 1: return 1• if n == 2: return 2• int a = 1, b = 2;• for i = 2 to n:

• b = a + b• a = b – a

• return b

Minimum Path Sum (Practice)• Given a n x m grid filled(G) with non-negative numbers, find a path

from top left to bottom right which minimizes the sum of all numbers along its path.• Note: You can only move either down or right at any point in time.

Minimum Path Sum• Given a n x m grid filled(G) with non-negative numbers, find a path

from top left to bottom right which minimizes the sum of all numbers along its path.• Note: You can only move either down or right at any point in time.

• We define K(i, j)• The minimum path sum along the path from G(0, 0) to G(i, j) (0 <=i<n,

0<=j<m)• The minimum path sum thus is K(n - 1, m - 1)

Minimum Path Sum• Initialization• K(0, 0) = G(0, 0)• K(0, j) = sum(G(0, 0)...G(0, j)), (0 < j < m)• K(i, 0) = sum(G(0, 0)…G(i, 0)), (0 < i < n)

• Recurrent Relation (0 < i < n, 0 < j < m)• K(i, j) = min (

• K(i – 1, j) + G(i, j),• K(i, j – 1) + G(i, j)

• )

Minimum Path Sum• int MPS(int [][] G):

• n = G.rows, m = G.cols• K = int[n][m]• K[0][0] = G[0][0]

• for i = 1 to n:• K[i][0] = K[i – 1][0] + G[i][0]

• for j = 1 to m:• K[0][j] = k[0][j – 1] + G[0][j]

• for i = 1 to n:• for j = 1 to m:

• K[i][j] = min(K[i – 1][j], K[i][j – 1]) + G[i][j]

• return K[n – 1][m – 1]

Conclusion• Dynamic programming is a method for solving a complex problem by • breaking it down into a collection of simpler subproblems• solving each of those subproblems just once, and storing their solutions -

ideally, using a memory-based data structure• The next time the same subproblem occurs, instead of recomputing its

solution, one simply looks up the previously computed solution, thereby saving computation time at the expense of a (hopefully) modest expenditure in storage space.