Dynamic Programming(DP)

Simpler, Better, Faster

Carl Hultquist

Rough definition

• Dynamic programming is:• Breaking a problem up into smaller sub-

problems• By finding the optimal solution to these smaller

sub-problems, being able to find the solution to the bigger problem

• Dynamic programming is not:• A type of programming language (like

declarative and imperative)

Learn by example: Fibonacci numbers• We all know the Fibonacci numbers

1 1 2 3 5 8 13…and that we can write a neat definition for these as:

f(0) = 1f(1) = 1f(n) = f(n – 2) + f(n – 1) for n>1

Fibonacci in code

int f(int n){

if (n < 2)return 1;

elsereturn f(n – 2) + f(n – 1);

}

Fibonacci and big-O

• So we have an algorithm for finding f(n), but is it any good? What’s the big-O for finding f(n)?

Answer: O(f(n))

So… is this good? Is it bad?

Recursive Fibonacci: bad

• As an indication, f(1000)=1,318,412,525

• Surely we can do better…

The trick: overlapping subproblems• For some n, f(n)=f(n-2)+f(n-1)

Now f(n-1)=f(n-3)+f(n-2)

Note the common f(n-2) – so to calculate the value of f(n), we actually calculate f(n-2) twice. Doing this using our recursive program is wasteful – we should only need to work it out once!!!

A better Fibonacci

#define MAX_N 10000int f[MAX_N];

int fib(int n){

f[0] = f[1] = 1;for (int i = 2; i <= n; i++)

f[i] = f[i – 2] + f[i – 1];return f[n];

}

Coin counting

• Yes, today’s problem was a DP. So we want to make change of value M, we have N coin denominations, and their values are Vi for i=1…N. Now we can write the solution like this:

coins(M) = min{coins(M – V1), coins(M – V2), …, coins(M – VN)} + 1

Now, doesn’t that look just a little bit like the Fibonacci definition? ;-)

A better solution…

int N, M;int V[N];int coins[M + 1];

coins[0] = 0;for (int i = 1; i <= M; i++){

int best = M;for (int j = 0; j < N; j++)

if (V[j] <= i && coins[i – V[j]] + 1 < best)best = coins[i – V[j]] + 1;

coins[i] = best;}

Checking for valid states

int N, M;int V[N];int coins[M + 1];

set(coins[0], coins[M], -1);coins[0] = 0;for (int i = 1; i <= M; i++){

int best = M;for (int j = 0; j < N; j++)

if (V[j] <= i && coins[i – V[j]] != -1 && coins[i – V[j]] + 1 < best)

best = coins[i – V[j]] + 1;coins[i] = best;

}

An alternative coin solution: memoizationint N, M;int V[N];int cache[M + 1];set(cache[0], cache[M], -1);

int coins(int amount){

if (cache[amount] == -1){

int best = M;for (int i = 0; i < N; i++)

if (V[i] <= amount && coins(amount) + 1 < best)best = coins(amount) + 1;

cache[amount] = best;}return cache[amount];

}

Two approaches to DP

• Top-down: recursion + memoization. Easier to code, but may have greater memory requirements. Also has extra stack + function call overhead.

• Bottom-up: iterative, solves subproblems from the smallest one up. Sometimes harder to code and/or work out exactly what’s going on, but more efficient.

Backtracking

• Some DP problems just call for the value of the best answer (like today’s problem).

• Others make your life harder: they ask for the “path” to the solution.• Today’s problem could have done this

by asking you to output the actual coins used

Backtracking (2)

• This usually isn’t too hard: in the same way that you have an array to store your best solution, you also keep an array indicating how you got there.• For the coins problem, this array can simply

store the last coin used to reach each total• You can then “backtrack” by subtracting the

coin value from the total, and look at the next coin that was needed. Repeat until you hit 0

Coins with backtrack

int N, M;int V[N];int coins[M + 1];int coinUsed[M + 1];

coins[0] = 0;for (int i = 1; i <= M; i++){

int best = M;int coin = -1;for (int j = 0; j < N; j++)

if (V[j] <= i && coins[i – V[j]] + 1 < best){

best = coins[i – V[j]] + 1;coin = j;

}coins[i] = best;coinUsed[i] = coin;

}

Higher dimensions

• So far, we’ve just seen 1D DP problems• At the IOI, 2D (or higher!) problems crop

up often• Consider this one: given a NxN grid of

numbers and M co-ordinate pairs (a,b);(c,d), find the sum of the values in the grid for each of the rectangles with top-left co-ordinate (a,b) and bottom-right co-ordinate (c,d).

The naïve approach

• For each of the M rectangles, loop over the rectangle and sum up the values. This has O(N2M).

• With DP, we can instead get O(N2+M) which will usually be better (unless M is huge)

Getting clever…

• Suppose we can compute and store the sum of all rectangles with top-left co-ordinate (1,1). Let’s stash these in a 2D array called sum[][].

• Then, to work out the sum of a rectangle from (a,b) to (c,d), we can work it out using our sum array like this:

sum(a,b,c,d) = sum[c][d] – sum[c][b-1] – sum[a-1][d] + sum[a-1][b-1]

Cool, now how do we create our sum array?• Suppose the grid values are in an array

called grid[][]. To work out sum[i][j], we just need to see that it can be calculated as:

sum[i][i] = sum[i][j-1] +sum[i-1][j] –

sum[i-1][j-1] +grid[i][j]

… which we can do with a quick 2D loopsum[0][0] = 0;sum[0][j] = 0; sum[i][0] = 0;for (int i = 1; i <= N; i++)

for (int j = 1; j <= N; j++)sum[i][j] = sum[i-1][j] +

sum[i][j-1] -sum[i-1][j-1] +grid[i][j];

Common applications

• Longest common subsequence problem• Floyd’s all-pairs shortest path• Knapsack problem• Duckworth-Lewis method!

… amongst many others. See:http://en.wikipedia.org/wiki/Dynamic_Programming

Questions?