Dynamic Programming

Our final algorithmic technique is dynamic programming.

Alice: Looking at problems upside-down can help!
Bob: (But be careful with your hat!)

Dynamic Programming

Two steps:

Find a recursive solution that involves solving the same problems many times.
Calculate bottom up to avoid recalculation.

Fibonacci Numbers

1 1 2 3 5 8 13 21 34 55 89...

(Each number is the sum of the previous two.)

Algorithm Fibonacci(n)
if n <= 1 then
  return 1
else
  return Fibonacci(n - 1) + Fibonacci(n - 2)
fi

      __5__
     /     \
    4       3
   / \     / \
  3   2   2   1
 / \ / \ / \
 2 1 1 0 1 0
/ \
1 0

We can approximate the running time with the number of additions, which we can write as a recurrence.

adds(0) = 0
adds(1) = 0
adds(n) = adds(n - 1) + adds(n - 2) + 1
   ||
   \/
adds(n) = Fibonacci(n) - 1
       ~= 0.7236 * 1.618^n - 1

A real timing:

to compute      takes
Fibonacci(40)  75.22 seconds
Fibonacci(70)   4.43 years

The Dynamic Programming Approach

Dynamic programming suggests we start at the bottom and work up.

Algorithm Fast-Fibonacci(n)
fib[0] <- 1, fib[1] <- 1
for i <- 2 to n do
  fib[i] <- fib[i - 2] + fib[i - 1]
od
return fib[n]

The number of additions is only n - 1!

to compute      took           now takes
Fibonacci(40)  75.22 seconds   2 microseconds
Fibonacci(70)   4.43 years     3 microseconds

A New Problem

Problem class MAKE-CHANGE:

Input: coin denominations d[0], d[1],..., d[n - 1], an amount a
Output: the number of coins needed to total a exactly

example:

input: the 1-cent, 3-cent, and 4-cent denominations, the amount a=6
output: 2 (we can use two 3-cent pieces, but there is no one-coin solution)

Formulating a Solution

1. Think of a recursive solution.

Make-Change(a) = 1 +    min     Make-Change(a - d[i])
                    0 <= i < n

          ___6___
         /    \  \
        5__    3  2
       / \ \  / \ |
     _4   2 1 2 0 1
    / |\  | | |   |
   3  1 0 1 0 1   0
  / \ |   |   |
 2  0 0   0   0
/ \
2 0
|
1
|
0

2. Compute bottom up.

Algorithm Make-Change(amt)
coins[0] <- 0
for a <- 1 to amt do
  coins[a] <- infinity
  for i <- 0 to n - 1 do
    if d[i] <= a and 1 + coins[a - d[i]] < coins[a] then
      coins[a] <- 1 + coins[a - d[i]]
    fi
  od
od
returns coins[amt]

This takes amt * n iterations.

Homework 2

Homework 2 could be approach with dynamic programming.

1. Think of a recursive solution.

Algorithm Calc-Spreadsheet(box)
if left side of box's formula is constant then
  left <- left hand constant
else
  left <- Calc-Spreadsheet(left hand reference)
fi

if right side of box's formula is constant then
  right <- right hand constant
else
  right <- Calc-Spreadsheet(right hand reference)
fi

if left and right are defined then
  return operation on left and right
else
  return undefined
fi

2. Compute bottom up.

Algorithm Calc-Spreadsheet
for each box i do result[i] <- undefined od
while changes are still being made do
  for each box i do
    if result[i] = undefined then
      left <- current left side
      right <- current right side
      if left != undefined and right != undefined then
        result[i] <- operation on left and right
      fi
    fi
  od
od

Graph Paths

Problem class ALL-PAIRS-PATHS:

Input: graph (V, E), edge distances d: E->R+.
Output: length p[s,t] of shortest path from s to t, for all pairs of vertices.

example:

    1
  2---4
3 |\_ | 1
  | 6\|
  1---3
    2

output:

        to
       1 2 3 4
      +-------
     1|0 3 2 3
from 2|3 0 2 1
     3|2 2 0 1
     4|3 1 1 0

The Recursive Solution

We define the following quantity:

 (k)    length of shortest path between s and t only passing through
p     = vertices 1, 2,..., k in between.
 s, t

We can calculate p[s,t]^(k) by recursive calls to compute p[u,v]^(k-1):

 (k)           (k - 1)   (k - 1)    (k - 1)
p     = min { p       , p        + p        }
 s, t          s, t      s, k       k, t

This is because the shortest path only through 1...k either passes through k or it doesn't. If the path doesn't, the first term will be the minimum. If it does, then the path will go from s to k only through 1...k-1 and from k to t only through 1...k-1, and so the second term will hold. The path will never go through k more than once, since then we could remove the loop involving k.

The Bottom-Up Solution

 (0)       d(s, t)   if (s, t) is an edge in the graph
p     <- {
 s, t      infinity  otherwise
for k <- 1 to n do
  for s <- 1 to n do
    for t <- 1 to n do
        (k)           (k - 1)   (k - 1)    (k - 1)
       p     = min { p       , p        + p        }
        s, t          s, t      s, k       k, t
    od
  od
od
        (n)
return p

Example: (@ denotes infinity here.)

    1
  2---4
3 |\_ | 1
  | 6\|
  1---3
    2


      0 3 2 @
 (0)  3 0 6 1
p   : 2 6 0 1
      @ 1 1 0

      0 3 2 @
 (1)  3 0 5 1
p   : 2 5 0 1
      @ 1 1 0

      0 3 2 4
 (2)  3 0 5 1
p   : 2 5 0 1
      4 1 1 0

      0 3 2 3
 (3)  3 0 5 1
p   : 2 5 0 1
      3 1 1 0

      0 3 2 3
 (4)  3 0 2 1
p   : 2 2 0 1
      3 1 1 0