Discrete Programming and Game Theory (DPG) Dynamic and ...agr/DPG/DP_slides.1-23.pdf · Discrete...

Discrete Programming and Game Theory (DPG)

Dynamic and Integer Programming (DIP)

Lecturers

◮ Andreas Grothey (Dynamic Programming)

◮ Peter Richtarik (Game Theory)

◮ Ken McKinnon (Integer Programming)

Course Secretaries

PG: Frances Reid, F.C.Reid@ed.ac.uk, 09:30–14:30, JCMB 6313UG: Katy McPhail, K.McPhail@ed.ac.uk, 13:30–17:00, JCMB 6312

Discrete Programming and Game Theory (DPG)

Dynamic and Integer Programming (DIP)

Organisation

◮ y2/3 and MSc course (OR, Stats, FMO & SES)

◮ Separate exams (in May) and assessments and tutorial classesfor UG and MSc

◮ Tutorials on Wed and Fri in weeks 3,5,7,9. Hand in theprevious Friday. Hand out 1 week before that.

◮ UG hand-in Friday 2pm. MSc students have a supervisedwrite-up session (under exam conditions) Fridays am.

◮ Tutorial hand in contributes 15% to final mark for DPG (andFMO MSc students)

◮ Tutorial hand in contributes 25% to final mark for DIP (OR,Stats & SES MSc)

◮ Lecture info + downloads on WebCT

Discrete Programming and Game Theory (DPG)

Dynamic and Integer Programming (DIP)

Tutorial Arrangements & Continuous Assessment

◮ There are tutorials in wk 3 & 5 (Wed/Fri for UG, Mon forMSc).

◮ There will be tutorial sheets with set questions.

◮ Additionally there will be two assignments.

◮ UG: hand-in wk 2 & 4, Fri 2pm (outside the Math Hub, 5thfloor JCMB)

◮ MSc: supervised write-up session, wk 2& 4, Fri 9am.

The two assignments in DP will (together) count 15%/25% of thetotal mark for the DP part of the course.

Topics Covered are

1. Dynamic Programming (DP)

2. Game Theory

3. Integer Programming (IP) (MSc only)

Example codes:

N - Notes only

L - in lectures,

S - for self study

Books (for Dynamic Programming)

1. Operations Research, Hillier and Lieberman,

2. Operations Research, Winston

Dynamic Programming

◮ Programming = Optimization6= Computer Programming

◮ Dynamic = making decisions over stages (of time)= (usually recursively)

◮ Dynamic Programming = “Recursive Optimization”

Dynamic Programming is an (computationally) “easy” techniquesfor (some, important) problems that ought to be “hard”.

Optimization

Optimization Problem

Optimization concerns finding the “best” decision from a possible(feasible) set of decisions

minx∈X

f (x), max{f (x) : x ∈ X}

◮ f (x) is the objective function

◮ X (⊆ IRn) is the feasible set

◮ X is (usually, often) defined by (in)equalities

Optimization

Optimization Problem

Optimization concerns finding the “best” decision from a possible(feasible) set of decisions

minx∈X

f (x), max{f (x) : x ∈ X}

◮ f (x) is the objective function

◮ X (⊆ IRn) is the feasible set

◮ X is (usually, often) defined by (in)equalities

Example: Linear Programming (from LPNA)

maxv1,v2v1 + 3v2

subject to 2v1 + v2 ≤ 10−v1 + v2 ≤ 7

v1, v2 ≥ 0

Easy & Hard Problems

Easy Problems

◮ Linear Programmes (LP). Can solve huge instances.(up to 106 − 109 variables)

◮ Convex Programmes. Can approximate as LPs.

Hard Problems

◮ Non-convex objective functions (can have many local maxima)

◮ Integrality constraints (disconnected decision space)

⇒ Hill climbing in fog analogy

DP and IP tackle hard problems which can have local maximawhich are not global maxima or disconnected decision spaces(integrality constraints).

Easy & Hard Problems

Uncertain data

Either:

1. Data uncertain but its probability distribution is know. (e.g.next week’s rainfall). Can extend LP, IP and DP to deal withthis.

2. Other intelligent agents with possibly conflicting goals. (e.g.union - employer negotiation). Use Game Theory.

Dynamic Programming (DP)

◮ Invented in 1950s by Richard Bellman.

◮ DP divides the original problem into subproblems and solvesthem recursively.

◮ Each subproblem is characterised by a “State”:same state <=> same subproblem.

◮ Formulating DP problems is difficult. The key thing is to spotwhat the States of a problem are – Do lots of examples.

Defining Characteristics of DP

◮ Stages: Points at which a decision is taken

◮ State: Possible “state” of the system at a stage.(=Information on which a decision is based).

◮ Recursion: Optimal decision at one state can be derived fromoptimal decisions at previous states.

Graphs

◮ A (Di)graph (or (directed) graph) is a set of vertices and a setof (di)edges connecting them (G = G(V, E))

◮ A (di)path is a continuous sequence of (di)edges, and a closed(di)path with no repeated edges is a (di)circuit.

◮ A connected (di)graph without a circuit is a tree.

◮ A digraph without a dicircuit is a directed acyclic graph orDAG.

◮ An edge is often also called an arc and a vertex is often calleda node. A digraph with values (such as lengths, times orcosts) associated with its diedges is called a network.

Graphs: Examples

Shortest & Longest Dipath Problems

Ex 1L: on blackboard/OHP

Shortest & Longest Dipath Problems

Principle of Optimality

Dynamic Programming Terminology

◮ The problem can be broken into states (vertices)

◮ At every state an optimal decision (action) has to be taken(which edge to follow).

◮ Every state has a (sub-)problem associated with it.

◮ A policy is a set of actions for every state.

◮ The optimal action in a state can be computed recursivelyfrom the optimal actions in the preceding states.

Principle of Optimality (Bellman)

In an optimal policy if any state (and the corresponding actions) isremoved, the remaining actions are still optimal for the remainingproblem.

Critical Path Method (CPM)

CPM is a longest path problem. It is a technique used in Project

Management

Consider a (fairly complicated) project. A project can be brokendown into activities (subtasks) that need to be achieved forcompletion. Assume that for each activity we know

◮ an estimate of the time needed to complete this activity

◮ a list of other activities that need to be completed before theactivity in question can be started

The task is to find the total completion time of the project.

Critical Path Method (CPM)

Critical Path Method (CPM)

Representation of project as a DAG

◮ The diedges in a DAG represent jobs which take the time given

◮ All jobs into a vertex must be complete before any job out ofthat vertex can start.

◮ Vertex z (FINISH) represents the end of the project.

Critical Path

The Critical Path is the longest path through the network

◮ Its length equals the total duration of the project

◮ Any delay of a job on the critical path results in delay of theproject

◮ Longest (Critical) Path can be found by DP.

Critical Path Method (CPM)

CPM: Activity on Arc representationEx2L: Project specified by table of jobs with duration andpredecessor

Job A B C D E F G H I J K

Duration 4 7 8 3 6 7 5 7 5 9 5Predecessors - - A A A B B B C,F D,G E,H

A

B

C

EF

H

I

J

K

D

G

Critical Path Method (CPM)

◮ The longest dipath from 1 to z is called the Critical Path

◮ Any delay in a job on the critical path delays the completionof the project

◮ The Earliest Start Time (EST) of jobs starting at a vertex isthe earliest time at which they can start if all preceding jobsstart as early as possible.

◮ The Latest Finish Time (LFT) of a job is the latest time thata job can complete if it is not to delay the completion of theproject.

◮ The slack on a job is the delay possible on that job is if is notto delay the completion of the project.