Bayesian Analysis of Stochastic System Dynamicsstaff.utia.cas.cz/kulhavy/sds07s.pdfWhy to study...

Bayesian Analysis of Bayesian Analysis of Stochastic System Stochastic System

DynamicsDynamics

Rudolf KulhavýRudolf Kulhavý

Why to study stochastic systems?Why to study stochastic systems?

� Dynamic modeling of the overall performance of

– value chains

– value networks

– virtual enterprisesValueadded

Valueadded

Price

Price

CC

AA

BB

Value added

Density

Why to study stochastic systems?Why to study stochastic systems?

� Dynamic modeling of the overall performance of

– value chains

– value networks

– virtual enterprises

� Estimation of probabilitiesof critical events or specificquantiles of randomvariables

Valueadded

Valueadded

Price

Price

CC

AA

BB

Value added

Density

Assets – Liabilities

Probabilitydensityfunction

Dynamic model ofassets & liabilities

Insolvencyprobability

Stochastic dynamic modelStochastic dynamic model

Generalization B

Conditionalprobability

density functions

Generalization A

Sampling period

State andmeasurement

“noise”

Discrete-timevalues

Timeindex

Stochastic dynamic modelStochastic dynamic model

Stochasticdifferential equationrepresentation

Conditional probability

representation

Markov chainMarkov chain

States… …

Controlled Markov chainControlled Markov chain

Exogenous inputs

States… …

Partially observed, controlled Markov Partially observed, controlled Markov

chainchain

Exogenous inputs

Measurements

States… …

Partially observed, controlled Markov Partially observed, controlled Markov

chain, with unknown parameterschain, with unknown parameters

… …

Parameters

Inputs

States

Measurements

Partially observed, controlled Markov Partially observed, controlled Markov

chain, with unknown parameterschain, with unknown parameters

… …

Parameters

Inputs

States

Measurements

Unknown model parameters can be Unknown model parameters can be

treated as extra statestreated as extra states

The augmentation of the state vector

� increases the dimensionality of the problem (and, thereby, uncertainty of the original states);

� adds additional nonlinearities.

On the other hand, it allows for explicit modelingof parameter variations.

Summary of modelSummary of model

Exogenousinputs

Measurements

States andparameters

…

Estimation of states (and parameters)Estimation of states (and parameters)

Time update

Measurement updateObservations

Past data sequences

QuantificationQuantification

of all uncertaintyof all uncertainty

via via probabilityprobability

BayesianBayesianinferenceinference

Functional recursionsFunctional recursions

Time updateTime update

Measurement updateMeasurement update

Likelihood PriorPosterior

Transitionprobabibility

PosteriorNext-stepprior

Product ruleProduct rule

Sum ruleSum rule

ProbabilityTheory

Sequential Monte Carlo approximationSequential Monte Carlo approximation

Time update

Measurement update

Replacing probabilities with samples

Particle filter (step 1)Particle filter (step 1)

Particle filter (step 2)Particle filter (step 2)

Data updateData update

Likelihood

pdf

Posterior

stateAlgorithm:

Resample from posterior samples with probabilities proportional to the likelihood values, then draw a sample from the corres-ponding kernel

density

pdf Replace posterior w/ samples

state

Likelihood

pdf Replace samples w/ smoothkernels

state

Likelihood

No need for explicit sampling, except Step 1

Samples come from the preced-ing iteration step

Particle filter (steps 3, 4)Particle filter (steps 3, 4)

Time updateTime update

pdf

Algorithm:

Pick up randomly one of the posterior samples, then draw a new sample from the corresponding transition probability density function

Particle filter has Particle filter has

been applied successbeen applied success--

fully in many areasfully in many areas

� Automated target recognitionAnuj Srivastava, Michael Miller , Ulf Grenander

� Bayesian networksDaphne Koller; Kevin Murphy

� Computational anatomyUlf Grenander, Michael Miller

� Mobile roboticsDieter Fox, Wolfram Burgard, Sebastian Thrun

� Neural networksNando de Freitas

� Signal processingPetar Djurić

� Tracking and guidanceDavid Salmond, Neil Gordon

� Visual shape and motionAndrew Blake, Michael Isard, John MacCormick

Illustrative exampleIllustrative example

� Consider a service company whose economic results depend critically on the performance of both the sales and service staff.

� The stocks of Sales Capacity and Service Capacityare measured in multiples of full-time equivalents (FTE) of an average sales or service person.

– This relates the labor capacity to the total performance of a team rather than the number of physical persons.

– Thus, hiring an additional person can increase the stock by more or less than one, depending on the actual person’s productivity.

Model structureModel structure

SimulationSimulation

resultsresults

estimate

EstimationEstimation

resultsresults

estimate

measurement

Model comparisonModel comparison

Posterior probability functionPosterior probability function

Model class index

Predictive density functionPredictive density function

Model class index

Approximate model comparisonApproximate model comparison

Monte Carlo approximationMonte Carlo approximation

Samples from posterior pdfSamples from posterior pdf

ReverendThomas BayesPierre-Simon,

Marquis de Laplace

So, what has So, what has Bayesian InferenceBayesian Inference to do to do

with System Dynamics?with System Dynamics?

Jay Wright Forrester

System Dynamics x Bayesian InferenceSystem Dynamics x Bayesian Inference

� System dynamics provides the modeler with practical methodology for convert-ing prior information into a dynamic model structure (highly informative priors).

Bayesian inference

� gives precise meaning to all modeling concepts;

� yields a coherent frame-work for consistently up-dating the prior state of knowledge with numerical evidence at hand;

� captures and combines all manifestations of uncertainty (stochastic fluctuations, measurement errors, unknown model parameters, unknown model structure).

Theoretical modelsWhite-box models

Phenomenological modelsGrey-box models

Empirical modelsBlack-box models

Problemoriented

Structurefocused

SDfocus

“[T]here is a loose connection between simplicity and plausibility, because the more complicated a set of possible hypotheses, the larger the manifold of conceivable alternatives, and so the smaller must be the prior probability of any particular hypothesis in the set.”

Edwin Jaynes, Probability Theory: The Logic of Science

Thus, among models of comparable predictive power, Bayesian inference assigns higher posterior probability to “simpler” ones.

Occam’s razorOccam’s razor

The amount of prior probability contained in the high likelihood region of parameter space

The maximum likelihood value

ConclusionConclusion

� The progress made in sequential Monte Carlo methods has made Bayesian inference an attractive option for system dynamics modeling, especially for problems where quantification of the state (and parameter) uncertainty is critical.