CS&E and Statistics

James Berger

Duke University andStatistical and Applied Mathematical

Sciences Institute (SAMSI)

Outline

• A Glimpse of the World of Statistical Modeling in Science, Engineering and Society from the Viewpoint of a Statistician

• Bringing the CS&E and Statistics Communities Together

• Research Themes

I. An Idiosyncratic Glimpse of the World of Statistical Modeling in Science, Engineering, and Society

• Example 1: Predicting Fuel Economy Improvements

• Example 2: Understanding the Orbital Composition of Galaxies

• Example 3: Protecting Confidentiality in Government Databases, while Allowing for their Use in Research

Example 1: An early 90’s study of the potential available gain in fuel economy,

to gauge the possibility of changing CAFE

• Statistical modeling of EPA data involved– physics/engineering-based data transformations– ‘multilevel random effects’ models, accounting

for vehicle model effects, manufacturer effects, technology type, … (about 3000 parameters)

– physics/engineering knowledge of effect on vehicle performance of technology changes, necessary to implement a ‘constant performance’ condition, some from simulation.

• Prediction of the effect of technology change (highly non-linear)– was done in a Bayesian fashion;– involved thousands of 3000-dimensional integrals;– utilized Markov Chain Monte Carlo methods.

• The total estimated fuel economy gains available by 1995 and 2001 were (within 2%)– 11% and 20% (Automobile) – 8% and 16% (Truck)

(Note that legislation had proposed CAFE increases of 20% by 1995 and 40% by 2001.)

See http://www.stat.duke.edu/~berger/papers/fuel.html

Example 2: Understanding the orbital composition of galaxies

• Consider a galaxy as made of a collection of ‘rings’ of orbiting stars; each ring specified by – its location – a given velocity for the stars in the ring.

• Available data is the luminosity in each (location,velocity) slit of the galaxy;– it is measured with noise.

• Goal: find the luminosity ‘weight’ of each ‘ring’.

• Finding the weights appears to be a linearly constrained quadratic minimization problem, but– there are many local minima, with nearly the same

minimum value, so the actual minimum is unimportant– characterization of the uncertainty in the weights is

crucial, leading to identification of the computationally ‘stable’ and ‘transient’ orbits.

• A solution is to employ Bayesian analysis, leading to the posterior distribution of weights:– here, dimensions of integration are roughly equal to

the number of orbits considered;– new Markov Chain Monte Carlo methods for highly

constrained spaces are required .

Example 3: Protecting Privacy in an Electronic, Post-9/11/01 World

• Underlying Tension: Federal statistical agencies must– protect confidentiality of data (and privacy of individuals

and organizations),

– disclose information to the public, researchers, …

• Current Milieu: Sophisticated ways to break confidentiality.– Example: Linkage to external databases (many) using

powerful software tools.

• The need: equally powerful models and tools to protect confidentiality .

• Full Data: Large (e.g., 40 dimensions x 10 categories) contingency table corresponding to a categorical database; note that there are 1040 cells in the full table (but most are 0).

• To Disseminate to Researchers: Set of marginal sub-tables that maximize utility of released information subject to a risk disclosure constraint

• The difficult computational challenges include– computation via MCMC or integer programming or ??

with huge contingency tables;– optimization of the utility, subject to the constraint;– determination of the statistical utility of sub-tables.

See NISS Digital Government Project: http://www.niss.org/dg

II. Bringing the CS&E and Statistics Communities Together

• Example : Inverse problems and validation for complex computer models

• Barriers to closer association

• Mechanisms for closer association

Example: Development, Analysis and Validation of Computer Models

– Consider computer models of processes, created via applied mathematical modeling, statistical modeling, microsimulation, or other strategy.

– Collect data from the real process, to• Find unknown parameters of the computer model

(the inverse problem), and characterize uncertainty• Find inadequacies of the computer model and

suggest improvements• Predict accuracy of the computer model

x

b(x)

Illustration: Math modeling of vehicle crashes

• A finite element applied math model– 100,000 elements– developed using

LS-DYNA – 12+ hours to run

• Accelerometer data is available at differing vehicle velocities– 36 computer runs– 36 field tests

Statistical modeling of velocities as a function of time:

vfield(t) = vtrue(t) + e(t), vmodel(t) = vtrue(t) + b(t),

where e(t) is noise and b(t) is computer model bias.

Analysis: Use Bayesian analysis and Markov Chain Monte Carlo implementation to– provide estimates (with uncertainties) of unknown

coefficients in the math model, e.g., damping;– assess accuracy of predictions of the computer model

(e.g., at initial velocity v=30 mph, there is a 90% chance that the computer model prediction is within 1.5 of the true process value)

– allows prediction of key engineering quantities, such as CRITV, the airbag deployment time.

See: http://www.niss.org/technicalreports/tr128.pdf

Barriers to Bringing the CS&E and Statistics Communities Together

• To many disciplinary scientists– we are each ‘providers of tools they can use’– we are indistinguishable quantitative experts

• Program and project funding rarely encourage inclusion of both CS&E and statistical scientists.

• Our traditional application areas generally differ– CS&E tradition: physical sciences and engineering– Statistics tradition: strongest – as the statistics

discipline – in social sciences, medical sciences,…(This could be an organizational strength for the CS&E

initiative, but is a barrier at the personal level.)

Mechanisms for Bringing the CS&E and Statistics Communities Together

• Most important is simply to bring them together on interdisciplinary teams.

• Institute programs (e.g., at SAMSI), for extended cooperation– joint workshops– joint working groups

• Emphasize need for joint funding on interdisciplinary projects.

• At Universities?

Organizing and Delivering Joint CS&E and Statistics Educational Programs

At SAMSI, we– provide integrated courses, jointly taught;– provide graduate students and postdocs with

year-long exposure to joint programs;– provide 1 week outreach programs to

undergraduates and high-school teachers, and 2 week outreach programs to beginning graduate students, to introduce them to the CS&E and Statistics worlds;

– begin opening program workshops with extensive tutorials.

Research Challenges

• Statistical computational research challenges: – MCMC development and implementation– data confidentiality and large contingency tables– dealing with large data sets

• in real time• off-line

– bioinformatics, gene regulation, protein folding, …– data mining– utilizing multiscale data – data fusion, data assimilation– graphical models/causal networks– open source software environments– visualization– many many more.

• Challenges in the synthesis of statistics and development of computer modeling:– Statistical analysis in non-linear situations can

require thousands of model evaluations (e.g., using MCMC), so the ‘real’ computational problem is the product of two very intensive computational problems; this is needed for

• designing effective evaluation experiments;• estimating unknown model parameters (inverse

problem), with uncertainty evaluation;• assessing model bias and predictive capability of

the model;• detecting inadequate model components.

– Simultaneous use of statistical and applied mathematical modeling is needed for

• effective utilization of many types of data, such as – data that occurs at multiple scales;– data/models that are individual-specific.

• replacing unresolvable determinism by stochastic or statistically modeled components (parameterization)

This general area of validation of computer models should be a Grand Challenge.