Model Prediction and Evaluation

Eric F. Wood Princeton University

DOE Workshop on Community Modeling and Long-term Predictions of the Integrated Water Cycle

Washington, DCSeptember 24-26, 2012

Workshop focus

“(the workshop participants) will be discussing challenges in predicting the water cycle and evaluating models that are used to do the predictions.”

“what may be the most important obstacles or challenges in predicting water cycle changes in the future.”

Questions relevant to this presentation

• What is the consistency among models in predicting the water cycle?

• Can a framework be developed for water and energy cycle model evaluation?

• Does adequate data exists for evaluating models, and if so where is it?

• Can we develop models with improved predictive capabilities?

Some representative examples How consistent are predictions of water cycle variables?

Estimates of water cycle variables over the pan-Arctic from observations, coupled GCMs, uncoupled LSMs, and remote sensing vary widely.

(from Rawlins, et al “Analysis of the Arctic System for Freshwater Cycle Intensification”, in review)

Seasonal Water Budgets for N. American Regions from CMIP5

Soil Moisture Precipitation Evapotranspiration Runoff

Wes

tern

NA

Cent

ral N

AEa

ster

n N

A

CMIP5 ModelsVIC off-line LSM

• Soil moisture tends to wet too early in CMIP5 models and has larger dynamic range (deeper soils, more P, more E)• Precipitation is too high in the west• Evapotranspiration is generally too high, regardless of precip • Runoff is too low and spring melt peaks too early

July 2012 initialized 7/1/2012 Aug. 2012 initialized 7/1/2012 Sept. 2012 initialized 7/1/2012

Verification of Aug. 2012 forecastVerification of Jul. 2012 forecast Verification of Sept. 2012 forecastNOAA’S forecast model doesn’t hold drought conditions. WHY?

Comparisons of re-analysis derived, and observed, water cycle variables?

The community has the belief that re-analysis derived model outputs (data sets) should be skillful since they assimilate a wide suite of observations from which the models predict water and energy cycle fluxes and states.

Are these predictions consistent among models since most of the assimilated data sets are the same?

Basins where we have been focusing large-scale analysis of model performance.

Precipitation (mm/year)

Runoff (mm/year)

Evapotranspiration (mm/year)

How consistent are re-analysis derived, and observed, water cycle variables?

The community has the belief that re-analysis derived model outputs (data sets) should be skillful since they assimilate a wide suite of observations from which the models predict water and energy cycle fluxes and states.

Are these predictions consistent among models since most of the assimilated data sets are the same?

These results and MANY OTHER STUDIES indicate significant differences among model predictions, and significant differences with observations. Is there a strategy for progress here?

How do atmosphere-land surface interactions operate and feed back onto the regional and larger scale climate system?

Recent papers:

Findell, K. L. & Eltahir, E. A. B. Atmospheric controls on soil moisture-boundary layer interactions. Part II: feedbacks within the continental United States. J. Hydrometeorol. 4, 570–583 (2003).

Koster, R. D. et al. Regions of strong coupling between soil moisture and precipitation. Science 305, 1138–1140 (2004).

Betts, AK 2004 Understanding hydrometeorology using global models” BAMS, 1673-1688, DOI:10.1175/BAMS-85-11-1673, November

Ferguson, CR and Eric F Wood. 2011. Observed Land–Atmosphere Coupling from Satellite Remote Sensing and Reanalysis, J Hydromet. 12(6):1221-1254, DOI: 10.1175/2011JHM1380.1

Ferguson, C R; Wood, EF; Vinukollu, RK, 2012. A Global Intercomparison of Modeled and Observed Land-Atmosphere Coupling J Hydromet. 13(3):749-784. DOI: 10.1175/JHM-D-11-0119.1, June.

Taylor, CM et al. 2012 Afternoon rain more likely over drier soils, Nature, doi:10.1038/nature11377

AIRS

MERRA

Fractional composition

MERRA AIRS

What do “observations” say about land-atmospheric coupling and convection?

(from Ferguson and Wood, 2011)

Afternoon convection over dry soilswet soilsUsing ASCAT/AMSR-E/CMORPH

(from Taylor et al, 2012)

Comparisons among models of their (Kendall ) correlation between land surface variables (SM: soil moisture; EF: Evaporative fraction) and measures of coupling (e.g. LCL: Lifting Condensation Level).

SM-LCL

SM-EF

EF-LCL

(Ferguson et al., 2012)

(Taylor et al., Nature 2012)

Differences in predictions of preferences for convection over wet or dry soils

20C Evaluations: Frequency of Short-Term

(4-6 month) Drought

VIC Off-line LSM

CMIP5 Models

Is there consistency on projections of global and regional drought from CMIP5?

20C Evaluations: Frequency of Long-Term (>

12 months) Drought

VIC Off-line LSMCMIP5 Models

CMIP5 Models

Is there consistency on projections of global and regional drought from CMIP5?

Correlation between Precipitation and other Land Budget Components (DJF)

Runoff Evap Soil Moist

Off-line LSM

CanESM2

CSIRO-Mk3

GISS-E2-H

GISS-E2-R

IPSL-CM5A-LR

MIROC-ESM-CHEM

Correlation between Precipitation and other Land Budget Components (JJA)

Runoff Evap Soil Moist

Off-line LSM

CanESM2

CSIRO-Mk3

GISS-E2-H

GISS-E2-R

IPSL-CM5A-LR

MIROC-ESM-CHEM

Point

Basin

Continent

Ocean

Globe

Day Month Year Decade

Temporal Scales

Spati

al S

cale

s

FluxNet Tower Data

Continental& basin scale water/ energybudgets

WV Divergence analyses

Ocean basinbudgets

Global budgets

Potential data sets and approaches

Can a framework be developed for water and energy cycle model evaluation?

Point

Basin

Continent

Ocean

Globe

Day Month Year Decade

FluxNet Tower Data

Continental& basin scale water/ energybudgets

WV Divergence analyses

Ocean basinbudgets

Global budgets

Temporal Scales

Spati

al S

cale

s

Temporal averaging

Spati

al s

ampl

ing

(MTE

)(R

epre

sent

ative

ness

)

Product validation: Some statistical issues

S

easo

nal v

aria

bilit

y

I

nter

-ann

ual v

aria

bilit

y

L

ong-

term

clim

atol

ogy

Trend analysis, Detection of climate change?

Point

Basin

Continent

Ocean

Globe

Day Month Year Decade

FluxNet Tower Data

Continental& basin scale water/ energybudgets

WV Divergence analyses

Ocean basinbudgets

Global budgets

Temporal Scales

Spati

al S

cale

s

Rn = E+H+G

Spati

al s

ampl

ing

(MTE

)(R

epre

sent

ative

ness

)

S

easo

nal v

aria

bilit

y

I

nter

-ann

ual v

aria

bilit

y

Temporal averaging

Continental and Basin budgets

Long

-ter

m c

limat

olog

y

P= E

Product validation: Some scaling challenges

Point

Basin

Continent

Ocean

Globe

Day Month Year Decade

Temporal Scales

Spati

al S

cale

s

Re

mot

e Se

nsin

g

Land

Sur

face

Mod

els

Rean

alys

is m

odel

s

FluxNet Tower Data

Continentalto basin scale water/ energybudgets

WV Divergence analyses

Ocean basinbudgets

Global budgets

E =dSa

dt-HQ + P

E = -dSl

dt+ P - D

Atmospheric Budget

River Basin Water Budget

Surface Radiation Budget

Rn = E + H + GLand Surface Models

Reanalysis ISCCP/SRB FluxNet GRACE in-situ in-situ Remote sensing (RS) RS LSM

Can we use basin water budgets to constrain ET?

Point

Basin

Continent

Ocean

Globe

Day Month Year Decade

Temporal Scales

Spati

al S

cale

s

Re

mot

e Se

nsin

g

Land

Sur

face

Mod

els

Rean

alys

is m

odel

s

FluxNet Tower Data

Continentalto basin scale water/ energybudgets

WV Divergence analyses

Ocean basinbudgets

Global budgets

H Q =dSl

dt+ D

-H Q + P = Einferred

Ocean/Continental Budgets

SSM/I GRACE in-situ + LSMQuikSCATReanalysis

dSa

dt -

Can we use continental water budgets to constrain ET?

Major strategic issues for evaluating climate models

• Need to develop consistent, global time series of water budget variables for the evaluation of global models i.e. Climate Data Records (GCOS/NOAA) or Earth System Data Records (NASA).

Some Evaluation issues:• How can we best assess the uncertainty among model

predictions/projections of the same variable?• What constraints can be applied to limit the uncertainty in the

water budget variables?• Can we develop a merged water budget, with budget closure?

(Yes, see Pan et al, J Climate, 25(9): 3191-3206 , May, 2012)

Hyper-Resolution, Global Land Surface Modeling: Are there pathways for addressing this need and will such models improve predictive capabilities?

To what extent can we improve process representation to improve predictive skill?

5

High Resolution Precipitation is needed

• Combine the spatial variability of NEXRAD radar data with the local accuracy of rain gauges.– Use the state – space linear estimation to correct the radar data

with rain gauges (Chirlin G. R. and Wood E. F., 1982).

State SpaceEstimation

0

1

2

3

4

5

6

7

8

-83.75 -83.7 -83.65 -83.6 -83.55

31.5

31.55

31.6

31.65

31.7

31.75

0

1

2

3

4

5

6

7

8

-83.75 -83.7 -83.65 -83.6 -83.55

31.5

31.55

31.6

31.65

31.7

31.75

mm

0

1

2

3

4

5

6

7

8

-83.75 -83.7 -83.65 -83.6 -83.55

31.5

31.55

31.6

31.65

31.7

31.75

0

1

2

3

4

5

6

7

8

-83.75 -83.7 -83.65 -83.6 -83.55

31.5

31.55

31.6

31.65

31.7

31.75

6

High Resolution Soil Properties are needed

• No data exists at high resolution – Average properties of different soil types

• Variability estimation– Define a linear relationship between Topographic index and % clay– Randomly sample % silt and sand– Use pedotransfer functions, to calculate hydraulic properties per grid cell.

loamy sand Ks (mm/day)

% Clay

TI

7

• Current resolutions models do not account for topography

• Data is available Globally– Hydrosheds (Lehner et al. 2008)

High Resolution Topography

1 km (30”) 500 m (15”) 90 m (3”)

-83.8 -83.7 -83.6 -83.5

31.4

31.5

31.6

31.7

31.8

Topographic Index

-83.8 -83.7 -83.6 -83.5

31.4

31.5

31.6

31.7

31.8

Topographic Index1/8⁰ Grids

Topographic Index• TOPLATS

– Only model to account for topography

– Use it to see what variables account for the most variability

Impact on including spatial variability in predicting soil moisture (from Assessment of large scale and regional scale models for application to a high resolution global

land surface model (Roundy, Chaney and Wood, AGU Fall Meeting, 2011)

% Volume Soil Moisture Realization

Julian Day (2006)

Spati

al V

aria

bilit

y (S

td)

Soil

Moi

stur

e (%

Vol

.)

Uniform + Precipitation + Topography

50 100 150 200 250 300 3500

2

4

6

8

10

12

Probes

50 100 150 200 250 300 3500

2

4

6

8

10

12

50 100 150 200 250 300 3500

2

4

6

8

10

12

50 100 150 200 250 300 3500

2

4

6

8

10

12

50 100 150 200 250 300 3500

2

4

6

8

10

12

+ PropertiesSoil

+ NoiseRandom

50 100 150 200 250 300 3500

2

4

6

8

10

12

50 100 150 200 250 300 3500

2

4

6

8

10

12

Probes

Summary

Needs and challenges:

The community must continue working with the space and data agencies (NASA, ESA, EUMETSAT, JAXA, NOAA, etc) to development of “validation” quality data records. They’ve done a pretty good job.

International programs and data centers **must** work harder to provide in-situ sets that are critical to assessment and validation (i.e. for “benchmarking”) or for merging with other data. They’ve done a poor job.

Alternative representations and algorithms for hydrologic process must get evaluated – more validation (“benchmarking”) and analysis activities are needed by the community – NEWS is making progress in this area but more $$ are needed.