New approaches for extreme value analysis in large-scale … · CIA World Factbook 2006. OAK RIDGE NATIONAL LABORATORY U. S. D EPARTMENT OF ENERGY Geospatial Correlation to Geospatial-Temporal

New approaches for extreme value analysis in large-scale geospatial-temporal data with applications to observed and climate-model simulated precipitation in South America

Gabriel Kuhn, Shiraj Khan, Auroop R Ganguly*

Oak Ridge National Laboratory, Oak Ridge, TN

* Presenter / Correspondence: [email protected]; 865-241-1305

2006 Fall Meeting of the American Geophysical Union, San Francisco, CA

Section: Hydrology

Session: Role of Observed Precipitation in Atmospheric and Land Surface Models I

Paper #: H32A-07

13 December, 2006

The following article has been submitted to a journal after this specific abstract was submitted to AGU 2006Kuhn, G., Khan, S., Ganguly, A.R.*, and M. Branstetter (2006): Geospatial-temporal dependence among weekly precipitation extremes with applications to observations and climate model simulations in South America, Advances in Water Resources (In Review). * Corresponding Author

OAK RIDGE NATIONAL LABORATORY

U. S. DEPARTMENT OF ENERGY

Motivation

Extreme Value Theory• Prob. (X > u): Univariate Extreme Value

Theory (EVT)

• Prob. (Y > v | X = x): Extremes and Multiple Covariates

• Prob. (Y > v | X > u): Multivariate Extreme Value Theory

• Hydrologic Observations

− Grid-based Precipitation

• Climate Model Simulations

− 1870 to Now: Past

− Now to 2100: Future

1870 Now 2100

Observations

Model Simulations

Gaps for real applications• Large-scale data: Scale up extreme

value theory for automated use

• Geospatial-temporal extremes: Extend multivariate EVT for space and time

Quantify Model Uncertainties

Generate Realistic Prediction Scenarios

Khan, S., Kuhn, G., Ganguly, A.R.*, Erickson, D.J., and G. Ostrouchov (2006): Spatio-temporal variability of daily and weekly Precipitation extremes in South America, Water Resources Research (In Review).

Kuhn, G., Khan, S., Ganguly, A.R.*, and M. Branstetter (2006): Geospatial-temporal dependence among weekly precipitation extremes with applications to observations and climate model simulations in South America, Advances in Water Resources (In Review).

Kuhn. G., Khan, S., and Ganguly, A.R.* (2006): New approaches for extreme value analysis in large-scale geospatial-temporal data with applications to observations and climate-model simulated precipitation in South America. American Geophysical Union, Fall Meeting, SFO, CA.

* Corresponding Author



Precipitation Data in South America

• Historical Precipitation− Grid-based: 1 degree spatial grids

− Daily data from January 1940 to June 2005

− NOAA data: Liebman and Allured, 2005, BAMS

• Simulated Precipitation − T85 grid: 1.4 degree over land and atmosphere

− Daily and 6-hourly data from 1940-2099

− IPCC runs from Community Climate System Model version 3 (CCSM3): Collins et al., 2005

− “A2” IPCC Scenario (PCMDI at LLNL)



Extreme Value Theory: One VariableGeneralized Pareto Distribution & Return Level

EXCEEDENCES OVER THRESHOLD: Prob. (X – u | X > u)

• T-year Return Level, RL(T)− Exceeded once every T years

− Prob. [X > RL(T)] in any year: 1/T



Population Threat

Metrics & Uncertainty

Precipitation

Extremes

Geo-Referenced Indices

for Disaster Readiness

Overview

South

America

Grid-based Precipitation

(Daily; 1o, 2.5o grids)

Sabesan, A., Abercrombie, K., Ganguly, A.R.*, Bhaduri, B.L., Bright, E.A., and P. Coleman (2006): Metrics for the comparative analysis of geospatial datasets with applications to high-resolution grid-based population data, GeoJournal (Invited: In Review).

Khan, S., Kuhn, G., Ganguly, A.R.*, Erickson, D.J., and G. Ostrouchov (2006): Spatio-temporal variability of daily and weekly Precipitation extremes in South America, Water Resources Research (In Review).

* Corresponding AuthorFuller, C.T., Sabesan, A., Khan, S., Kuhn. G., Ganguly, A.R.*, Erickson,

D., and G. Ostrouchov (2006): Quantification and visualization of the human impacts of anticipated precipitation extremes in South America. American Geophysical Union, Fall Meeting, San Francisco, CA.

Global Extent

LandScanTM Global 2004GIST Group, CSE Division, ORNL

Grid-based Population Database

(30// lat-lon)

Precipitation Grids (2005) ESRL, PS Division, NOAA

Spatial Statistics / GIS Extreme Value Theory

Geospatial Modeling

Gross Domestic ProductCIA World Factbook 2006



Geospatial Correlation to Geospatial-Temporal Dependence

• Spatial Correlation Functions (Cressie, 1993)− Captures linear correlation and works well for multivariate normal

− Spatial extensions of ACF and CCF used in time series analysis

− Spatial ACF and spatial CCF are function of spatial lags

• Kendall’s Tau (Kendall and Gibbons, 1990)− Captures linear correlation and monotonic dependence

− Function at spatial lags analogous to spatial correlation function

• Spatio-Temporal Correlation and Dependence− Relates time series at multiple spatial grids or points

− Linear: Cross-correlation among time series at multiple spatial locations

− Linear + Monotonic dependence: Kendall’s Tau for the above

• Measures for Complete Dependence Structures− Information theoretic (Mutual Information): Khan et al., 2006, GRL

− Copulas: Described in later slides



The Kendall Tau for Dependence


Kendall’s Tau: Definition

Empirical Estimator for iid Samples



Multivariate Tail DependenceMotivation

• Conditional Exceedence

− Prob. (X > u | Y > v)

− Extremes of river flows conditional on extremes of El Nino?

• Joint Exceedence

− Prob. (X > u, Y > v)

− Precipitation extremes of nearby spatial locations co-occur?

• Joint and conditional probabilities are related

• Applications

− Extreme dependence among high-risk variables?

− Are there regions where heat waves may co-occur with significant storms?



Multivariate Tail DependenceThe Concept of Copula

• Measure of dependence structure

− Joint distribution: Marginal distribution AND dependence

− Copula: Quantifies dependence from joint distributions by combining univariate distributions in a specific way

Figures courtesy Dorey & Joubert (2005):Dorey, M., and P. Joubert (2005): Modeling Copulas: An Overview, The Staple Inn Actuarial Society, 27 pages.



Multivariate Tail DependenceCopula Definition & Sklar’s Theorem

• “Multivariate CDF defined on the n-dimensional unit cube [0, 1]n such that every marginal distribution is uniform on the interval [0, 1]”− Complete information on variable dependence

− No information on marginal distributions

• C(u,0) = 0 = C(0,v); C(u,1) = u; C(1,v) = v

• Sklar’s Theorem (Bivariate): H(x,y) = C(F(x),G(y))− H(x,y): Bivariate Distribution

− F(x), G(y): Marginal DistributionsG(y)=H((-∞, ∞),y); F(x)=H(x,(-∞, ∞));

− F(x) & G(y) continuous � C is uniqueIf not, C is unique on the range of values of the marginals

Courtesy: “Wikipedia, the free encyclopedia”



Multivariate Tail DependenceTail Copula and Tail Dependence

Definition of the Tail Copula of XXXX

Pair-wise Tail Dependence Coefficients

Empirical Estimation for Elliptical CopulaKuhn, G., Khan, S., Ganguly, A.R.*, and M. Branstetter (2006): Geospatial-temporal dependence among weekly precipitation extremes with applications to observations and climate model simulations in South America, Advances in Water Resources (In Review).

Kuhn, G. (2006): On dependence and extremes, Ph.D. thesis, Munich University of Technology. (Chapter 3).

Pair-wise Geospatial-Temporal Tail Dependence

Time series at two pairs of spatial locations (grids)

Pair-wise tail dependence based on tail dependence measure



Multivariate Tail DependenceIntuition on Tail Dependence


• Location X has t-year return level, RL(t), of zx

• Location Y has t-year return level of zy

• Locations X and Y simultaneously exceed RL(t)

Simultaneous exceedence of RL(t) is a (t/λλλλxy)-year event



Multivariate Tail DependenceIntuition on Tail Dependence

• 100-year precipitation for location X is RL(100;X)

• 100-year precipitation for location Y is RL(100;Y)

• Prob. (X > RLX(100)) = 1/100

• Prob. (Y > RLY(100)) = 1/100

• Case A: Processes at locations X & Y are independent − Prob. ( (X > RLX(100)), (Y > RLY(100))) = (1/100)*(1/100)

− Simultaneous exceedence of 100-year level is a 10000-year event!!

• Case B: Processes at locations X & Y are dependent − Consider a λXY of 0.5 and t of 100

− Prob. ( (X > RLX(t)), (Y > RLY(t))) ~= λXY / t = 1/200

− Simultaneous exceedence is a mere 200-year event!!



Geospatial-Temporal Extreme DependenceLumped Temporal & Spatio-Temporal Dependence

Observations

Simulations



Geospatial-Temporal Extreme DependencePair-wise dependence: One selected point (X) with all others

Observations Simulations

Correlation on left columns

Tail Dependence on right columns



Geospatial-Temporal Extreme DependenceTrends in Pair-wise Dependence

Observations Simulations

Correlation on bottom row; Tail dependence in top row

Results using (1965-1990) data on the left and (1980-2005) on the right



Insights and Comparisons

• Average and pair-wise dependence* in simulations and observations show good match on the whole

• Differences between simulated and observed dependence*

− Average dependence* is higher for simulated data

− Average dependence* within observations show increasing trend but simulations exhibit no such trend

− Average simulated dependence* exhibit a longitudinal tilt but observed dependence* do not

− Pair-wise dependence* is higher and more spread-out in simulations compared to observations

− Pair-wise dependence* exhibits subtle differences in specific instances, both in space and in time

* Dependence �� Correlation and tail dependence



Future Research

• Statistical Methodologies: Important Gaps and Issues

• Multiple Regions: Continental US, Sahel region of Africa, etc.

• Multiple Variables: Heat Waves and Precipitation Extremes

• Long-range Extreme Dependence: El Nino and Precipitation

• Model Evaluation: Uncertainty & Degree of Belief

• Predictive Scenarios: Scenario-based Predictions

• Feedback: Model Improvements

• Decision-making: Visualization and collaboration tools



Backup Slides“Copula for Geophysicists”

• This preliminary “tutorial” on copula uses the following sources

− Our manuscript (Kuhn et al., 2006)

− Clemen, R.T., and T, Reilly (1999): Correlations and Copula for Decisions and Risk Analysis, Management Science, 45(2): 208-224.

− Li, D. X. (2000): On Default Correlation: A Copula Function Approach, The RiskMetrics Group, Working Paper Number 99-07.



Copula Tutorial – (1)

From Li (2000)

Note: This is a tutorial and not presentation of original results /

write-ups and/or results / write-ups

generated by any of the authors.

The reproduction is almost an exact copy of the reference cited.




From Clemen and

Reilly (1999)Note: This is a tutorial and not

presentation of original results /


generated by any of the authors. The reproduction is almost an

exact copy of the reference cited.




From Li (2000)








From Li (2000)







Acknowledgments and Copyrights

This research was sponsored by the SEED money funds of the Laboratory Directed Research and Development program of the Oak Ridge National Laboratory (ORNL), managed by UT-Battelle, LLC for the U. S. Department of Energy under contract no. DEAC05-00OR22725. (Title of SEED project: Multivariate dependence in climate extremes; Principal Investigator: Auroop R. Ganguly).

Auroop Ganguly would like to thank Professor Tailen Hsing of Ohio State, Dr. Rick Katz of NCAR, as well as Drs. David J Erickson III, George Ostrouchov and Marcia Branstetter of ORNL for supporting and participating in the SEED project, Drs. Budhendra L. Bhaduri, John B. Drake, and Virginia H. Dale of ORNL for their helpful comments, and Professor Sunil Saigal of the University of South Florida for his help. The reviews from all ORNL-internal and external publications or manuscripts that contributed to this research are all gratefully acknowledged.

This work was performed at the Oak Ridge National Laboratory, which is managed by UT-Battelle, LLC under Contract No. DEAC05-00OR22725. This work has been authored by employees and contractors of the U.S. Government, accordingly, the U.S. Government retains a non-exclusive, royalty-free license to publish or reproduce the published form of this contribution, or allow others to do so, for U.S Government purposes.

Documents

New approaches for extreme value analysis in large-scale … · CIA World Factbook 2006. OAK RIDGE NATIONAL LABORATORY U. S. D EPARTMENT OF ENERGY Geospatial Correlation to Geospatial-Temporal