View
7
Download
0
Category
Preview:
Citation preview
CSEP Overview and Status
Max Werner!
z
T. Jordan, M. Liukis, P. Maechling, D. Schorlemmer, J. Zechar and many more
Collaboratory for the Study of Earthquake Predictability!
• CSEP goal is an infrastructure for conducting earthquake predictability experiments and research. This entails:
– Rigorous procedures for registering forecasting and prediction experiments
– Reproducible evaluations of predictability hypotheses and forecasting models
– Automated, blind, prospective testing in a standardized, controlled environment (“zero degrees of freedom”)
– Community-endorsed standards for assessing forecasts & predictions
– Experiments in a variety of tectonic environments
• Why? – Understand earthquake predictability, brick-by-brick
– Reduce controversies surrounding earthquake prediction
– Help government agencies in assessing the utility of earthquake forecasts and predictions in the context of risk reduction.
Jordan (2006)
Los Angeles
Zurich
Tokyo
Wellington
GNS Science Testing Center
ERI Testing Center
EU Testing Center SCEC
Testing Center
Testing Center
Upcoming
Beijing
China Testing Center
Collaboratory for the Study of Earthquake Predictability!Infrastructure for automated, prospective assessment of forecasting models in a variety of tectonic environments
CSEP Testing Centers
Los Angeles
Zurich
Tokyo
Wellington
GNS Science Testing Center
Japan 203 models
ERI Testing Center
Italy 48 models
EU Testing Center
California 64 models
SCEC Testing Center
Testing Center
Upcoming
Testing Region
Upcoming
Global 9 models
Beijing
China Testing Center
North-South Seismic Belt
New Zealand 85 models
CSEP Testing Regions & Testing Centers 434 models under test
in September, 2014
Collaboratory for the Study of Earthquake Predictability!
Western Pacific 16 models
Infrastructure for automated, blind, prospective assessment of forecasting models in a variety of tectonic environments
Oceanic Transform Faults 1 model
Models under Evaluation!434
Darfield
El Mayor
Tohoku
RELM Regional Earthquake Likelihood Models PPE Proximity to Past Earthquakes TripleS Simple Smoothed Seismicity EEPAS Every Earthquake a Precursor According
to Scale
STEP Short-Term Earthquake Probabilities ETAS Epidemic-Type Aftershock Sequences DBM Double Branching Model K3 Kernel-based space-time-magnitude
smoothing
Example models:
Status Updates!
• Multiplicative hybrid RELM models (California) • 3-month model evaluation (California) • 1-day model evaluation (California) • 1-year global model challenges (today) • A teaser result from the Canterbury experiment (today) • Prototype external forecast & prediction (EFP) experiment
for M8
Original models Transformation Hybrid model
€
f (λ) = exp[a + b log(1+ λ)( )c ]
order-preserving function, e.g.
D. Rhoades, M. Gerstenberger, A. Christophersen, J. Zechar, D. Schorlemmer, M. Werner & T. Jordan (2014, BSSA, accepted)
Models for SoCal
Target earthquakes: Mainshocks + Aftershocks
Some hybrids show better performance."(penalized for extra parameters)!
Also: best inf gain since 2011/11 (5 eqks. out-of-sample)
Time-series of T- and W-test results, with EEPAS-0F as the reference model.
Schneider M et al. Geophys. J. Int. 2014;198:1307-1318
© The Authors 2014. Published by Oxford University Press on behalf of The Royal Astronomical Society.
3-month model evaluation California!
• 4 new models since 10/2012 • Conan (adaptive space-time kernels) • Janus (hybrid model) • EAS (early-aftershock)
• 6 earthquakes M4.95+
• No significant differences yet
Testing region: California Forecast model: Conan
(Helmstetter & Werner, 2012)
Testing period (retrospective): 10/2012 – 7/2014 Target events: M ≥ 4.95 (6)
Conan (adaptive space-time smoothing)
Evaluating Conan in Japan (VISES)!
Jan10 Apr10 Jul10 Oct10 Jan11 Apr11 Jul11 Oct11100
101
102
103
Num
ber o
f M3.
95+
ForecastObserved
Jan10 Apr10 Jul10 Oct10 Jan11 Apr11 Jul11 Oct114
5
6
7
8
Mag
nitu
de
130˚ 132˚ 134˚ 136˚ 138˚ 140˚ 142˚ 144˚ 146˚
32˚
34˚
36˚
38˚
40˚
42˚
44˚
130˚ 132˚ 134˚ 136˚ 138˚ 140˚ 142˚ 144˚ 146˚
32˚
34˚
36˚
38˚
40˚
42˚
44˚
−4 −3 −2 −1 0
log10 expected earthquakes M3.95+ per 3 months per 0.1o 2
Retrospective comparison with extent CSEP-Japan models in progress
Werner, Tsuruoka, Yokoi, Helmstetter & Hirata
Pearson residuals for the EEPAS-0R model in the aftershock region of the El Mayor earthquake, computed for the El Mayor period and subsequent eight periods.
Schneider M et al. Geophys. J. Int. 2014;198:1307-1318
Residuals-based Evaluation "
Next-day M3.95+ experiment in CA+
Zechar, 2014
ETAS 1.0 vs. the rest
Kagan-Jackson
STEP Java
ETAS 1.1 (Zhuang)
ETAS (H&W)
(ETAS+K3)/2
ETAS (Rhoades)
K3
ETAS/PPE (Rhoades)
Information gain per earthquake
2012-10-01 to 2014-03-01 62 earthquakes M4+
Zechar, 2014
KJSS (Kagan & Jackson)
Zechar, 2014
Sub-24-hour Earthquake Predictability!
1-day models
Testing region: California Forecast model: ETAS (Werner, Helmstetter, Jackson & Kagan, 2014)
Testing period (retrospective): 1992 – 2012 Target events: M ≥ 3.95 (1396) Reference model: spatially uniform Poisson
Evaluations of Global Models!Testing region: Global M5.95+ <70km depth (high-resolution, 0.1 deg) Forecast model: SHIFT_GSRM Global Strain Rate Map & SHIFT (Bird & Kreemer, 2010) Testing period: 1/1/2014 – 8/7/2014
N-test
• 3 high-resolution (0.1 deg) models, more on the way (GEAR). • forthcoming binary format required for likelihood tests & information gains. • new models should agree on common depth and magnitude thresholds.
RETROSPECTIVE EXPERIMENT!2010-12 Canterbury, NZ
Joint SCEC/REAKT/GNS project • Improve understanding of earthquake triggering • Improve time-dependent hazard estimates • Understand influence of “real-time” data on forecasts
Participating Models!
Coulomb Steacy, Gerstenberger Cattania et al.
STEP/ETAS Helmstetter & Werner Hainzl et al.
Smoothing Helmstetter & Werner
Installed Models!
Reference Models Uniform Poisson model
Testing region: Canterbury Forecast model: CRS1
Coulom/Rate-State w/ uncertainties (Cattania et al., 2014) Testing period: From 7.1 Darfield eqk. to 2/28/2012
Coulomb/Rate-State w uncertainties
Information gain
External Forecasts & Predictions!
M8
Kossobokov, Liukis, Rierola, Zechar
Results CSEP
Example of M8 TIPS
This is an example for the TIP alarms. This alarm was released on1.1.1985.
−100 0 100
−100
−50
050
100
M8 TIP 1985a
Pos. TIPNeg. TIPNo Data
Green circles representTIPS where an alarm hasbeen set: ”Yes”
Red circles representTIPS where no alarm hasbeen issued: ”No”
Blue circles show TIPSwhere not enough datawas available. ThoseTIPS have are not beingconsidered in the scoring.
August 19, 2014 1 / 3
Rierola & Zechar, 2014
Delta R Score for fixed odd gambling
The Delta R gambling score approach has been introduced by Zhuangand Zechar in 2010.
The Delta R gambling score considers an earthquake forecast liketaking a bet in a casino. The gambler, i.e. M8 is placing a betagainst the house. For each forecast the gambler (M8) makes, heplaces one reputation point.
The house in this case is a Poisson reference model for which wecalculated historic rates based on the ANSS catalog.
The Delta R gambling score allows for unbounded (∞) winnings whilethe losses per bet cannot be more than 1 by definition.
August 19, 2014 2 / 3
Rierola & Zechar, 2014
Next steps and CSEP integration
Because of the asymmetrical behavior of the Delta R gambling scorewe believe that it is not the best measure for earthquake forecastmodels.We will therefore extend our research and score the forecasting modelusing the pari-mutuel gambling score.Currently everything is coded locally in R. We hope to be able tomove the code into CSEP by the end of this year.
0 2000 4000 6000 8000 10000
0.0
0.2
0.4
0.6
0.8
1.0
ECDF
change in reputation
fract
ion
of s
imul
atio
ns
August 19, 2014 3 / 3
Rierola & Zechar, 2014
Frac
tion
of s
imul
atio
ns
1.0
0
Pessimist M8 Poisson
returns
Summary!
• Hybrid & ensemble modeling help construct better models.
• Residuals are an interesting visual aid in evaluating models.
• 1-day model evaluation in California suggests real differences.
• 30-minute forecast group open for business.
• Global forecasts require resources and consensus.
• Interesting results from the retrospective Canterbury experiments suggest Coulomb models improving.
• EFP experiments (M8 & QuakeFinder) pose interesting questions about gambling scores.
CSEP Structure!
Forecast Model 1
Forecast Model N
Testing Procedures
Authoritative Data Source A
Authoritative Data Source B
Authoritative Eqk Catalog
Results
CSEP
Data R
egistry
…
Forecast Registry
External Forecasting Procedure
Special Data Source
Accommodation of External Forecasting
Original models Transformation Hybrid model
€
f (λ)
€
λ
D. Rhoades, M. Gerstenberger, A. Christophersen, J. Zechar, D. Schorlemmer, M. Werner & T. Jordan (2014, BSSA, accepted)
Recommended