Upload
adia
View
38
Download
0
Embed Size (px)
DESCRIPTION
Metrics for Model Skill Assessment. Model Error time series (model-data misfit): ME(i) = model - data Total Root-Mean-Square Error: RMS_Total RMS_Total 2 = RMS_Bias 2 + RMS_Variability 2 RMS_Bias = difference between means RMS_Variability (centered pattern RMS) = - PowerPoint PPT Presentation
Citation preview
Metrics for Model Skill Assessment
• Model Error time series (model-data misfit):– ME(i) = model - data
• Total Root-Mean-Square Error: RMS_Total
• RMS_Total2 = RMS_Bias2 + RMS_Variability2
– RMS_Bias = difference between means– RMS_Variability (centered pattern RMS) =
mean [difference of deviations from mean]
• RMS_Variability: Correlation, Amplitude --> Taylor diagram– Amplitude of deviations– Correlation of deviations
€
RMS _Total =1
NME(i)2
i=1
N
∑ ⎛
⎝ ⎜
⎞
⎠ ⎟
1/ 2
Taylor Plot
Graphical relationship between time series based on four statistics: 1) Overall Mean
Bias2) Seasonal Variance
Standard Deviation3) Timing/Phase
Correlation Coefficient4) Root Mean Square Error
Centered RMS Distance (RMS_V)
Taylor Diagram Example
old Run 751
new Run 801
Target Diagram as Skill Assessment Tool
RMS_T2 = RMS_B2 + RMS_V2
model-data misfit = variability in data
model-data misfit = error in data
SAB SST climatology
SST
Chlorophyll
Chlorophyll
Summary
Taylor & Target diagrams are two complimentary ways
of assessing model skill
- Taylor: Correlation of variability Amplitude of variability (Bias)
- Target: Total RMS Relative bias and variability components
SST• Correlation: ~0.9 always
satellite, in situ, 2004, climatology• Amplitude of variability: good
especially for satellite 2004 comparisonsunderestimate in FL, GAoverestimate everywhere north of SC
• Bias: low underestimate in SAB in climatology, better using 2004
• RMS_bias ≈ RMS_variability
MLD• Correlation: always positive
Higher in MAB (.8) than SAB (.5)Higher in outer SAB (>.6) than inner SAB (<.4)
• Amplitude of variability: overestimate variabilityExcept for MAB Outer shelf
• Bias: generally low typically overestimate (FL inner, DE outer)occasionally underestimate (FL, GA outer, MAB outer)
Summary (cont.)
Surface chlorophyll - much greater challenge!
• Correlation: -0.6 to 0.9 (same for Clim and 2004)lower off NC, SC, NYHigher off FL, DE, NJ
• Amplitude of variability: so-so (worse for 2004 in SAB)underestimate in SABoverestimate in MAB
• Bias: large negative bias everywhere underestimate in GA, SC (benthic production?)underestimate on inner MAB shelf
• RMS_bias >> RMS_variability
• Little correlation between where MLD/SST is modeled well (poorly) and where chlorophyll is modeled well (poorly)
Summary (cont.)
Use these Taylor/Target diagrams to compare runs
• With/without tides• With/without DOM
Plot other quantities: kPAR, productivity, oxygen, salinity
Examine other regions: Gulf of MaineGulf Stream/Sargasso
Use these for the OCRT meeting?Use these for the Oceanography article?
Future Work