Upload
adrian-richards
View
222
Download
0
Embed Size (px)
Citation preview
Satellite Analysis Branch Satellite Analysis Branch 2008 Year in Review2008 Year in Review
Anthony E. Salemi – Satellite Analysis Branch (SAB)
Gregory Gallina, Keith Liddick, Alan Schwartz, Matthew Seybold, Michelle Spampata, and Michael Turk
METSAT Conference - 28 April 2009
Milestones• Dec 2007 – SSD Management affirms findings of
ATET—ADT unsuitable replacement for SAB subjective estimates
• Dec 2007 – Bulletin format changed to reflect that of JTWC and CPHC
• Summer 2008 – SAB disseminates near RT microwave (MW) fixes as experimental product
• Fall 2008 – SAB incorporates MW fixes in text bulletins for Eastern and Southern Hemispheres
• Jan 2009 – SAB offers MW analysis on case-by-case basis to JTWC, TCWCs and RSMCs by request
• Mar 2009 – SAB providing Dvorak and MW fixes in ATCF format to TPC – Awaiting Testing by TPC
Milestones: Microwave FixesMilestones: Microwave Fixeshttp://www.ssd.noaa.gov/PS/TROP/2008/midata/archive.html
Sample MW History FileSample MW History File
Milestones: New Bulletin FormatMilestones: New Bulletin Format
WWPN20 KNES 160945 A. 25W (HAISHEN) B. 16/0857Z C. 27.5N D. 153.5E E. THREE/MTSAT F. T3.0/3.0/D1.0/24HRS G. IR/EIR/VIS/AMSU/SSMIS/WINDSAT H. REMARKS...CONVECTION MEASURES 4/10 RESULTING IN A DT OF
2.5. PT AGREES WHILE MET IS 3.0. FT BASED ON MET...ADT CI OF 2.9 AND 0739Z WINDSAT IMAGE DEPICTING A PARTIAL EYEWALL. ~TURK
I. ADDL POSITIONS 16/0324Z 26.9N 151.3E AMSU 16/0654Z 27.4N 152.4E SSMIS 16/0739Z 27.5N 153.2E WINDSAT
695
6
459
147
449
229
652
612
37
597
147
650
270
781
515
15
481
123
440
192
843
507
146
514
129
518
183
854
659
12
580
109
648
226
815
673
6
527
173
530
137
1054
912
76
463
200
535
247
799
511
112
716
186
544
273
992
594
60
443
281
707
290
832
756
63
624
236
714
177
812
643
53
540
173
574
222
843
0
500
1000
1500
2000
2500
3000
3500N
um
ber
of
Cla
ssif
cati
on
s
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 10-yrAvg
SAB Dvorak Classifications
Atlantic CPAC EPAC NIO SIO SPAC WPACG.Galllina SAB 01/01/09
3207
3382
2637
3094
2609
2851
30493100
32323334
3050
Validation Methodology•Intensity
•Use Atkinson-Holliday wind/pressure relationship
•Ground Truth consists of, in order of importance:
•Recon
•Other in situ observations (ship/surface reports)
•SAB CI number (when validating experimental techniques)
•Ground Truth used to stratify results by intensity
•Position
•Use recon in Atlantic
•Microwave Analysis (within 90 minutes) elsewhere
SAB Performance 1995-2008 As Compared to Recon
35
54
40 26
35
21
46
44
30 33
42
33
22
33
8481
74
8987
6983
86
74
77
68
80
87
78
0
10
20
30
40
50
60
70
80
90
100
95 96 97 98 99 2000 01 02 03 04 05 06 07 08
Year
Per
cen
tag
e
Within 1/4 T-no Within 3/4 T-no
1995-2007 Average: 35% within 1/4 T-no; 80% within 3/4 T-no
M Turk SAB 9/20/08
E Hem Intensity Errors 2004-2008Based on ground truth observations within 3 hrs
1
5
9
26
23
9
7
1 1
0
5
10
15
20
25
30
< -2.0 -2.0 -1.5 -1.0 -0.5 0 0.5 1.0 1.5 2.0 > 2.0
Difference in T-number
Nu
mb
er o
f O
ccu
rren
ces
N=82A Schwartz 3/22/09
32% within 1/4 T-no 71% within 3/4 T-no
NIntensity Parameter % w/in 1/4 T-no % w/in 3/4 T-no
Tropical Depression CI number 3 0.0 100.0Pressure 0.0 100.0
Tropical Storm CI number 31 32.3 67.7Pressure 19.4 54.8
Category1 & 2 CI number 37 32.4 62.2Pressure 18.9 45.9
Category 3+ CI number 11 36.4 100.0Pressure 0.0 36.4
Overall CI number 82 31.7 70.7Pressure 17.1 50.0
SAB-Observed
E Hem Intensity Errors 2004-2008
SAB performance based on Intensity
CIMSS SATCON vs SAB in 2008: A Homogeneous Comparison
Closest observation within 3 hours of each other
0
36
83
28
7
25
45
29
2 1 00220 00
42
0
10
20
30
40
50
60
70
80
90
-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0
Differences in T-no.
Nu
mb
er
of
Occu
rren
ces
CIMSS SATCON - Recon
SAB - Recon
N = 151M.Spampata 12/08
Underestimated intensity Overestimated intensity
CIMSS SATCON 55% accuracy
97% within .7 T-no.
SAB Classifications30% accuracy
77% within .7 T-no.
RSMC ADT Declared Operational Apr ‘09
• RSMC Nadi• TCWC in Perth, Darwin and Brisbane• RSMC La Reunion• TCWC Jakarta & Port Moresby (presumably, but
untested)
Northern Hemisphere• RSMC Tokyo (untested) and New Delhi (work in
progress)
http://www.ssd.noaa.gov/PS/TROP/adt.html***ADT using JTWC forecasts are on same page***
JT ADT Agrees Better with SABRSMC vs JTWC ADT (v7.2.3)
A Homogeneous Comparison of CI Number Differences With SAB for 2008-2009 SH Systems
2
11
3735
39
1
6
10
20
46 46
10
1
3
13
33
0
5
10
15
20
25
30
35
40
45
50
< -2.0 -2.0 -1.5 -1.0 -0.5 0 0.5 1.0 1.5 2.0 > 2.0
Differences (ADT-SAB) in T-number
Nu
mb
er
of
Occ
urr
ence
s
RSMC JTWC
M Turk SAB N=143
RSMC
25% within 1/4 T-num of SAB78% within 3/4 T-num of SAB
JTWC
32% within 1/4 T-num of SAB78% within 3/4 T-num of SAB
RSMC ADT Posits Somewhat BetterRSMC vs JTWC ADT (v7.2.3)
A Homogeneous Comparison Against MW Data for 2008-2009 S Hem Systems
42
10
45
2 2 2
15
30
9
45
7
3
13
23
13
3
2323
0
5
10
15
20
25
30
35
40
45
0s 10s 20s 30s 40s 50s 60s 70s 80s 90s 100+
Position Difference (nmi)
Nu
mb
er
of
Oc
cu
rre
nc
es
RSMC JTWC
M Turk SAB 4/2/09 N=143
RSMC
Average: 31 nmi66% LT 30 nmi (PCN 1)77% LT 40 nmi (PCN 3)
JTWC
Average: 32 nmi57% LT 30 nmi (PCN 1)77% LT 40 nmi (PCN 3)
Results by Intensity%Agreement %Desirable %Agreement %Desirable
CI number 14.3 71.4 0.0 66.7Pressure 23.8 85.7 19.0 85.7
18 Position (nmi)CI number 37.7 91.3 53.6 97.1Pressure 27.5 71.0 33.3 81.2
57 Position (nmi)CI number 7.4 55.6 3.7 44.4Pressure 3.7 11.1 3.7 11.1
24 Position (nmi)CI number 15.4 69.2 30.8 73.1Pressure 11.5 15.4 7.7 30.8
20 Position (nmi)
CI number 24.5 77.6 32.2 78.3Pressure 19.6 51.7 21.0 59.4
119 Position (nmi)
Position differences based on MW analysis NOT SAB position. Since MWdata not always available, N is less than that used in intensity estimates
22.0 32.0
36.8 36.5
16.7 16.5
JTWC ADT-SAB
143
69
27
31.2 32.4
42.1 37.4
21
Overall
26MajorHcn
TropicalStorm
Category1 & 2 Hcn
%Agreement: % of estimates within 1/4 T-no or 2.5 mb of SAB estimate%Desirable: % of estimates within 3/4 T-no or 7.5 mb of SAB estimate
NRSMC ADT-SAB
TropicalDepressi
on
Intensity Parameter
Results by RSMC
%Agreement %Desirable %Agreement %Desirable
CI number 37.8 86.5 45.9 89.2Pressure 18.9 62.2 29.7 67.6
31 Position (nmi)
%Agreement %Desirable %Agreement %Desirable
CI number 15.2 69.7 19.7 74.2Pressure 16.7 47.0 15.2 54.5
57 Position (nmi)
%Agreement %Desirable %Agreement %Desirable
CI number 27.5 82.5 40.0 75.0Pressure 25.0 50.0 22.5 60.0
31 Position (nmi)
%Agreement: % of estimates within 1/4 T-no or 2.5 mb of SAB estimate%Desirable: % of estimates within 3/4 T-no or 7.5 mb of SAB estimatePosition differences based on MW analysis NOT SAB position. Since MWdata not always available, N is less than that used in intensity estimates
JTWC ADT-SAB
Overall40
21.1 20.5
Intensity N ParameterReunion ADT-SAB
ParameterAustralia ADT-SAB JTWC ADT-SAB
Overall66
25.6 33.6
Intensity N ParameterFiji ADT-SAB JTWC ADT-SAB
Overall37
51.7 41.9
Intensity N
NESDIS Enhancements NESDIS Enhancements on Operational Productson Operational Products
Gregory M. Gallina – Satellite Analysis Branch (SAB)
Keith Liddick, Anthony E. Salemi, Alan Schwartz, Matthew Seybold,
Michelle Spampata, and Michael Turk
METSAT Conference - 28 April 2009
Product System Development & Implementation (PSDI) Program
• Operational Implementation of the CIMSS Advanced Dvorak Technique & New Development phase (ADT v8.0)
• Operational Implementation of an Ensemble Tropical Rainfall Technique (eTRaP)
• Global Tropical Cyclone Formation Probability Product
• GOES-Based Multi-Platform Tropical Cyclone Surface Wind Product
• AMSU-Based Tropical Cyclone Intensity and Structure Estimation from Aqua and MetOp
Ike (09L): ADT Unable to Resolve Small EyeIke (09L): ADT Unable to Resolve Small Eye
• Small eye barely discernible in conventional and microwave imagery
• ADT used CDO cloud scene, resulting in falling T-numbers and underestimates in intensity
Ike (09L) Current Intensity Time Series
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
1715
Z18
45Z
0645
Z14
15Z
1715
Z19
15Z
2115
Z23
15Z
0645
Z07
45Z
0945
Z11
15Z
1815
Z20
15Z
1815
Z20
15Z
2115
Z23
15Z
0645
Z07
15Z
0745
Z08
45Z
0945
Z11
15Z
1145
Z19
45Z
2115
Z23
15Z
0315
Z06
45Z
0715
Z08
15Z
0945
Z11
45Z
1345
Z17
15Z
1915
Z21
15Z
2315
Z02
15Z
0645
Z07
15Z
0915
Z11
15Z
1445
Z17
15Z
1915
Z20
45Z
2315
Z02
15Z
0345
Z06
45Z
0715
Z08
15Z
1145
Z13
15Z
1515
Z17
15Z
2045
Z22
15Z
2315
Z00
45Z
0145
Z02
45Z
0345
Z06
45Z
55666666777777888899999999991010101010101010101010111111111111111111111212121212121212121212121313131313
ADT CI Recon CI SAB CI
First 6 hr PeriodValid @ 0600 UTC
Third 6 hr PeriodValid @ 1800 UTC
Fourth 6 hr PeriodValid @ 0000 UTC
How To: Make an Ensemble TRaP A Combination Approach
Example using Single-Orbit TRaP Ingredients for Rita eTRaP:
Multiple Sensor RainRates, Forecast Periods, and Forecast Agencies
# ensemble members = 7 x 9 x 6 x 3 = 1134Culled to 200 members (this is configurable)
24 h eTRaP forecast for Ritavalid 0000 UTC 25 Sept 2005
AVG QPF PM QPF P => 50 mm
P => 100 mm P => 150 mm P => 200 mm
Smoothes the maxima out &Increases areal coverage Maintains Max values
eTRaP Validation Statistics for 2008eTRaP Validation Statistics for 2008
0
0.5
1
1.5
2
0 50 100 150 200
Rain threshold (mm)
Fre
qu
ency
bia
s
eTRaP
TRaP
0
0.1
0.2
0.3
0.4
0.5
0 50 100 150 200
Rain threshold (mm)
Eq
uit
ab
le t
hre
at
sc
ore
eTRaP
TRaP
0
0.5
1
1.5
2
0 50 100 150 200 250
Rain threshold (mm)
Fre
qu
en
cy
bia
s
eTRaP
TRaP
0
0.1
0.2
0.3
0.4
0.5
0 50 100 150 200 250
Rain threshold (mm)
Eq
uit
ab
le t
hre
at
sco
re
eTRaP
TRaP
6 h
24 h
eTRaP Verification of Probability ForecastseTRaP Verification of Probability Forecasts
ReliabilityReceiver Operating
Characteristic (ROC)
6 hr
24 hr
20082008
Update on CIRA GOES-PSDI Project Extension5.2bFY03 Tropical Cyclone Formation Probability (TCFP) Product
Collection of developmental datasets needed toexpand current product to global domain has begun
Satellite water vapor imagery for IndianOcean & S. Hemisphere collected from • Meteosat 5 & 7 (centered at 63 E)
Nov 2005 – present• Meteosat 9 (centered at 0 E):
June 2007 – present
Starting collection and quality control of best tracks for Indian Ocean and Southern Hemisphere
Global GFS analyses already archived at CIRA back to 1982
Tropical Cyclone Forecast Probability Product Tropical Cyclone Forecast Probability Product Verification of Probability Forecasts for WPAC/CPAC Verification of Probability Forecasts for WPAC/CPAC
in 2008in 2008Reliability
Receiver Operating Characteristic (ROC)
Entire Domain
0
5
10
15
20
25
30
35
40
45
50
0 5 10 15 20 25 30 35 40 45 50
Expected Genesis Occurance Rate (%)
Ob
se
rve
d G
en
es
is O
cc
ura
nc
e R
ate
(%
)
Product GOR
Clim GOR
Perfect
0
5
10
15
20
25
30
35
40
45
50
0 5 10 15 20 25 30 35 40 45 50
Expected Genesis Occurance Rate (%)
Ob
se
rve
d G
en
es
is O
cc
ura
nc
e R
ate
(%
)
Product GOR
Clim GOR
Perfect
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
False Alarm Rate
Pro
bab
ilit
y o
f D
etec
tio
n
Clim
Product
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
False Alarm Rate
Pro
ba
bil
ity
of
De
tec
tio
n
Clim
Product
Over 5x5 Subgrids
No skill l
ine
No skill l
ine
Over Confident
Over Confident
Under Confident
Under Confident
Skillful
Skillful
Project recently funded; will enjoy 24/7 support in 2 yrs:•May 2008: Development Phase Underway•May 2009: Pre-Operational Phase•June 2010: Operational Phase Begins
Multiplatform Surface WindMultiplatform Surface Wind
Product Description• Wind Analysis is produced 6-hourly (~ 25 past
synoptic) and contains– IR flight-level proxy winds– AMSU 2-d non-linear balance winds– Scatterometery (A-SCAT, QuikSCAT)– Cloud/Feature track winds below 600 mb.
• Fixes come from the wind analysis– MSLP dynamically estimated from the axi-symmetric
winds – Wind Radii to the nearest 5 n. mi
• Request for transition to operations by JTWC
MSLP VerificationMSLP Verification
Avg. Error 1.73
MAE 7.29
RMSE 9.92
R^2 80.68
Mean 980.6
Validated against 2008 Recon in Atlantic (within 2-hrs)
MAE - Mean Absolute ErrorPOD - Probability of DetectionFAR - False Alarm RateRMSE - Root Mean Square ErrorR^2 - Variance Explained
Statistical Terms
NENW
SE
SW G.Gallina SAB 04/15/09
Avg. Error: 0.1nmMAE: 14nmN: 56POD: 98%FAR: 7%Variance Explained or R^2 33%
All Quadrants TotaledAvg. Error: - 13nmMAE: 20nmN: 81POD: 98%FAR: 5%Variance Explained or R^2 49%
Avg. Error: - 20nmMAE: 35nmN: 325POD: 99%FAR: 9%Variance Explained or R^2 33% Avg. Error: 5nm
MAE: 16nmN: 81POD: 97% FAR: 6%Variance Explained or R^2 40%
Avg. Error: 14nmMAE: 30nmN: 128POD: 98% FAR: 11%Variance Explained or R^2 52%
Avg. Error: 9nmMAE: 15nmN: 56POD: 96% FAR: 10%Variance Explained or R^2 27%
Avg. Error: - 17nmMAE: 29nmN: 81POD: 97% FAR: 3%Variance Explained or R^2 33%
Avg. Error: - 27nmMAE: 41nmN: 128POD: 99% FAR: 3%Variance Explained or R^2 37%
Avg. Error: 8nmMAE: 15nmN: 56POD: 97% FAR: 9%Variance Explained or R^2 25%
Avg. Error: - 7nmMAE: 18nmN: 56POD: 98% FAR: 0%Variance Explained or R^2 36%
Avg. Error: 4nmMAE: 15nmN: 56POD: 99% FAR: 6%Variance Explained or R^2 30%
Avg. Error: - 21 nmMAE: 43 nmN: 128POD: 100% FAR: 4.5%Variance Explained or R^2 30%
Avg. Error: - 19nmMAE: 25nmN: 81POD: 99% FAR: 0%Variance Explained or R^2 54%
Avg. Error: 1.5nmMAE: 36nmN: 128POD: 99% FAR: 12%Variance Explained or R^2 43%
Avg. Error: - .5nmMAE: 20nmN: 81POD: 97% FAR: 6%Variance Explained or R^2 56%
34 Kts(Black)
50 Kts(Green)
64 Kts(Red)
Multiplatform Satellite Derived Tropical Cyclone Surface Wind Field
Validation Statistics
Wind Radius by color
Validated against 2008 Recon in Atlantic (within 2-hrs)
Update on CIRA POES-PSDI Project( Request FY99-38) FY08 AMSU-Based Tropical Cyclone Intensity
and Structure Estimation from Aqua and MetOp
• Retrieval codes for Aqua and MetOp AMSUs were obtained from SMCD
• Operational code has been modified to incorporate retrieval subroutines.
• The operational scripts are being modified by CIRA with coordination with NCO and TPC. Once completed the BUFR scripts will be modified by NCO
• Operational code will be finalized and tested once the operational data stream is finalized.
Brief Statistics OverviewForecast Event
and Occurs (POD)
‘a’
Type 1 Error
False Alarm Rate (FAR)
‘b’
Type 2 Error
Failure to Detect (FOD)
‘c’
NO Forecast where Event
did NOT Occur
‘d’
Observed
Forecast
T
T F
F
Equitable Threat Score (ETS)
ETS a ar
a b c ar
ar a c a b
Relative Operating Characteristic
Hit Rate or Prob. of Detection (POD) = FAR=
ROC Plot is Hit Rate vs. FAR
_____
b + d
____
a + b
Bias – comparison of areal coverage between forecast and observed. Ideal ratio is 1.0Reliability – the ratio of the variation of the true score and the variation of the observed score OR Observed Frequency vs. Forecast Probability