View
217
Download
0
Category
Tags:
Preview:
Citation preview
Atmospheric Science DepartmentUniversity of Missouri - Columbia
A Procrustes Shape Analysis Verification Tool
Steve Lack Sakis Micheas Neil Fox Chris Wikle
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Methodology
• Procrustes Scheme– In Greek mythology, Damastes, was a thief
who often went by the name Procrustes and offered travelers a bed to rest
– The twist was he would “fit” the travelers to the bed…stretching them if they were too short and cutting their limbs off if they were too tall
Journal of Hydrology article forthcoming Micheas et al.
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Methodology
• So in this context we will fit forecast cells to truth cells by shifting location, resizing the object, and rotating them if necessary
• Origin use is in biology (anatomy) for lining up jaw bones (landmarks) or other structure for classification
• Works with gridded data of any dimensions for spatial objects, originally designed for ensembles
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Methodology
• Define a penalty function– Takes into account
max,mean,min intensity, location fitting
– May be user defined (apply weights)
– The lower the penalty the better the forecast
kjijkjk
j zercz jk̂ˆˆˆ
kkkk SSSSSSRSSD maxminavg
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Advantages
• Reduction of dimensionality for speed (less than 1 minute to run on a fair sized domain)
• Takes into account shape information with intensity data
• Can alter the penalty function to get more meaningful data
• Error decomposition• User defined thresholds
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Shortcomings
• Memory issue when running a large domain with a lot of identifiable cells– Can break down domain however
• The total penalty function adds error components that may not of the same magnitude– Can refine these in the future
• Relative comparison (interpretation)
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Results
• Entire domain penalty was calculated by using 0.10” (memory issue)
• Minimum size of cell set to 10 pixels
• Divided domain into NE,NW,SE,SW quadrants
• Recalculated penalties based on 0.01” again minimum size set to 10 pixels
Atmospheric Science DepartmentUniversity of Missouri - Columbia
13 May 2005 – WRF4NCAR
Atmospheric Science DepartmentUniversity of Missouri - Columbia
13 May 2005 – Stage 2
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Fits – WRF4NCAR
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Total Penalties – Entire Domain
WRF4NCAR WRF4NCEP WRF2CAPS
RSS (1*10^3) 5.9151 5.4487 4.8603
SStot (1*10^3) 1.7531 2.3083 1.9005
# Cells
(Truth = 17)
26 61 39
Tot Error (1*10^5)
2.6607 5.7858 2.8416
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Error Decomposition – Entire Domain
WRF4NCAR WRF4NCEP WRF2CAPS
Min Intensity 2 4 4
Mean Intensity 8926 26825 9079.9
Max Intensity 2521220 5463000 2702100
Translation 4.6711 22.669 11.596
Rotation 0 0 0
Dilation 428290 7736700 1176000
Atmospheric Science DepartmentUniversity of Missouri - Columbia
NW Domain- 13 May 2005(clockwise NCAR, NCEP, CAPS, Stage 2)
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Total Penalties – NW Domain
WRF4NCAR WRF4NCEP WRF2CAPS
RSS (1*10^3) 5.8255 5.7583 5.34
SStot (1*10^3) 3.4659 3.8712 3.4335
# Cells
(Truth = 39)
50 33 37
Tot Error (1*10^4)
0.96153 1.8417 1.1106
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Error Decomposition – NW Domain
WRF4NCAR WRF4NCEP WRF2CAPS
Min Intensity 0 0 0
Mean Intensity 104.78 297.67 148.31
Max Intensity 3685 12361 5618
Translation 32.488 13.969 15.15
Rotation 0 0 0
Dilation 797540 691740 758880
Atmospheric Science DepartmentUniversity of Missouri - Columbia
1 June 2005 – WRF4NCAR
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Stage 2
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Total Penalties
WRF4NCAR WRF4NCEP WRF2CAPS
RSS (1*10^3) 4.3458 3.8631 4.7510
SStot (1*10^3) 1.7761 2.3113 1.8553
# Cells
(Truth = 48)
59 109 49
Tot Error (1*10^5)
3.8423 3.4230 3.4501
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Error Decomposition
WRF4NCAR WRF4NCEP WRF2CAPS
Min Intensity 18 13 11
Mean Intensity 16151 12394 14009
Max Intensity 363710 326030 326240
Translation 5.5781 8.139 6.3785
Rotation 0.0001 0 0
Dilation 1085000 1945500 1316600
Atmospheric Science DepartmentUniversity of Missouri - Columbia
NW Domain- 01 June 2005(clockwise NCAR, NCEP, CAPS, Stage 2)
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Total Penalties – NW Domain
WRF4NCAR WRF4NCEP WRF2CAPS
RSS (1*10^4) 1.1838 1.6253 1.4684
SStot (1*10^3) 2.9221 3.9807 2.9628
# Cells
(Truth =30)
53 50 58
Tot Error (1*10^4)
1.7542 3.8873 2.3150
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Error Decomposition – NW Domain
WRF4NCAR WRF4NCEP WRF2CAPS
Min Intensity 0 0 0
Mean Intensity 184.91 425.37 60.646
Max Intensity 5519 22194 8405
Translation 59.107 12.883 30.551
Rotation 0 0 0
Dilation 3217400 357250 827400
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Conclusions
• A lot of weight is given to intensity forecasts as well as position/shape
• June 1 forecast are all handled equally well by the models
• WRF4NCAR handles intensity the best for May 13 although WRF2CAPS is close on position
• WRF4NCEP underestimates intensity in most cases
Atmospheric Science DepartmentUniversity of Missouri - Columbia
Acknowledgements
• Stat Group: Sakis Micheas, Chris Wikle, Yong Song, Kent Peng
• Rains Group: George Limpert and Neil Fox
Award # ATM-0434213
Recommended