Embed Size (px)
Citation preview
CALIOP trained neural network cloud top pressureand height for imagers
Nina Håkansson1, Claudia Adok, Anke Thoss1, Ronald Scheirer1, SaraHörnquist1
1Swedish Meteorological and Hydrological Institute
Presentation at ICWG Madison
NWCSAF PPS Package
• Software for cloud and precipitationproducts
• Polar orbiting satellites• Instruments:− AVHRR− MODIS− VIIRS
• Nowcasting & Climate data recordproduction (CLARA-A2)
• Next release December 2018• CTTH product− Cloud top temperature and height
(also pressure)
PPS-v2018 CTTH
S-NPP 8th March 2015, at 10:57UTC
Neural Network Cloud Top HeightNN-CTTH:https://www.atmos-meas-tech.net/11/3177/2018/amt-11-3177-2018.html
Suomi-NPP 8th August 2015, at 20:25UTC
PPS-v2018 CTTH(MAE scene:0.9km)
PPS-v2014 CTTH(MAE scene:1.7km)
• MAE: Mean absolut error compared to CALIOP.
Algorithms for comparison
• ESA-CLOUD-CCI andPATMOS-X (end of talk):− Optimal estimation
• PPS-v2014 CTTH− Spatial information: T11,T12 for
32x32 pixels
• MODIS-C6 uses:− Spectral information: CO2
channels at 13 micron− Lapse rate
• Validation: CALIOP (V4, 1km)and:− 6 days MODIS (Aqua)
! Plot of error: lower bars are better PPS-v2014MODIS-C60
20
40
60
80
100
120
Mea
n Ab
solu
te E
rror (
MAE
) hPa Validation CALIOP
PPS-v2018 CTTH Development• NN-OPAQUE− T12
− Surface pressure− Temperature for 6 pressure
levels
• NN-BASIC− Including T11 − T12,
which holds information ofsemi-transparency of cloud
• PPS-v2018-(MSE)− Including variables calculated
from neighbouring pixels(T11,T12)
• PPS-v2018− MAE as loss function PP
S-v20
14
MODIS-C6
NN-OPAQUE
NN-BASIC
PPS-v
2018
-MSE
PPS-v
2018
0
20
40
60
80
100
120
MAE
hPa
Validation CALIOP
NN-CTTH for VIIRS on Suomi-NPP
• Trained with MODIS 2010• Applied to VIIRS 2015• S-NPP VIIRS data: 15
orbits co-located withCALIOP (V4, 1km)
• Robust performance
PPS-v2014
MODIS-C6
PPS-v2018
PPS-v2014 (NPP)
PPS-v2018 (NPP)0
250
500
750
1000
1250
1500
1750
2000
MAE
m
Validation CALIOP
Validation with CloudSat• Validation with independent truth: CPR radar.• Neural network good for all cloud classes (low, medium and high).• PPS-v2014 good for medium level clouds.
PPS-v2014
MODIS-C6
PPS-v2018
PPS-v2014
PPS-v20180
500
1000
1500
2000
MAE
m
Low
PPS-v2014
MODIS-C6
PPS-v2018
PPS-v2014
PPS-v2018
Medium
PPS-v2014
MODIS-C6
PPS-v2018
PPS-v2014
PPS-v2018
High
What about bias and STD?
• NWCSAF requirements• MAE much smaller than
MODIS-C6 and PPS-v2014:− Expected good performance
according to requirements
• STD better for PPS-v2018, butstill outside threshold accuracy.Why?
1 Algorithms not good enough?2 Validation method?
• Average imager and truth=⇒ not pixel-level validation
• Remove thin clouds and cloudedges, how? =⇒ validationscores valid for part of data
3 Measures not suitable?PPS-v2014
MODIS-C6
PPS-v20182000
1500
1000
500
0Bi
as (m
)
0
1000
2000
3000
STD
(m)
Non-Gaussian error distributions
• NN-CTTH and correspondingGaussian distribution− Same bias & STD− Distributions are different⇒ bias & STD not enough to
describe the error distribution!
• Bias: not most common error• Measures:− Q2: Median− IQR: Interquartile range Q3-Q1
• Note: some large errors expecteddue to sensor and FOVdifferences.
4 2 0 2 4 6Error (km)
0
2
4
6
8
Per
cent
NNCTTHbias: 350STD: 2467IQR: 956Q2: 47
Gaussianbias: 348STD: 2467IQR: 3323Q2: 351
Bias can be misleading• Bias & STD ⇒ blue (bias “corrected”) is best• Median (Q2) & STD ⇒ red is best• PE0.5km = Part of Error larger than 0.5km• MAE, PE0.5km, and plot show: trust Median!
4 2 0 2 4 6Error (km)
0
2
4
6
8
Per
cent
bias=350mSTD=2467mMAE=1300mIQR=956mQ2=47mPE0.5=49%
PPSv2018
4 2 0 2 4 6Error (km)
0
2
4
6
8
bias=0mSTD=2467mMAE=1405mIQR=956mQ2=397mPE0.5=60%
Bias "corrected"
Suitable measures for non-Gaussian error distributions:
0
2
4
6
8
Per
cent
a) Within threshold accuracybias=900mSTD=1601mMAE=1474mIQR=2161mQ2=899mPE0.5=78%
b) Within target accuracybias=0mSTD=922mMAE=900mIQR=1799mQ2=10mPE0.5=97%
2.5 0.0 2.5 5.0error (km)
0
2
4
6
8
Per
cent
c) Outside threshold accuracybias=350mSTD=2467mMAE=1300mIQR=956mQ2=47mPE0.5=49%
2.5 0.0 2.5 5.0error (km)
d) The worst (in STD) is the bestNNCTTHGaussianBimodal
Comparison with several CDRs• 30 orbits (N=45000) NOAA-18 from 2009 co-located with CALIOP (V4, 5km).• This PPS-v2018 trained on GAC data.• Best results for PPS-v2018!
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
500
1000
1500
2000
2500
3000
MAE
(m)
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
500
1000
1500
2000
2500
3000
3500IQ
R (m
)
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
20
40
60
80
100
Part
Erro
r >0.
5km
(%)
Comparison with several CDRs• Best results for PPS-2018, also for the non-suitable measures bias and STD.• CONSTANT method: height = 7477m everywhere.
CONSTANT
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC
1500
1250
1000
750
500
250
0
MED
IAN
(m)
CONSTANT
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC
2500
2000
1500
1000
500
0
BIAS
(m)
CONSTANT
CLARA-A2
V5-PATMOSX
V2-CLOUD-CCI
PPS-v2018-GAC0
1000
2000
3000
4000
5000
STD
(m)
Comparison with several CDRs
• MAE, IQR and PE0.5km correctly detects CONSTANT method as bad.
CONSTANT
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
1000
2000
3000
4000
MAE
(m)
CONSTANT
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
2000
4000
6000
8000IQ
R (m
)
CONSTANT
CLARA-A2
V5-PATM
OSX
V2-CLO
UD-CCI
PPS-v
2018
-GAC0
20
40
60
80
100
Part
Erro
r >0.
5km
(%)
Error estimatesUpper and lower threshold with quantile regressionneural networks, Simon Pfreundschuh et al. (2018):https://www.atmos-meas-tech.net/11/4627/2018/amt-11-4627-2018.html
Upper limit Lower limit
• Method not restricted to AVHRR/MODIS/VIIRS although PPS-software package is.
Future and SummaryFuture without CALIOP:• Neural network training requires a truth• After CALIPSO comes EarthCARE, but then ...• Current NN-CTTH can be trained on historic IMAGER/CALIOP data• But for newer channels or other retrievals?
Summary:1 NN-CTTH performes well:− validated with CALIOP and CPR (Cloudsat)− for sensors MODIS, VIIRS and AVHRR− compared to PPS-v2014, MODIS-C6, PATMOS-X, ESA-CLOUD-CCI and
CLARA-A2
2 The retrieval errors are NOT well described by Bias and STD.
3 MAE, IQR, PE0.5km and Median give important information of the errors.
Thank you for listening!
Suomi-NPP 8th March 2015, at 10:58UTC
PPS-v2018 CTTH(MAE:1.4km)
PPS-v2014 CTTH(MAE:3.1km)
