View
29
Download
0
Category
Tags:
Preview:
DESCRIPTION
Reconstruction of missing data in satellite and in situ data sets with DINEOF A. Alvera-Azcárate, A. Barth, J.-M. Beckers GHER – University de Liège FNRS – National Fund for the Scientific Research, Belgium. Characteristics of DINEOF. DINEOF: Data Interpolating Empirical Orthogonal Functions. - PowerPoint PPT Presentation
Citation preview
Reconstruction of missing data in satellite and in situ data sets with DINEOF
A. Alvera-Azcárate, A. Barth, J.-M. Beckers
GHER – University de Liège
FNRS – National Fund for the Scientific Research, Belgium
• Reconstruction method for gappy data based on an EOF decomposition
• Parameter-free, no need of a priori information
• resolves time and space
• much faster than objective analysis
• Determines optimal number of EOFs by cross-validation. Error estimation
• Lanczos method for EOF decomposition (Toumazou and Crétaux, 2001): large matrices
Characteristics of DINEOF
DINEOF: Data Interpolating Empirical Orthogonal Functions
N EOFs
●●
●● ● ● ●
k
pj
Tpippji vuX
1, )()(
2nd: EOF decomposition with N=1 EOF
Calculate missing values:
Improved guess for missing values
Convergence: best value for missing data with 1 EOF
cross validation: error
EOF decomposition with N=2 EOFs
Calculate missing values
Improved guess for missing values
Then we repeat with N= 3 EOFs
and so on…
1st: Demeaned matrix: missing data flagged and set to zero
Some data are set aside for cross-validation
err
or
How it works
SST
Chlorophyll
Wind-induced filaments in Adriatic Sea
Mississippi river plume advected by Loop Current
Some Examples
Alvera-Azcárate et al, 2005
Multivariate DINEOF: examples
Radial currents: positive values: current towards the antenna negative values: current away from the antenna
˚C m/s
SST + HF-Radar currents
Time series reconstructionHourly air temperature at Izas catchment (Pyrenees)
˚C
hours
hours
hours
months months
Ori
gin
al d
ata
, ˚C
Reconstruction, ˚C
Time series reconstruction:original vs. reconstruction
-30 -20 -10 0 10 20 30-30
-20
-10
0
10
20
30
f(x) = 0.91x + 0.37R² = 0.92
Error maps are calculated for SST reconstructions using:i) the EOF basis from DINEOF as background covarianceii) the location of valid data
Outliers (pixels with value larger than the statistically expected misfit calculated during the analysis) will be objectively identified and removed from initial data
Error maps and outlier detection
Spike removal in temporal EOFs
When too few data are present: temporal EOFs poorly constrained: unrealistic discontinuities
Sharp transition
Spike removal in temporal EOFs
We apply a filter to the temporal covariance matrix
Alvera-Azcárate et al, 2009
CFF=C T~
F is a Laplacian filter Filter on C instead of X: C is much smaller and less sensitive to missing data Filter applied iteratively: more iterations, further reach of the filter
days days
days
days
Spike removal in temporal EOFs
Conclusions
• DINEOF is a reliable, easy to use technique to reconstruct missing data
Widely tested and results validated with in situ data
• A variety of data and platforms can be used
SST, chlorophyll, TSM, HF-Radar….
• Univariate and Multivariate approaches
• Satellite data and time series
• Error maps and outliers
Generation of error maps to assess the confidence of the reconstruction
Outliers (cloud edges, haze...) can be removed from original data
• Time filter to remove spikes
• Code is freely available; installation instructions; mailing list
http://modb.oce.ulg.ac.be/mediawiki/index.php/DINEOF
References (chronologic)
• J.-M. Beckers and M Rixen. EOF calculations and data filling from incomplete oceanographic data sets. Journal of Atmospheric and Oceanic Technology, 20(12):1839-1856, 2003.
• A. Alvera-Azcárate, A. Barth, M. Rixen, and J. M. Beckers. Reconstruction of incomplete oceanographic data sets using Empirical Orthogonal Functions. Application to the Adriatic Sea. Ocean Modelling, 9:325-346, 2005.
• J.-M. Beckers, A. Barth, and A. Alvera-Azcárate. DINEOF reconstruction of clouded images including error maps. Application to the Sea Surface Temperature around Corsican Island. Ocean Science, 2(2):183-199, 2006.
• A. Alvera-Azcarate, A. Barth, J. M. Beckers, and R. H. Weisberg. Multivariate reconstruction of missing data in sea surface temperature, chlorophyll and wind satellite fields. Journal of Geophysical Research, 112:C03008, 2007.
• A. Alvera-Azcárate, A. Barth, D. Sirjacobs, J.-M. Beckers. Enhancing temporal correlations in EOF expansions for the reconstruction of missing data using DINEOF. Ocean Science, 5, 475-485, 2009.
• A. Alvera-Azcárate, A. Barth, D. Sirjacobs, F.Lenartz, J.-M. Beckers. Data Interpolating Empirical Orthogonal Functions (DINEOF): a tool for geophysical data analyses. Submitted to Mediterranean Marine Science. 2009.
• U. Ganzedo, A. Alvera-Azcarate, G.Esnaola, A. Ezcurra and J. Saenz. 2009. Reconstruction of sea surface temperature by means of DINEOF. A case study during the fishing season in the Bay of Byscay. Accepted in International Journal of Remote Sensing.
• D. Sirjacobs, A. Alvera-Azcárate, A. Barth, G. Lacroix, Y. Park, B. Nechad, K. Ruddick, J.-M. Beckers. Cloud filling of ocean color and sea surface temperature remote sensing products over the Southern North Sea by the Data Interpolating Empirical Orthogonal Functions methodology. Accepted in Journal of Sea Research, 2010.
GHER
References
• Y.Z. Ding, Z.H. Wei, Z.H. Mao, X.F. Wang, D.L. Pan. Reconstruction of incomplete satellite SST data sets based on EOF method. Acta Oceanologia Sinica, 28 (2): 36-44 2009.
•E. Mauri, P.M. Poulain, Južnič-Zonta, Z. MODIS chlorophyll variability in the northern Adriatic Sea and relationship with forcing parameters. Journal of Geophysical Research, vol 112, C03S11, 2007.
• E. Mauri, P.M. Poulain, G. Notarstefano. Spatial and temporal variability of the sea surface temperature in the Gulf of Trieste between January 2000 and December 2006. Journal of Geophysical Research. Vol. 113, No. C10, C10012. 2008.
• T.N. Miles, R. He, M. Li. Characterizing the South Atlantic Bight seasonal variability and cold-water event in 2003 using a daily cloud-free SST and chlorophyll analysis. Geophysical Research Letters, 36: Art. No. L02604 JAN 22 2009.
• N.P. Nezlin, K. Kamer, J. Hyde, E.D. Stein, ED. Dissolved oxygen dynamics in a eutrophic estuary, Upper Newport Bay, California. Estuarine, Coastal and Shelf Science, 82 (1): 139-151 MAR 20 2009.
• R. F. Sánchez, P. Relvas, M. Delgado. Coupled ocean wind and sea surface temperature patterns off the western Iberian Peninsula. Journal of Marine Systems, 68, 103-127. 2007.
… and other applications:
Recommended