Upload
thais
View
46
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Distribution of Medspiration products from JPL Global Data Assembly Center (GDAC) GHRSST Medspiration and GlobCOLOUR Users Meeting 4-6 Dec 2006 Villefranche sur mer, France. Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang - PowerPoint PPT Presentation
Citation preview
Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang
Distribution of Medspiration products from JPL Global Data Assembly Center (GDAC)
GHRSST Medspiration and GlobCOLOUR Users Meeting 4-6 Dec 2006
Villefranche sur mer, France
Introduction - GHRSST data management
GHRSST international and regional task (and cost) sharing model: Built on 3 core components
1) Regional centers of SST expertise produce near realtime data (L2P/L4) for their sensors2) Distribution and coordination from regional centers requires “Clearinghouse” for data and metadata management
• Fulfilled by the JPL Global Data Assembly Center (GDAC)• Built on the 13 year expertise of the NASA Physical
Oceanography DAAC in satellite data management3) Archiving and reanalysis occurs the Longterm Stewardship and Reanalysis Center (NOAA NODC)
The GDAC serves as the link between data users and producers, and between producers and longterm data stewardship (between 1 and 3)
GDAC System Architecture
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Overview of GDAC tasksIngest realtime project data files
L2P, L4, L2P_GRIDDED data and metadata files from RDACs
Stage on public FTP site for 45 daysDeliver to long-term archive (LTSRF; NOAA NODC) after 30 daysProvide a variety data access methods (FTP, OPenDAP, POET)Ingest and quality assure metadata into the Master Metadata Repository. Produce FGDC metadata products for LTSRFProvide web-base interface for querying database and data discoveryAncillary data production for unfilled L2P files Provide USO and applications developmentMODIS L2P (collaboration with the NASA OBPG)
Support and value added services
FGDC metadataCreating and staging FGDC metadata for LTSRF interface
OPenDAPL2/L4 accessible via JPL server http://dods.jpl.nasa.gov
IDL and C L2P software readersRead entire L2P structure, can be used to test integrity of L2P
RDAC assistance with L2P/L4/HRDDS file format and metadata anomaliesL2P subsetting design in progressValue add L2P with ancillary filling
Wind , sea ice, solar insolation, aerosol ( from ECMWF, FNMOC etc.)
Filling for solar insolation
ECMWF Solar insolation
Same day AVHRR L2P filled with solar insolation data
Services continued
Web portal to GHRSST activities and data: http://ghrsst.jpl.nasa.govWebex site for intraproject collaboration on applications and user informationApplication User Services (AUS)
Supporting applications development (S. Heinz) Actively addressing user inquiries
Data products and supported sensors
GHRSST Level 2P European Medspiration AATSR SST GHRSST Level 2P European Medspiration AVHRR16 Global SST GHRSST Level 2P European Medspiration AVHRR16 Local SST GHRSST Level 2P European Medspiration AVHRR17 Global SST GHRSST Level 2P European Medspiration AVHRR17 Local SST GHRSST Level 2P European Medspiration AVHRR18 Global SST GHRSST Level 2P European Medspiration AVHRR18 Local SST GHRSST Level 2P European Medspiration NAR16 SST GHRSST Level 2P European Medspiration NAR17 SST GHRSST Level 2P European Medspiration SEVI RI SST GHRSST Level 2P European Medspiration AMSRE SST GHRSST Level 2P European Medspiration TMI SST GHRSST Level 4 European Medspiration Ultra-High Foundation SST GHRSST Level 2P European Medspiration NAR18 SST GHRSST Level 4 European UKMET OSTIA SST GHRSST Level 2P USA Remote Sensing Systems AMSRE SST GHRSST Level 2P_G RID DED USA Remote Sensing Systems AMSRE SST GHRSST Level 2P USA Remote Sensing Systems TMI SST GHRSST Level 2P_G RID DED USA Remote Sensing Systems TMI SST
GHRSST Level 2P USA NASA MODI S Terra SST GHRSST Level 2P USA NASA MODI S Aqua SST GHRSST Level 2P NAVOCEANO AVHRR18 Local SST GHRSST Level 2P USA OSDPD GOES11 SST GHRSST Level 2P USA OSDPD GOES12 SST GHRSST Level 2P USA NAVOCEANO AVHRR17 Local SST GHRSST Level 2P USA NAVOCEANO AVHRR17 Global SST GHRSST Level 2P USA NAVOCEANO AVHRR18 Global SST
AVHRR
AATSR
MODIS
AMSRE
TMI
SEVIRI
GOES
L4 blends
27 unique L2 and L4 products
MODIS L2P
SST
Bias
Flags
STD
MODIS L2P granule with OCSST
K490
Chlorophyll A
GOES L2P
SST AOD
WindSSI
Data Ingest Statistics -20062006 data ingest volumes
3.5 TB total
0
100
200
300
400
500
600
700
1 2 3 4 5 6 7 8 9 10 11
month (2006)
Volu
me i
n G
B
2006 files ingested327408 files total
0
10000
20000
30000
40000
50000
60000
70000
1 2 3 4 5 6 7 8 9 10 11
month (2006)
Nu
mb
er
of
file
s
Data Distribution Statistics -2006Volume distributed in 2006Total is 3.1 TB, 560K files
0
200
400
600
800
1000
1200
1 2 3 4 5 6 7 8 9 10 11
month (2006)
GB
User breakdown2
1
3
9
531111
8
78
99
Brazil
Canada
Commercial
Denmark
France
JPL
US Defense
NASA
Org
Russia
UK
Unknown
180 153.5 4579548 1044.1 18794153 1806.8 21101840 91.9 117036
Users Vol(GB) #FilesEUR
JPL
NAVO
REMSS
Data uses and impact studies
Data Use and Impact Studies from the Multisensor Improved Sea Surface Temperature Project (MISST)
NAVOCEANO: impact of assimilation of high resolution L2P including AATSR and SEVIRI on NCODA (NRL Coupled Ocean Data Assimilation system) and K10 merged SST product
NOGAPS TC track
Although there was no significant difference in the track forecast errors overall, there were areas where the use of the MISST SST analyses resulted in significantly improved NOGAPS TC track forecasts
Data Use Cases and impact studies
Additional Data Use and Impact StudiesNOAA-RSS: Hurricane intensity impact used blended microwave L4UK Met: L2P for OSTIA L4 merged productsRSS: L4 blend using AMSRE, TMI, MODISNOAA: improved Reynolds OI using TMI and AATSR L2PFNMOC: evaluate AATSR and SEVIRI in operational SST blendCanadian Meteorological Forecast Office: AVHRR L2P
MISST research
MISST researchAerosols
• AATSR used to study impact of aerosols on infrared measurements as compared to AVHRR and MODIS
– Crucial for assigning flag or weight on IR measurements in SST blends
Diurnal warming models for deriving foundation temperature
• SEVIRI measurements have excellent calibration and resolution necessary to resolve the daily diurnal cycle of SST
Skin layer temperature• Used to derive CO2 flux maps (B. Ward)• AATSR has most accurate skin layer measurement available
Data Use Cases continued
US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities coordinated by the GDAC.
• Shorterm prediction and research center (SPORT)
SPORT supports infusion of NASA observations, data assimilation and modeling research into NWS forecast operations and decision making at the regional and local level.Using the OSTIA product for modeling and forecasting activities for the US Gulf and North Atlantic regions.SPORT is generating daily images available at: http://weather.msfc.nasa.gov/sport/sstAnimation/
Data Use Cases
The PDC has pledged their support to prepare and disseminate gridded GHRSST data via the APNHVA.Atlas is published both as Asia Pacific basin-wide and Hawaiian Island region.Support disaster community applications for preparedness, mitigation, response and recovery activities.User include Hawaii State Civil Defense, FEMA, NOAA, UN agencies, Thailand’s National Disaster Warning Center and Vietnam’s Disaster Management Committee. http://www.pdc.org
Pacific Disaster Center
Data Use Cases continued
US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities.
• Shorterm prediction and research center (SPORT)• Pacific Disaster Center
– Asia Pacific Natural Hazards Information Network• NASA Aquarius Salinity Mission• Gulf of Maine Ocean Observing System (GOMOOS)• Southern California Coastal Ocean Observing System
(SCCOOS)• Ecosystem Management
– California State University Humbolt Bay– NOAA Protected Species Management
• Southeast Phytoplankton Monitoring Network (SEPMN)• ECCO assimilation
GHRSST metadata model
Follows the NASA Directory Interchange Format (DIF)Granule records ( File Record [FR] )Dataset record ( Dataset Description [DSD] )XML based
L2P/L4 products only accepted with valid metadataMetadata ingested into a “clearinghouse”: the Master Metadata Repository (MMR)Support for FGDC metadata standard implemented for distribution and archiving at the LTSRFFuture support
ISO19115
Master Metadata Repository (MMR)
MMRmySQL database (ver 4x)
• Fully relational, transactions implementedSeparate multi-table relational databases for DSD and FR records linked with one common fieldDSD metadata records jointly maintained by Met Office and GDAC Over 150K FR records ingested without errorQA checkers/XML parsers for FR and DSD records implementedWeb search interfaceSupports output to FGDC format through XML stylesheet conversions
MMR Search Interface
Interface to the LTSRF
FGDC records created for 30 day (and older data).30 day old data and metadata extracted from an operational FTP site by NOAAGHRSST data products expired from GDAC (PO.DAAC) FTP site after 45-60 days (“rolling store”)
Size of the rolling store can easily be adjusted to accommodate changing disk space resources
To date 100% file integrity between GDAC and LTSRF
DIF to FGDC conversion
Extract FR from MMRFormat to XML
Extract andformat DSD to XML
Apply XSL
FGDC formatted record
GDAC future activities
Integrated into the PO.DAAC in 2008ISO19115 support
Routine Ancillary L2P fillingDelivery of L4 and L2P products via the POET visualizer and subsetter
Google Earth browse capability
Additional L2P, L2P_GRIDDED, and L4 productsNetCDF 4 supportIngestion of metadata into ECHO
GDAC team
Jorge Vazquez - Project managerEd Armstrong - GDAC Cognizant EngineerAndy Bingham - Data management and Oceanids developerSue Heinz - Applications and User Services lead ([email protected])Alex Cervantes - Software engineer for Oceanids development and L2P ancillary fillQui Chau - Software engineer for L2P ancillary fill, MODIS L2P, L2 subsetting developmentWendy Zhang - Software engineer on MMRTim McKnight - Operations engineer ([email protected]) Pat Liggett - PO.DAAC team and management
Backup - Data Access
ftp://podaac.jpl.nasa.gov/GHRSSTOrganized by:datatype / sensor / RDAC / year / DOY
OPenDAP:dods.jpl.nasa.gov
User support:Email [email protected]
OCEANIDS OVERVIEW
CATALOGUE
INGEST / QC
DISTRIBUTE
REPO
RTS
& AC
COUN
TABI
LTY
DATA
STORE
DATA PROV IDERS
USERS