Upload
mariko
View
26
Download
0
Embed Size (px)
DESCRIPTION
The Ocean, the Atmosphere, and the Grid A UK perspective. David Webb Southampton Oceanography Centre. Natural Environment Research Council. San Francisco 4/5th August 2001. ECMWF Winds. Ocean Models. Satellite Observations. Alace floats. Moorings. Ship observations. - PowerPoint PPT Presentation
Citation preview
The Ocean, the Atmosphere, and the Grid
A UK perspective
David WebbSouthampton Oceanography Centre
NaturalEnvironmentResearchCouncil San Francisco 4/5th August 2001
Satellite Observations
Ocean Models
ECMWF Winds
Ship observations
Alace floats
Autonomous vehicles
Moorings
Bugs
The Oceanographers View ...
The user wants ...
Seamless movement from
data search
to data extraction
to visualisation
to comparison with other data
to analysis
to …
and back again.
Some of the problems ...•Very large individual data files - up to 8
Gb
•Large numbers of files - i.e. one every six hours
•Very diverse dataWe need: - Quick look data
- Sub-samples in space and time
- Track local and cache copies
- Map conversionsWe want to: - Inter-compare 2-D and 3-D fields
- Overlay station data on 2-D and 3-D fields- Simplify access -Use browsers -Move logic to the user.
Outline Data Grid
HPC
Browser
C/Fortran
Cache
GIS
Users
Cache
Data Source
Farm
Data Centres
Middleware
+ Agents
SOC
Metadata
BODC
BADC
3D System
WWW
Data Centres Middleware+ Agents
User interface
Organisation
•Simple data transforms
Change gridSub-sampleInterpolateFormats
•Complex transformsDensity/vorticity/etcExtreme eventsHeat fluxes
•Transform metadata
•Logical operationsHunt for related
dataData quality tests
•Handle caching/delays
…and W3C compliant
• MetadataDataCached dataModel GridsAlgorithms
•Formats + conventions
HDFnetCDF/ferret
•RetrievalSpeedDelays
•Pre-processingSub-samplingData compression
•Extreme events (Do you want 200 GB?)
•Browser (Netscape) and anonymous users
•Fortran/C program
•Matlab /GIS
•Handle caching/delays
•Data Sources
The Atmospheric Data Centre View ...
BADC- Migrating from “downloads” to E-Science
BADC Data & Catalogue
At USER Institution
At BADC
which will eventually be “griddable”.
Database Client
Computation
Graphics
3
we are about to provide tools at the data centre,
Database Client
Database
2
In the beginning, users downloaded files,
Graphics
Computation
1
New Concept: The Catalogue Interface Resource Broker (more than just the SRB)
Accessing Atmospheric Data at the BADC
via Grid Technologies
•The CIRB would provide access to data both within the BADC and in other locations. It would be “cache aware”.
•No one data centre would be the unique CIRB, rather each would be a peer in a network of CIRBs fronting each data source.
•The CRIB will provide access to the data lying in the other data stores. Security and resource issues will be dealt with by “e-science” software agents.
•Users would be able to access data via a GUI on their own machine, on our systems, or even via user written APIs.
Possible US Links:
NCAR - Atmospheric Data
NOAA - Ocean and Atmospheric Data
Los Alamos - Ocean Model Data
LLNL - Climate Change Model Data Sets
Summary
San Francisco 4/5th August 2001
The data grid needs:
√ Dataset sub-volumes
• Dataset sub-sampling
• Data cache and cache metadata
• Conversion and compression
User software to seamlessly integrate
• Data discovery from many data centres
• Plotting/comparison of data
• As GIS, Browser and subroutines
The compute grid needs:
• link to compute portals• access and security• libraries• scheduling for
- ensemble experiments - loose coupled models - fully coupled models
• automatic cataloguing• globally accessible databases• code maintenance