Embed Size (px)
Citation preview
1
Accomplishments
2
Overview of Accomplishments
Sustaining the Production Earth System Grid• Serving the current needs of the climate modeling
community• Three operational portals with 16,000 registered
users have delivered > 0.7 PB of data to date Developing and Deploying the Next Generation Earth
System Grid• Addressing the future needs of the climate modeling
community• Scaling out and extending the capabilities of the
Earth System Grid• Meeting needs and timeline for CMIP5/AR5
3
Sustaining the Production Earth System Grid
The production ESG hasbecome the source formodel data for the world-wide climate modeling community• Over 16,000 users from
more than 500 sites The ESG-CET team currently operates three distinct ESG
installations, serving different collections of data• WCRP CMIP3 portal• Main ESG portal (CCSM, POP, CISM, CLM, NARCCAP,
PCM)• C-LAMP portal
ESG usage: over 500 sites worldwide
4
WCRP CMIP3 Portal
World Climate Research Program (WCRP) Coupled Model Intercomparison Project, Phase 3 (CMIP3)
Results from 12 sets of numerical experiments performed using 23 different global coupled ocean-atmosphere models
Input to Intergovernmental Panel on Climate Change (IPCC) 4th Assessment Report (AR4)• Over 300 peer reviewed papers derived from this
collection went into AR4• Nobel Peace Prize 2007
Portal located at LLNL Program for Climate Model Data Intercomparison (PCMDI)
5
WCRP CMIP3 Portal Statistics
Data holdings:• 35 TB• 74,000 files• 12 experiments• 23 models• 13 countries
3,000+ registered analysis projects (users)
~500 scientific papers to date
Downloads to date:• 0.6+ PB• 1.8M+ files• Avg ~500 GB/day
6
Main ESG Portal
Hosts data collections for many of our key collaborators• Community Climate System Model (CCSM)• Parallel Climate Model (PCM)• Parallel Ocean Program (POP) • Community Sea Ice Model (CSIM)• Community Land Model (CLM)• North American Regional Climate Change
Assessment Project (NARCCAP)• Distribution point for models, data processing tools
Portal located at NCAR, data nodes at LANL, NCAR, NERSC, ORNL
7
Main ESG Portal Statistics
Data holdings:• 198 TB• 1,032,000 files• 6+ years of joint
DOE/NSF climate modeling experiments
• Recently added 6-hourly CCSM datasets
13,000 registered users
Downloads to date:• 100 TB• 2,400 downloads of
CCSM code & input datasets
8
C-LAMP Portal
Carbon-Land Model Intercomparison Project (C-LAMP)• Evaluation of
biogeochemical models A sub-project of the SciDAC
CCSM project, using computational resource of the ORNL LCF Computational Climate End Station (CCES)
Portal located at ORNL Currently used by working
group, plan to open up to public soon
Data holdings:• 149 GB• 1,575 files
9
Developing and Deploying the Next Generation Earth System Grid
Climate community plans (CMIP5) drive ESG to a new scale of operation
CMIP5 also gives us an aggressive timeline Initial ESG-CET focus on addressing core needs for…
• New software architecture• Extending data model• Federation mechanisms for users, (meta)data
Future work on…• Analysis and visualization services• Extending to WG2 (impacts) and WG3 (mitigation)
user communities
10
Gateway Web Application
The next-generation gateway has been extensively revised…
Provides a better user experience for searching and browsing
Global federation of metadata, user authentication
Richer metadata model
11
New Data Node with Greatly Expanded Capabilities
Next-generation data nodes provide much more capability than current production system
Data publication and metadata harvesting
Server-side processing capabilities based on LAS, Ferret, CDAT
Data delivery Integrated with security
infrastructure
12
New Security Architecture forSingle Sign-On Across Federation
Single sign-on (SSO) for both web and rich client access• OpenID for web SSO• MyProxy (PKI) for
rich client SSO Federation of many
sites, some already having their own user bases
Web services for user attributes retrieval
13
ESG Domain Model and Architecture for Federation
Comprehensive domain object model, including • Scientific metadata• Location metadata• Access Control metadata• Metrics metadata
Collaboration with Curator, MetaFor projects on…• Ontologies (being integrated
into ESG search & browse)• Hierarchical grid specifications• Metadata for atmospheric
dynamical cores Extract search & discovery
metadata, share through OAI-PMH for federation
14
BeStMan: New Tool for Accessing Deep Storage
Storage Resource Manager (SRM) spec. developed by Open Grid Forum
BeStMan is Berkley implementation of SRM standard, in collaboration with SciDAC SDM Center
Provides end-to-end management of complex data movements
• Many files, large files Data Mover Lite (DML) GUI client Manages data movement for
production ESG Basis for next-generation bulk
data movement solution
Tomcat servlet engine Tomcat servlet engine
MCSMetadata Cataloguing Services
MCSMetadata Cataloguing Services
RLSReplica Location Services
RLSReplica Location Services
SOAP
RMI
MyProxyserver
MyProxyserver
MCS client
RLS client
MyProxy client
GRAMgatekeeper
GRAMgatekeeper
CASCommunity Authorization Services
CASCommunity Authorization Services
CAS client
diskMSS
Mass Storage System
HPSSHigh PerformanceStorage System
disk
HPSSHigh PerformanceStorage System
disk
disk
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
BeStManStorage Resource
Management
gridFTP
gridFTP
gridFTPserver
gridFTPserver
gridFTPserver
gridFTPserver
gridFTPserver
gridFTPserver
gridFTPserver
gridFTPserver
openDAPgserver
openDAPgserver
gridFTPStripedserver
gridFTPStripedserver
LBNL
LLNL
ISI
NCAR
ORNL
ANL
BeStManStorage Resource
Management
BeStManStorage Resource
Management
15
Outreach, Presentations and Publications
Representing ESG’s work and perspective to the world-wide climate, computer science, and SciDAC communities…
14 papers
30 presentations
8 posters
21 other outreach activities
