Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
B2STAGE
How to shift large amounts of data
Version 3
June 2014
1
www.eudat.eu | http://www.eudat.eu/b2stage B2STAGE Training
B2STAGE is part of EUDAT...
a pan-European initiative building a sustainable
cross-disciplinary and cross-national data
infrastructure providing a set of shared services for
accessing and preserving research data
supporting multiple research
communities by working closely
with them to deliver these
technical services as part of the
EUDAT Collaborative Data
Infrastructure (CDI) www.eudat.eu | http://www.eudat.eu/b2stage B2STAGE Training
A truly pan-European Infrastructure
general data centres
community centres
representing all the associated
community data centres
Research Communities
National Data centres
Technology providers
Offering permanence,
persistence, reliability
and long term
solutions
www.eudat.eu | http://www.eudat.eu/b2stage B2STAGE Training
Where is B2SHARE in the EUDAT suite?
B2STAGE represents an extension of the B2SHARE and offers communities a light approach to ingest and replicate data. Data ingested through B2STAGE is registered with a Persistent Identifier (PID) using the same mechanism adopted by B2SAFE www.eudat.eu | http://www.eudat.eu/b2stage B2STAGE Training
A reliable, efficient, lightweight and easy-to-use service to ship large amounts of research data between EUDAT storage resources and workspace areas of high-performance computing systems.
5
B2STAGE is... B2STAGE does...
B2Stage can be used to simply ingest community data onto EUDAT resources using a high performance protocol, like GridFTP.
www.eudat.eu www.eudat.eu/b2stage
Why use B2STAGE?
Research challenges are getting larger and more complex : full-Earth climate simulation, coupled simulations of multiple organs in the human body, seismic analyses of earthquakes at continental scale and
Researchers’ data and compute demands are rising fast
Efficient shipping of data to high performance computing (HPC) workspaces is essential especially in distributed computing, where resources are geographically dispersed
6 www.eudat.eu www.eudat.eu/b2stage
Why use B2STAGE?
Facilitate transfer of large data collections from EUDAT storage resources to external HPC facilities.
Offers reliable, efficient, easy-to-use tools to manage data transfers.
Provides the means to re-ingest computational results back into the EUDAT infrastructure.
Ingests data sets onto EUDAT resources for long-term preservation.
7 www.eudat.eu www.eudat.eu/b2stage
Who can use B2STAGE?
Researchers can transfer large data collections from EUDAT storage resources to HPC facilities for processing.
Community Managers can replicate community data through a lightweight service and ingest data sets to EUDAT storage resources for long term preservation.
8 www.eudat.eu www.eudat.eu/b2stage
Why is B2STAGE unique?
The DSS is the only tool handling data transfer using PIDs.
Easy, reliable and fast solution for data ingestion and transfer onto and from EUDAT resources.
9 www.eudat.eu www.eudat.eu/b2stage
How can you use B2STAGE?
10
For more information please email: [email protected]
EUDAT offers B2STAGE to all registered researchers and interested communities enabling them to make use of the service to stage data out of EUDAT, and ingest computational results back.
Access to remote HPC facilities should be negotiated and
arranged by individual users in parallel.
To help researchers to use the B2STAGE service, EUDAT offers documentation, educational material and a service helpdesk.
www.eudat.eu www.eudat.eu/b2stage
B2STAGE User communities
VPH Community to ingest data onto EUDAT resources
Approximately 12TB will be ingested thought this service
NeuGRID and INCF are considering its adoption to replicate data
Collaboration with other e-infrastructures
VPH to transfer data across EUDAT, PRACE, EGI
11 www.eudat.eu www.eudat.eu/b2stage
B2STAGE currently...
The current version of B2STAGE offers:
data staging functionalities to easily and efficiently ship data across EUDAT storage resources and HPC facilities;
a powerful mechanism to ingest data onto EUDAT resources;
a script to facilitate the staging, the ingestion and the retrieving of PID information of transferred data.
12 www.eudat.eu www.eudat.eu/b2stage
Where does B2STAGE fit within EUDAT?
13
B2STAGE represents an extension of the B2SHARE and offers communities a light approach to ingest and replicate data. Data ingested through B2STAGE is registered with a Persistent Identifier (PID) using the same mechanism adopted by B2SAFE
www.eudat.eu www.eudat.eu/b2stage
Future features...
Optimization of transfers on the basis of data location within the EUDAT infrastructure (under evaluation).
Improvement of user experience with the Data Staging Script (i.e. data path autocompletion, multi-pid parallel handling, etc.).
Foster the collaboration with EGI and PRACE to develop cross-infrastructure usage: the B2STAGE will be the main service to enable the
interoperability of these infrastructures.
14 www.eudat.eu www.eudat.eu/b2stage
Thanks
For more info: www.eudat.eu/b2stage www.eudat.eu | http://www.eudat.eu/b2stage B2STAGE Training