Upload
cleopatra-richard
View
214
Download
0
Embed Size (px)
Citation preview
Your university or experiment logo here
The Protocol ZooA Site Presepective
Shaun de Witt, STFC (RAL)
Your university or experiment logo here
Site Perspective on Protocol Zoo
• Outline– Dealing with the Animals
• Problems associated with multiple protocols– Replacing Lions with Tigers
• Impact of change– Why does WLCG want a Fossa while the rest of the world
wants Giraffes and Rhinos?• Xrootd/HTTP/WebDAV
– What happens when we need to clean the cages• Migrating to new storage technology
Your university or experiment logo here
XRoot GridFTP SRM
RFIO
DCAP
S3
HTTP
WebDAV
FILE
Your university or experiment logo here
Dealing with the Animals
• In the ‘bad’ old days…– Sites had to deal with a mix of protocols
• SRM/GridFTP were common• Some mix of file/dcap/rfio/… and experiments needed to handle
the differences• Internal protocols were somewhat optimised for the storage
systems (good performance) but lacked some features (eg security)
• SRM seen as a bottleneck and didn’t fully deliver on its promises
• Today– The zoo is more homogeneous
• SRM/GridFTP/xroot at all sites, many also offer WebDAV and/or http
• Internal protocols largely deprecated– But makes life harder for zoo keepers
• Difficulty linking protocol server logs to storage system logs• Differing log formats…
Your university or experiment logo here
Your university or experiment logo here
Replacing Lions with Tigers
• Speed of Change– Change in Protocol change is SLOW!!!!
• Development for each Storage System• Validation/Verification• Deployment at all sites• Removing old protocols (PLEASE!!!)
• Site Admins need to understand the protocol– Assessing hardware requirements (and purchasing)– Understanding logs– Identifying failure modes and integration into monitoring
• Protocols supporting fallback can make diagnostics more difficult
• Why did the file not get copied from my site?
Your university or experiment logo here
Your university or experiment logo here
• Many sites are looking to extend their customer base– Non HEP/PP– XRootD works well for WLCG (and offers many benefits)…
• Scalability, fallback, integration with ROOT, …– … but selling XRootD outside of HEP is difficult– Need some sort of standard protocol– Most mature experiments can use gridFTP for bulk
movement…• But in some storage systems it MUST be mediated by SRM
– … but it’s not what they want for analysis• WLCG Movement towards standards…
– Movement towards HTTP/WebDAV across the board is appreciated
– But WebDAV still is not widely adopted outside of HEP
Your university or experiment logo here
Cleaning the Cages
Your university or experiment logo here
• At RAL we are looking to deploy a more modern disk-only storage system– Based around object storage– To be used by many communities/projects
• ISIS, DIAMOND, CLF, CCLRC,H2020 projects– Will also be used for WLCG– Protocols still a problem…
• Want a file put in through protocol X to be directly accessible through any other supported protocol
• Don’t really want protocol bridges– Again use of standards would be helpful
• Consider S3 (or CDMI – but this has its own problems)