About SSEC
The University of Wisconsin Space Science and Engineering center (SSEC) is a research and development center focusing on geophysical research and technology to enhance our understanding of the atmosphere of Earth, the other planets in our Solar System, and the cosmos. Major SSEC Initiatives § atmospheric studies of Earth and other planets § interactive computing, data access, and image processing § spaceflight hardware development and fabrication Noted Scientific Work § satellite-based and other weather observing instruments § remote sensing applications in earth and atmospheric science § spaceflight instrumentation § planetary meteorology § data analysis and visualization § diagnostic and numerical studies of the atmosphere
2
What Shapes our Storage Solutions?
§ The SSEC contains many groups and projects, often with different funding sources. Individual groups and projects ultimately control their own expenditures. § The Technical Computing Group provides an infrastructure to share and combine
these resources. § Atmospheric research via satellite meteorology is the major focus of SSEC.
§ This has large data needs. Files are typically fairly large – we use 100MB files as a generic representation of a lot of our data.
§ Most projects are research, and not strictly operational so high availability is usually not required, but data integrity is critical.
3
Uniform Components,Varied Solutions
NAS
4
SAS Cables
SAS Cables
Split Mode
MDS/MDT
OSS 1
OSS 2
FDR-‐10
FDR-‐10
FDR-‐10
OST0 OST1 OST2
OST3 OST4 OST5
High Performance Storage Infrastructure
SSEC has recently created a generally available storage infrastructure, largely focused on Lustre. Making large file systems available generally, and not just on compute clusters has changed how people work with large data. Key components include: § Infiniband
§ Once you have taken the step to infiniband infrastructure (which was required by some of our NWP needs) it’s hard to pay more for lower performance with ethernet
§ Lustre Routers § Allow connecting the infiniband resources (storage, compute clusters) to 10G and
1G ethernet systems. § Patchless Lustre Clients
§ This is particularly useful for extending lustre filesystems to non-cluster nodes. Linux is the main focus here, our ‘clients’ are what many would consider servers, typically ethernet connected.
§ BETA - Providing a mountable file system aggregating all resources (NAS, Gluster, Lustre) as a single file system to desktops. § Work in progress – custom application in house by a research group
5