ADIOS IO introduction

YufeiDec 10

System at Oak Ridge

• 672 OSTs• 10 Petabytes of storage• 60 GB/sec = 480 Gbps aggregate performance (theoretical)• 225,000 compute cores

Prior Work

• Driver-level work for disk access• System-level strategies for effective buffering• Alternative file organizations (striping as Lustre)• Novel method

        data staging        alternative file formats or organizations        better ways to organize and update file metadata

Interference

 

Adaptive I/O

• Dynamic and proactive methods for managing IO interference

        shift work from heavily used areas of the storage system to those that are more lightly loaded.

IO Middleware

Internal Interference Effect

 

External Interference Effect

 

Imbalanced Concurrent Writers

 

Adaptive IO

 

Components

• Writer• Sub-Coordinator• Coordinator

Experimental EvaluationMPI-IO vs Adaptive IO

 

FAQ