Co-Design Breakout

A. Maccabe & M. Sato

Park Vista HotelGatlinburg, TennesseeSeptember 5-6, 2014

Participants

• Dr. Mitsuhisa Sato, RIKEN• Barney Maccabe, ORNL

• Robert Harrison, BNL• Galen Shipman, ORNL• Scott Klasky, ORNL• Shaun Gleason, ORNL• Dr. Kenji Ono, RIKEN,

AICS• Jeff Larkin, Nvidia

• Bronson Messer, ORNL

• Tjerk Straatsma, ORNL (Materials)• Hooney Park, ORNL (Life Sciences)• Sadaf Alam, CSCS (Life Sciences)• Pat Worley, ORNL (Global Change)• Seung-Hwan Lim, ORNL

(Engineering)• Jeremy Archuleta, ORNL

(Engineering)• David Bernholdt, ORNL (Fusion)• Miwako Tsuji, RIKEN

Break-out Agenda

Time Activity Who

8:30 Overview of the breakout Send out emissaries

Life Sciences: Hooney Park,Materials by Design: Tjerk Straatsma,Global Change Prediction: Pat Worley,Computational Engineering: Jeremy Archuleta,Nuclear Sciences: Miwako TsujiFusion Sciences: David Bernholdt

8:45 Robert Harrison,Scott Klasky,Kenji Ono, Mitsuhisa Sato

10:00 Break

10:30 Introductions of remaining participants with reactions to overview talks

11:30 Identification of key challenges, from a computer science perspective

Break-out Agenda (continued)

Time Activity

Noon Lunch

1:00 Re-write charge questions

1:30 Draft answers to charge questions

3:00 Break

3:30 Finalize answers to charge questions

4:30 Integrate issues identified by emissaries

5:30 Adjourn

Co-design• Models for co-design

– Embedding• Embed computer scientists in each application team• Embed application scientists in each computer science team

– Define and provide abstractions• possibly subject to abuse• Obsolescence

• Symmetry: What’s good for Apps is good for CS– CS needs abstract representations of applications

• Benchmarks, mini apps, skeletons, motifs, etc

– Apps need abstract representations of machines• Programming models and languages (parcels, X10, Chapel), simulators, performance prediction

tools (e.g., Aspen), etc

• The role of software engineering?– What is the research challenge?

• Why co-design now?– Computation is changing– Explore flexibility on both sides: apps and systems

• Re-examine boundaries

– Need to ensure continuity into the future

Examples of Systems co-design questions

• Hardware/NIC– What if the NIC can inject messages directly into an L1-cache?– What if the NIC provides computational capabilities?

• Collectives, matching, atomics, etc

• Streaming remote data to memory (bypass storage system)?• Application profiles to guide compiler optimizations?• How to manage different types of memory/storage?

– What is the memory access pattern?– How much cache?– How much NVRAM?

• Which data reduction strategies can be deployed in storage or communication?

• Where can we tolerate overheads?– OS Noise, etc

Programming Models• Programming models must be more holistic

– Encompass more than nodes and machines• Not MPI+X, but integration of I/O, storage, rest of world

– End-to-end design• Expose the whole context for computation so we can address the real problem• No point in doing co-design in areas that aren’t the bottleneck

• We need a hierarchy of abstractions– MPI + X

• Good for nodes & machines• Misses the broader context

– Hadoop • Encompasses the full data context and enables moving computation to data• Too inefficient for scientific applications

• Network to the outside world– On-node interconnect, In-machine interconnect, Machine room

interconnect, “rest of world”

• Multiuser, Interactive Supercomputing

Commodity – why it’s hard to define the programming model

• What is the right commodity– CPU + Cell did not become the commodity– CPU + GPU has shown to be much more sustainable

commodity

• What is the right level for commodity– Commodity within a node (individual

components/processors)– Commodity in software layers (e.g., Linux)– Commodity in infrastructure (e.g., cloud technologies)

Breakout Charge Questions, continued

1. What technical breakthroughs in science and engineering research can be enabled by exascale platforms and are attractive targets for Japan-US collaboration over the next 10 years?Exascale Challenges– Support for many core architectures– Communication layers e.g., PGAS– Application composition (beyond communication)– Memory / storage hierarchy (including staging and file systems)– Workflow management– Lightweight / micro kernels

– Energy efficient scheduling

– Performance measurement, modeling and prediction

– Resilience: Fault models and minimizing duplication in detection and response

Breakout Charge Questions, continued

2. What is the representative suite of applications systems in your research area, available today, which should form the basis of your co-design communication with computer architects application teams?

a) How are these applications currently constrained by compute and data resources, programming models, or available software tools?

b) What are the gaps in available applications and application workflows, and requirements to fill these gaps?

c) Which of these are ripe for collaboration within the context of Japan-US cooperation?

– I/O systems (and frameworks), e.g., ADIOS– Communication layers, e.g., PGAS, UCCS– Application composition, e.g., COMPOSE/Hobbes– Visualization and analysis services– Simulation, emulation and modeling tools

Breakout Charge Questions, continued

3. How can the application systems research community, represented by a topical breakout at this workshop, constructively engage the vendor community in co-design?

a) How should these various aspects of the application and architecture be optimized for effective utilization of exascale compute and data resources?

b) Consider all aspects of exascale application: formulation and basic algorithms, programming models & environments, data analysis and management, hardware characteristics.

– Full engagement of vendors and application teams, – Connected to procurement process

• CAAR at ORNL (one way, applications adapt to architecture)• Co-design for post-K (two way, apps influence architecture and vice versa)

Breakout Charge Questions, continued

4. How can you best manage the “conversations” with computer designers/architects application developers around co-design such that (1) they are practical for computer design, and (2) the results are correctly interpreted within both communities?

a) What are the useful performance benchmarks from the perspective of your domain?

b) Are mini-apps an appropriate and/or feasible approach to capture your needs for communication to the computer designers?

c) Are there examples of important full applications that are an essential basis for communication with computer designers?

d) Can these be simplified into skeleton apps or mini-apps to simplify and streamline the co-design conversation

Breakout Charge Questions, continued

5. Describe the most important programming models and environment in use today within your community and characterize these as sustainable or unsustainable.

a) Do you have appropriate methods and models to expose application parallelism in a high-performance, portable manner?

b) Are best practices in software engineering often or seldom applied?

c) Going forward, what are the critically important programming languages?

d) On which libraries and/or domain-specific languages (DSL) is your research community dependent?

e) Are new libraries or DSL’s needed in your research domain?

f) Are these aspects of your programming environment sustainable or are new models needed to ensure their availability into the future?

Breakout Charge Questions, continued

6. Does your community have mature workflow tools that are implemented within leadership computing environments to assist with program composition, execution, analysis, and archival of results? If no, what are your needs and is their opportunity for value added?

a) For example, do you need support for real-time, interactive workflows to enable integration with real-time data flows?

Breakout Charge Questions, continued

7. What are the new programming models, environments and tools that need to be developed to achieve our science goals with sustainable application software?

Breakout Charge Questions, continued

8. Is there a history, a track record in your research community for co-design for HPC systems in the installed machines in the past, and is there any co-design study done for these systems to document the effectiveness of co-design?