Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
BSC 2012 © 2012 IBM Corporation
Developing Parallel Applications with the
Eclipse Parallel Tools Platform
Greg Watson IBM STG
BSC 2012 © 2012 IBM Corporation 2
Parallel Tools Platform Enabling Parallel Application Development
parallel tools pla*orm (ptp) Tools to assist new
breed of programmers to develop parallel
programs
Best practice tools for experienced parallel
programmers
Improve parallel tools and the productivity of
tool developers
Leverage Eclipse ecosystem and community for
development and support
Provide focal point for parallel tool development for a broad
range of architectures
BSC 2012 © 2012 IBM Corporation 3
PTP Timeline
2005
2006
2008
2012
Eclipse 3.1
Callisto 3.2
Europa 3.3
Ganymede 3.4
Galileo 3.5
Helios 3.6
Indigo 3.7
PTP 5.0 June ’11
PTP 4.0 June ’10
PTP 3.0 Nov ’09
PTP 2.0 Mar ’08
PTP 1.0 Mar ’06
PTP 2.1 Nov ’08
PTP 1.1 Feb ’07
Creation Apr ’05
2010
2007
2009
18 committers from 11 different organizations
Nearly 65,000 downloads since Feb 24 “Eclipse for Parallel Application Developers”
SR1 Sep ’10
SR2 Feb ’11
2011
2013
PTP 6.0 June ’12
Juno 4.2
PTP 6.0 is part of the Eclipse “Juno” simultaneous release of over 70
Eclipse projects
SR1 Sep ’11
SR2 Feb ’12
BSC 2012 © 2012 IBM Corporation
PTP Application Development Cycle
4
Application Execution
Application Debugging
Coding & Static Analysis
Dynamic & Performance Analysis
BSC 2012 © 2012 IBM Corporation
Eclipse provides a wide variety of coding assistance tools – Project management, Editing and formatting, Navigation, Advanced
searching, Refactoring, Version control C/C++ Development Tools (CDT)
– Standard (Makefile) and managed builders, Support for arbitrary toolchains, Visual debugging using GDB, High level views (outline view, call hierarchy, type hierarchy, include browser), Refactorings
Parallel Tools Platform (PTP) – Fortran, New project wizards (MPI, OpenMP) Content Assist, Hover help,
Built-in API descriptions (MPI, OpenMP, LAPI, UPC), Location of parallel “artifacts” in code (MPI, OpenMP, PAMI, and UPC), Barrier analysis, Deadlock detection
5
Coding & Static Analysis
BSC 2012 © 2012 IBM Corporation
Assistance tools to increase productivity of parallel programmers
– New project wizards (MPI, OpenMP)
– Content Assist (command/API completion), hover help, built-in API help descriptions in an html help “view” (MPI, OpenMP, LAPI, UPC)
– Location of parallel “artifacts” in code: MPI, OpenMP, LAPI APIs, and UPC
6
Coding & Static Analysis Parallel Language Development Tools (PLDT)
BSC 2012 © 2012 IBM Corporation
Static analysis tools Provides “advanced” error checking Not always 100% accurate MPI analysis
– Barrier deadlock detection
OpenMP analysis – Concurrency analysis – Common OpenMP problems
7
Coding & Static Analysis Parallel Language Development Tools (2)
BSC 2012 © 2012 IBM Corporation
Fortran Development Tools Photran project
– http://eclipse.org/photran – Tech lead at UIUC, Jeff Overbey – More UIUC students are contributing – Merged with PTP in 2009 – Photran 8.0 released in Jun ’12
with PTP 6.0 / Eclipse Juno Photran features:
– Supports Fortran 77-2008 – Syntax-highlighting editor – GUI interface to gdb – Makefile-based compilation – Compiler error extraction – Outline view – Open declaration – Fortran refactorings – C preprocessor support
8
BSC 2012 © 2012 IBM Corporation
Fortran Refactoring
9
Image courtesy Jeff Overbey, UIUC
Some samples: Rename
– Change name of every use of a variable, function, etc. – Only proceeds if new name will be legal
Extract procedure – Moves statements into a new subroutine, replacing
statements with a call – Local variables are passed as arguments
Introduce implicit none – Adds an ‘implicit none’ statement – Adds explicit variable declarations
Photran 7.0: 31 refactorings
BSC 2012 © 2012 IBM Corporation
PTP Application Development Cycle
10
Application Execution
Application Debugging
Coding & Static Analysis
Dynamic & Performance Analysis
BSC 2012 © 2012 IBM Corporation
Open MPI
11
Application Execution Launching & Monitoring
PBS/Torque
SLURM
MPICH2
Improves visibility into target system Single point of interface for launching and
control Manages interaction with different runtime
systems and job schedulers
LoadLeveler
PE
BSC 2012 © 2012 IBM Corporation
Extensible framework for launching & monitoring – System and node status information
– Job status (e.g. position in queue) & application status
– Job submission & control
– Debugger launch Configuration files to support different resource managers
– Job schedulers (LoadLeveler, PBS, Torque, SLURM, GridEngine) – Interactive runtimes (e.g. PE, Open MPI, MPICH2, MVAPICH)
– Systems (AIX, Linux, Power, x86, BG/Q, Cray) Local or remote system support
– Command-line tools executed locally or via ssh connection
12
Application Execution Target Configuration Framework
BSC 2012 © 2012 IBM Corporation
PTP Application Development Cycle
13
Application Execution
Application Debugging
Coding & Static Analysis
Dynamic & Performance Analysis
BSC 2012 © 2012 IBM Corporation 14
Application Debugging PTP Parallel Debugger
Mid-scale integrated debugger Tightly integrated with Eclipse Supports debugging multiple jobs
simultaneously Utilizes backend debugger (e.g. gdb) for
low level operations Targeted at SPMD programming models Supports mixed MPI & thread
debugging Single process and group operations Platform for building new debugging
paradigms
BSC 2012 © 2012 IBM Corporation
Scalable debugger using multicast reduction network Integrated with PTP and launched using target configurations Supports basic debug dommand Uses gdb on backend
15
Application Debugging IParallel Debugger Architecture
process process
process
gdb
sdm
gdb gdb
process process
process
gdb
sdm
gdb gdb
process process
process
gdb
sdm
gdb gdb
sdm master
BSC 2012 © 2012 IBM Corporation
PTP Application Development Cycle
16
Application Execution
Application Debugging
Coding & Static Analysis
Dynamic & Performance Analysis
BSC 2012 © 2012 IBM Corporation 17
Dynamic & Performance Analysis Dynamic Analysis Tools
17
Perform analysis on the running application using external tools
Generate results that must be brought back into Eclipse as part of the development workflow
May require external tool for visualization or other purposes
BSC 2012 © 2012 IBM Corporation
Dynamic & Performance Analysis Tools Integrated with PTP
Tuning and Analysis Utilities (TAU) – Instrumentation and transparent re-build of application
executable – Execution of profiled application and collect performance data – Performance data visible in UI
– Launches paraprof visualization client from Eclipse Graphical Explorer of MPI Programs (GEM)
– Formal Dynamic Verification of MPI Applications
– Detects all deadlocks, assert violations, MPI object leaks, and default safety properties
– Matches sends and receives – Allows post-verification review of highlighted bugs
– Works with a variety of MPI implementations
18
BSC 2012 © 2012 IBM Corporation
Conclusion
19
Complexity of peta-scale application development is becoming clearer to developers
– IDEs are starting to be seen as key to improving HPC developer productivity
– IDEs are becoming more accepted in the HPC community
Eclipse is now a very mature platform – Has more users than VS – Hundreds of commercial products based on Eclipse
PTP is the only available platform on which to build an HPC development environment
– PTP developer community is starting to reach critical mass – Beginning to see a diverse user base forming
It’s Free!
BSC 2012 © 2012 IBM Corporation
Online Information
Information about PTP – Main web site for downloads, documentation, etc.
– http://eclipse.org/ptp
– Developers’ wiki for designs, planning, meetings, etc. – http://wiki.eclipse.org/PTP
– Articles and other documents – http://wiki.eclipse.org/PTP/articles
20
BSC 2012 © 2012 IBM Corporation
Community
PTP Mailing lists – Major announcements (new releases, etc.) - low volume
– http://dev.eclipse.org/mailman/listinfo/ptp-announce – User discussion and queries - medium volume
– http://dev.eclipse.org/mailman/listinfo/ptp-user – Developer discussions - high volume
– http://dev.eclipse.org/mailman/listinfo/ptp-dev
Meetings – Monthly developer conference calls
– Announced on ptp-dev list – Monthly user conference calls
– Announced on ptp-user list – Annual developer/user meeting
– Tentatively scheduled for Sept 18/19 in Chicago
21
BSC 2012 © 2012 IBM Corporation
Demo
22
BSC 2012 © 2012 IBM Corporation
Thank You
23