View
9
Download
0
Category
Preview:
Citation preview
LLNL-PRES-806064This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC
github.com/spack/spack
ManagingHPCSoftwareComplexitywithSpack
Supercomputing2017Full-dayTutorialNovember,2017Denver,Colorado
LLNL-PRES-8060642github.com/spack/spack
Materials: Downloadthelatestversionofslidesandhandoutsat:
http://spack.readthedocs.io
Slidesandhands-onscriptsareinthe“Tutorial”Sectionofthedocs.
§ Spack GitHubrepository: http://github.com/spack/spack
§ Spack Documentation: http://spack.readthedocs.io
TutorialMaterials
Tweet at us!@spackpm #SC17
LLNL-PRES-8060643github.com/spack/spack
ToddGamblinGregoryBeckerGregLeeMattLegendre
MassimilianoCulpo
AdamStewart
MarioMelara
TutorialPresenters
LLNL-PRES-8060644github.com/spack/spack
Thecomplexityoftheexascale ecosystemthreatensproductivity.
§ Everyapplicationhasitsownstackofdependencies.§ Developers,users,andfacilitiesdedicate(many)FTEstobuilding&porting.§ Oftentradereuseandusabilityforperformance.
80+ software packagesx5+ target architectures/platforms
Xeon Power KNLNVIDIA ARM Laptops?
x
Up to 7 compilersIntel GCC Clang XL
PGI Cray NAGx
= up to 1,260,000 combinations!
15+ applications
x10+ Programming Models
OpenMPI MPICH MVAPICH OpenMP CUDAOpenACC Dharma Legion RAJA Kokkos
2-3 versions of each package + external dependenciesx
We must make it easier to rely on others’ software!
LLNL-PRES-8060645github.com/spack/spack
§ Someone’shomedirectory?
§ LLNL?LANL?Sandia?ANL?LBL?TACC?— Environmentsatlarge-scalesitesareverydifferent.
§ WhichMPIimplementation?
§ Whichcompiler?
§ Whichdependencies?
§ Whichversionsofdependencies?— Manyapplicationsrequirespecificdependencyversions.
Whatisthe“production”environmentforHPC?
Real answer: there isn’t a single production environment or a standard way to build.Reusing someone else’s software is HARD.
LLNL-PRES-8060646github.com/spack/spack
§ HowtoinstallSpack:
§ Howtoinstallapackage:
§ HDF5anditsdependenciesareinstalledwithintheSpack directory.
§ Unliketypicalpackagemanagers,Spack canalsoinstallmanyvariantsofthesamebuild.— Differentcompilers— DifferentMPIimplementations— Differentbuildoptions
Spack isaflexiblepackagemanagerforHPC
GetSpack!http://github.com/spack/spack
$ git clone https://github.com/spack/spack$ . spack/share/spack/setup-env.sh
$ spack install hdf5
@spackpm
LLNL-PRES-8060647github.com/spack/spack
PeoplewhowanttouseordistributesoftwareforHPC!
1. EndUsersofHPCSoftware— InstallandrunHPCapplicationsandtools
2. HPCApplicationTeams— Managethird-partydependencylibraries
3. PackageDevelopers— Peoplewhowanttopackagetheirownsoftwarefordistribution
4. UsersupportteamsatHPCCenters— PeoplewhodeploysoftwareforusersatlargeHPCsites
WhocanuseSpack?
LLNL-PRES-8060648github.com/spack/spack
Spack isusedworldwide
Sessions at spack.readthedocs.io by location, since April 2017
LLNL-PRES-8060649github.com/spack/spack
§ InNovember2015,LLNLprovidedmostofthecontributionstoSpack
§ Sincethen,we’vegonefrom300toover2,100packages
§ Mostpackagesarefromexternalcontributors!
§ Manycontributionsincore,aswell.
§ WearecommittedtosustainingSpack’s opensourceecosystem!
ContributionstoSpack aregrowing!
LLNL-PRES-80606410github.com/spack/spack
§ 30+organizations140+contributorsSharingover1,500packages andgrowing
§ Otherusecases:— ARMusingforentirecompilerregressionsuite.— LIGO collaborationusingfordeployment— Intel usingSpack topackageMLsoftware— NERSC usingSpack onCori:Craysupport.— EPFL (Switzerland)contributingcorefeatures.— Fermi,CERN,BNL:highenergyphysics.— ANL usingSpack onproductionLinuxclusters.— NASA packaginganIcemodelcode.— ORNL workingwithusonSpack forCORAL.— Kitware:corefeatures,ParaView,Qt,UV-CDAT
support
TheSpack communitynowspansDOEandbeyond
3IBM Brand ElementsIBM Impact 2010 DESIGN GUIDELINES
IBM Brand Elements
IBM Logo
IBM has one of the world’s most recognizable logos. Maintaining its integrity is an essential part of protecting the brand. The 8-bar blue logois more than a signature to our ads or a name on our products andservices. To customers, it is a promise that represents trust, value, quality and advanced technology. In a world of confusing, competitiveads and product claims, the IBM logo is uniquely reassuring, conveyingauthenticity, reliability and quality. The IBM logo is present on every formof IBM communication.
Color of the IBM Logo
IBM has one logo, which may appear only in black, white, or IBM Blue.The logo should appear consistently and without alteration, as shownhere. On a white or light background, use the IBM logo in black or IBMBlue. On a black or dark background, use the IBM logo in white or IBMBlue. The color itself is an important symbol of IBM, and should bespecified for printed documents using Pantone 2718 as a spot colorwhenever possible. Use of this spot color is necessary when setting thelogo in IBM Blue on a colored background.
IBM Blue
PMS 2718CMYK C75 M43 Y0 K0RGB R75 G107 B175WEB 006699
1 white on black or dark background
2 black on white or light background
3 IBM blue on white or light background
4 IBM blue on black or dark background
1 2
3 4
LLNL-PRES-80606411github.com/spack/spack
§ Majornewfeatures:1. Binarypackagingsupport(we’llusethistoday)2. Aninterfaceforpassingbuildinfobetweenpackages(demoedtoday)3. Testdependencies4. Multi-valuedvariants5. Morecontrolovercompilerflags6. BetterhelpoutputforSpack’s manycommands7. Betterbuildoutputhandlinganderrordisplay8. Numerousotherbugfixesandspeedimprovements
§ Over2,100packages
§ Spack nowhasitsownGitHuborganization(https://github.com/spack)
Spack v0.11.0wasjustreleased
LLNL-PRES-80606412github.com/spack/spack
§ Features:— Build/testinfrastructuretobuildandvalidatepackagebuildsregularly— Hostinginfrastructuretomakeselectpackageconfigurationsavialable asbinaries— Supportforcreatingcustomisolated“environments”(cf.Conda,Nix,Virtualenv)— Betterdependencyresolution(concretization)andcompilersupport— Distributed-parallelbuilds
§ Collaborations:— WorkwithUSExascale Programcodeteamstocoordinatesoftwarereleases— WorkwithDOElabstobuildandtestsoftwareonlargesupercomputers— Continueworkingwithbroadercommunitytointegratecontributions
WhatisontheSpack roadmap?
LLNL-PRES-80606413github.com/spack/spack
Spack isnotthefirsttooltoautomatebuilds— Inspiredbycopiouspriorwork
1. “Functional”PackageManagers— Nix https://nixos.org/— GNUGuix https://www.gnu.org/s/guix/
2. Build-from-sourcePackageManagers— Homebrew http://brew.sh— MacPorts https://www.macports.org
OthertoolsintheHPCSpace:
§ Easybuild http://hpcugent.github.io/easybuild/— Aninstallation toolforHPC— FocusedonHPCsystemadministrators– differentpackagemodelfromSpack— Reliesonafixedsoftwarestack– hardertotweakrecipesforexperimentation
§ Hashdist https://hashdist.github.io
RelatedWork
LLNL-PRES-80606414github.com/spack/spack
1. Foreveryone: Welcome&Overview 8:30- 8:50
2. Foreveryone: BasicsofBuildingandLinking 8:50- 9:00
3. Foreveryone: Spack Basics(handson) 9:00- 9:45
4. Foreveryone: CoreSpack Concepts 9:45- 10:00
-- Break-- 10:00- 10:30
5. Foreveryone: Configuration(hands-on) 10:3011:00
6. Forpackagers: MakingyourownPackages(handson) 11:00– 12:00
-- TutorialsLunch(FourSeasonsBallroom)-- 12:00- 1:30
7. Forpackagers: AdvancedPackaging(buildsystems) 1:30– 2:15
8. Forpackagers: AdvancedPackaging(virtualdependencies:BLAS/LAPACK/MPI) 2:15-3:00
-- CoffeeBreak-- 3:30- 3:30
9. ForHPCcenters: DeployingSpack withLmod atHPCFacilities(hands-on) 3:30- 4:15
10. Foreveryone: BuildyourownpackageswithhelpfromtheSpack team! 4:15– 5:00
TutorialOverview(timesareestimates)
LLNL-PRES-80606415github.com/spack/spack
Pleasesubmityourfeedbackonthistutorialat:
Tutorialreviewsareonlinethisyear
http://bit.ly/sc17-evalYour reviews help us keep the tutorial going!
LLNL-PRES-80606416github.com/spack/spack
Building&LinkingBasics
LLNL-PRES-80606417github.com/spack/spack
What’sapackagemanager?
§ Spack isapackagemanager— Doesnot areplaceCmake/Autotools— PackagesbuiltbySpack canhaveany
buildsystemtheywant
§ Spack managesdependencies— Drivespackage-levelbuildsystems— Ensuresconsistentbuilds
§ Determiningmagicconfigurelinestakestime— Spack isacacheofrecipes
• Manages package installation• Manages dependency relationships• Drives package-level build systems
Package Manager
• Cmake, Autotools• Handle library abstractions• Generate Makefiles, etc.
High Level Build
System
• Make, Ninja• Handles dependencies among commands in a single build
Low Level Build
System
LLNL-PRES-80606418github.com/spack/spack
§ Staticlibraries:libfoo.a— .afilesarearchivesof.ofiles(objectfiles)— Linkerincludesneededpartsofastaticlibraryintheoutputexecutable— Noneedtofinddependenciesatruntime– onlyatbuildtime.— Canleadtolargeexecutables— Oftenhardtobuildacompletelystaticexecutableonmodernsystems.
§ Sharedlibraries:libfoo.so (Linux),libfoo.dylib (MacOS)— Morecomplexbuildsemantics,typicallyhandledbythebuildsystem— Mustbefoundbyld.so ordyld (dynamiclinker)andloadedatruntime
• Cancauselotsofheadacheswithmultipleversions— 2mainways:
• LD_LIBRARY_PATH:environmentvariableconfiguredbyuserand/ormodulesystem• RPATH:pathsembeddedinexecutablesandlibraries,sothattheyknowwheretofindtheirowndependencies.
Staticvs.sharedlibraries
LLNL-PRES-80606419github.com/spack/spack
§ API:ApplicationProgrammingInterface— Sourcecodefunctionsandsymbolnamesexposedbyalibrary— IfAPIofadependencyisbackwardcompatible,sourcecodeneednotbechangedtouseit— May needtorecompilecodetouseanewversion.
§ ABI:ApplicationBinaryInterface— Callingconventions,registersemantics,exceptionhandling,etc.— Definedbyhowthecompilerbuildsalibrary
• BinariesgeneratedbydifferentcompilersaretypicallyABI-incompatible.— Mayalsoincludethingslikestandardruntimelibrariesandcompilerintrinsicfunctions— Mayalsoincludevaluesofhard-codedsymbols/constantsinheaders.
§ HPCcode,includingMPI,istypicallyAPI-compatiblebutnotABI-compatible.— Causesmanybuildproblems,especiallyfordynamicallyloadedlibraries— OftenneedtorebuildtogetaroundABIproblems— LeadstocombinatorialbuildsofsoftwareatHPCsites.
APIandABICompatibility
LLNL-PRES-80606420github.com/spack/spack
1. Make(usuallyGNUMake)— https://www.gnu.org/software/make/
2. GNUAutotools— Automake: https://www.gnu.org/software/automake/— Autoconf: https://www.gnu.org/software/autoconf/— Libtool: https://www.gnu.org/software/libtool/
3. CMake:— https://cmake.org
3majorbuildsystemstobeawareof
Spack has built-in support for these plus Waf, Perl, Python, and R
LLNL-PRES-80606421github.com/spack/spack
§ ManyprojectsopttowritetheirownMakefiles.— Canrangefromsimpletovery
complicated
§ Makedeclaressomestandardvariablesforvariouscompilers— ManyHPCprojectsdon’trespectthem— Nostandardinstallprefixconvention— Makefiles maynothaveinstalltarget
§ AutomatingbuildswithMakeusuallyrequireseditingfiles— Typicaltousesed/awk/someother
regularexpressiontoolonMakefile— Canalsousepatches
MakeandGNUMake
<edit Makefile>make PREFIX=/path/to/prefix
Typical build incantation
None. Typically must edit Makefiles.
Configure options
Environment variablesCC CFLAGS LDFLAGSCXX CXXFLAGS LIBSFC FFLAGS CPPF77 FFLAGS
LLNL-PRES-80606422github.com/spack/spack
§ Threepartsofautotools:— autoconf:generatesaportable
configurescripttoinspectbuildhost— automake:high-levelsyntaxfor
generatinglower-levelMakefiles.— libtool:abstractionforsharedlibraries
§ Typicalvariablesaresimilartomake
§ Muchmoreconsistencyamongautotools projects— Wideuseofstandardvariablesand
configureoptions— Standardinstalltarget,staging
conventions.
Autotools
./configure --prefix=/path/to/install_dirmakemake install
Typical build incantation
./configure \--prefix=/path/to/install_dir \--with-package=/path/to/dependency \--enable-foo \--disable-bar
Configure options
Useful environment variablesCC CFLAGS LDFLAGSCXX CXXFLAGS LIBSFC FFLAGS CPPF77 FFLAGS
Environment variables
LLNL-PRES-80606423github.com/spack/spack
§ Gainingpopularity
§ Arguablyeasiertouse(fordevelopers)thanautotools
§ Similarstandardoptionstoautotools— differentvariablenames— Moreconfigurationoptions— Abstractsplatform-specific
detailsofsharedlibraries
§ MostCMake projectsshouldbebuilt“outofsouce”— Separatebuilddirectoryfrom
sourcedirectory
CMake
mkdir BUILD && cd BUILDcmake –DCMAKE_INSTALL_PREFIX=/path/to/install_dir ..makemake install
Typical build incantation
cmake \–D CMAKE_INSTALL_PREFIX=/path/to/install_dir \-D ENABLE_FOO=yes \-D ENABLE_BAR=no \..
Configure options
Useful environment variablesCMAKE_C_COMPILER CMAKE_C_FLAGSCMAKE_CXX_COMPILER CMAKE_CXX_FLAGSCMAKE_Fortran_COMPILER CMAKE_Fortran_FLAGSCMAKE_SHARED_LINKER_FLAGS CMAKE_EXE_LINKER_FLAGSCMAKE_STATIC_LINKER_FLAGS
Common –D options
LLNL-PRES-80606424github.com/spack/spack
Spack Basics
LLNL-PRES-80606425github.com/spack/spack
§ Eachexpressionisaspecforaparticularconfiguration— Eachclauseaddsaconstrainttothespec— Constraintsareoptional– specifyonlywhatyouneed.— Customizeinstallonthecommandline!
§ Specsyntaxisrecursive— Fullcontroloverthecombinatorialbuildspace
Spack providesaspecsyntaxtodescribecustomizedDAGconfigurations
$ spack install mpileaks unconstrained$ spack install mpileaks@3.3 @ custom version$ spack install mpileaks@3.3 %gcc@4.7.3 % custom compiler$ spack install mpileaks@3.3 %gcc@4.7.3 +threads +/- build option$ spack install mpileaks@3.3 cppflags="-O3 –g3" setting compiler flags$ spack install mpileaks@3.3 os=CNL10 target=haswell setting target for X-compile$ spack install mpileaks@3.3 ^mpich@3.2 %gcc@4.9.3 ^ dependency information
LLNL-PRES-80606426github.com/spack/spack
`spack list`showswhatpackagesareavailable
$ spack list==> 303 packages.activeharmony cgal fish gtkplus libgd mesa openmpi py-coverage py-pycparser qt tcladept-utils cgm flex harfbuzz libgpg-error metis openspeedshop py-cython py-pyelftools qthreads texinfoapex cityhash fltk hdf libjpeg-turbo Mitos openssl py-dateutil py-pygments R the_silver_searcherarpack cleverleaf flux hdf5 libjson-c mpc otf py-epydoc py-pylint ravel thriftasciidoc cloog fontconfig hwloc libmng mpe2 otf2 py-funcsigs py-pypar readline tkatk cmake freetype hypre libmonitor mpfr pango py-genders py-pyparsing rose tmuxatlas cmocka gasnet icu libNBC mpibash papi py-gnuplot py-pyqt rsync tmuxinatoratop coreutils gcc icu4c libpciaccess mpich paraver py-h5py py-pyside ruby trilinosautoconf cppcheck gdb ImageMagick libpng mpileaks paraview py-ipython py-pytables SAMRAI uncrustifyautomaded cram gdk-pixbuf isl libsodium mrnet parmetis py-libxml2 py-python-daemon samtools util-linux automake cscope geos jdk libtiff mumps parpack py-lockfile py-pytz scalasca valgrindbear cube gflags jemalloc libtool munge patchelf py-mako py-rpy2 scorep vimbib2xhtml curl ghostscript jpeg libunwind muster pcre py-matplotlib py-scientificpython scotch vtkbinutils czmq git judy libuuid mvapich2 pcre2 py-mock py-scikit-learn scr wgetbison damselfly glib julia libxcb nasm pdt py-mpi4py py-scipy silo wxboost dbus glm launchmon libxml2 ncdu petsc py-mx py-setuptools snappy wxpropgridbowtie2 docbook-xml global lcms libxshmfence ncurses pidx py-mysqldb1 py-shiboken sparsehash xcb-protoboxlib doxygen glog leveldb libxslt netcdf pixman py-nose py-sip spindle xerces-c bzip2 dri2proto glpk libarchive llvm netgauge pkg-config py-numexpr py-six spot xzcairo dtcmp gmp libcerf llvm-lld netlib-blas pmgr_collective py-numpy py-sphinx sqlite yasmcallpath dyninst gmsh libcircle lmdb netlib-lapack postgresql py-pandas py-sympy stat zeromqcblas eigen gnuplot libdrm lmod netlib-scalapack ppl py-pbr py-tappy sundials zlibcbtf elfutils gnutls libdwarf lua nettle protobuf py-periodictable py-twisted swig zshcbtf-argonavis elpa gperf libedit lwgrp ninja py-astropy py-pexpect py-urwid szipcbtf-krell expat gperftools libelf lwm2 ompss py-basemap py-pil py-virtualenv tarcbtf-lanl extrae graphlib libevent matio ompt-openmp py-biopython py-pillow py-yapf taskcereal exuberant-ctags graphviz libffi mbedtls opari2 py-blessings py-pmw python taskdcfitsio fftw gsl libgcrypt memaxes openblas py-cffi py-pychecker qhull tau
§ Spack hasover2,100packagesnow.
LLNL-PRES-80606427github.com/spack/spack
§ Alltheversionscoexist!— Multipleversionsof
samepackageareok.
§ Packagesareinstalledtoautomaticallyfindcorrectdependencies.
§ Binariesworkregardlessofuser’senvironment.
§ Spack alsogeneratesmodulefiles.— Don’thave tousethem.
`spack find`showswhatisinstalled
$ spack find==> 103 installed packages.-- linux-redhat6-x86_64 / gcc@4.4.7 --------------------------------ImageMagick@6.8.9-10 glib@2.42.1 libtiff@4.0.3 pango@1.36.8 qt@4.8.6SAMRAI@3.9.1 graphlib@2.0.0 libtool@2.4.2 parmetis@4.0.3 qt@5.4.0adept-utils@1.0 gtkplus@2.24.25 libxcb@1.11 pixman@0.32.6 ravel@1.0.0atk@2.14.0 harfbuzz@0.9.37 libxml2@2.9.2 py-dateutil@2.4.0 readline@6.3boost@1.55.0 hdf5@1.8.13 llvm@3.0 py-ipython@2.3.1 scotch@6.0.3cairo@1.14.0 icu@54.1 metis@5.1.0 py-nose@1.3.4 starpu@1.1.4callpath@1.0.2 jpeg@9a mpich@3.0.4 py-numpy@1.9.1 stat@2.1.0dyninst@8.1.2 libdwarf@20130729 ncurses@5.9 py-pytz@2014.10 xz@5.2.0dyninst@8.1.2 libelf@0.8.13 ocr@2015-02-16 py-setuptools@11.3.1 zlib@1.2.8fontconfig@2.11.1 libffi@3.1 openssl@1.0.1h py-six@1.9.0freetype@2.5.3 libmng@2.0.2 otf@1.12.5salmon python@2.7.8gdk-pixbuf@2.31.2 libpng@1.6.16 otf2@1.4 qhull@1.0
-- linux-redhat6-x86_64 / gcc@4.8.2 --------------------------------adept-utils@1.0.1 boost@1.55.0 cmake@5.6-special libdwarf@20130729 mpich@3.0.4adept-utils@1.0.1 cmake@5.6 dyninst@8.1.2 libelf@0.8.13 openmpi@1.8.2
-- linux-redhat6-x86_64 / intel@14.0.2 -----------------------------hwloc@1.9 mpich@3.0.4 starpu@1.1.4
-- linux-redhat6-x86_64 / intel@15.0.0 -----------------------------adept-utils@1.0.1 boost@1.55.0 libdwarf@20130729 libelf@0.8.13 mpich@3.0.4
-- linux-redhat6-x86_64 / intel@15.0.1 -----------------------------adept-utils@1.0.1 callpath@1.0.2 libdwarf@20130729 mpich@3.0.4boost@1.55.0 hwloc@1.9 libelf@0.8.13 starpu@1.1.4
LLNL-PRES-80606428github.com/spack/spack
Userscanquerythefulldependencyconfigurationofinstalledpackages.
§ Architecture,compiler,versions,andvariantsmaydifferbetweenbuilds.
$ spack find callpath==> 2 installed packages.-- linux-x86_64 / clang@3.4 ———————— -- linux-x86_64 / gcc@4.9.2 -------------callpath@1.0.2 callpath@1.0.2
Expand dependencies with spack find -d
$ spack find -dl callpath==> 2 installed packages.-- linux-x86_64 / clang@3.4 ----------- -- linux-x86_64 / gcc@4.9.2 -----------xv2clz2 callpath@1.0.2 udltshs callpath@1.0.2ckjazss ^adept-utils@1.0.1 rfsu7fb ^adept-utils@1.0.13ws43m4 ^boost@1.59.0 ybet64y ^boost@1.55.0ft7znm6 ^mpich@3.1.4 aa4ar6i ^mpich@3.1.4qqnuet3 ^dyninst@8.2.1 tmnnge5 ^dyninst@8.2.13ws43m4 ^boost@1.59.0 ybet64y ^boost@1.55.0g65rdud ^libdwarf@20130729 g2mxrl2 ^libdwarf@20130729cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13g65rdud ^libdwarf@20130729 g2mxrl2 ^libdwarf@20130729cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13ft7znm6 ^mpich@3.1.4 aa4ar6i ^mpich@3.1.4
$ spack find -dl callpath==> 2 installed packages.-- linux-x86_64 / clang@3.4 ----------- -- linux-x86_64 / gcc@4.9.2 -----------xv2clz2 callpath@1.0.2 udltshs callpath@1.0.2ckjazss ^adept-utils@1.0.1 rfsu7fb ^adept-utils@1.0.13ws43m4 ^boost@1.59.0 ybet64y ^boost@1.55.0ft7znm6 ^mpich@3.1.4 aa4ar6i ^mpich@3.1.4qqnuet3 ^dyninst@8.2.1 tmnnge5 ^dyninst@8.2.13ws43m4 ^boost@1.59.0 ybet64y ^boost@1.55.0g65rdud ^libdwarf@20130729 g2mxrl2 ^libdwarf@20130729cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13g65rdud ^libdwarf@20130729 g2mxrl2 ^libdwarf@20130729cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13cj5p5fk ^libelf@0.8.13 ynpai3j ^libelf@0.8.13ft7znm6 ^mpich@3.1.4 aa4ar6i ^mpich@3.1.4
LLNL-PRES-80606429github.com/spack/spack
Spack managesinstalledcompilers
§ Compilersareautomaticallydetected— AutomaticdetectiondeterminedbyOS— Linux:PATH— Cray:`moduleavail`
§ Compilerscanbemanuallyadded— IncludingSpack-builtcompilers
compilers:- compiler:
modules: []operating_system: ubuntu14paths:cc: /usr/bin/gcc/4.9.3/gcccxx: /usr/bin/gcc/4.9.3/g++f77: /usr/bin/gcc/4.9.3/gfortranfc: /usr/bin/gcc/4.9.3/gfortran
spec: gcc@4.9.3- compiler:
modules: []operating_system: ubuntu14paths:cc: /usr/bin/clang/6.0/clangcxx: /usr/bin/clang/6.0/clang++f77: nullfc: null
spec: clang@6.0 - compiler:
...
$ spack compilers==> Available compilers-- gcc ----------------------------------gcc@4.2.1 gcc@4.9.3
-- clang --------------------------------clang@6.0
compilers.yaml
LLNL-PRES-80606430github.com/spack/spack
CoreSpack Concepts
LLNL-PRES-80606431github.com/spack/spack
§ Traditionalbinarypackagemanagers— RPM,yum,APT,yast,etc.— Designedtomanageasinglestack.— Installone versionofeachpackageinasingleprefix(/usr).— Seamlessupgradestoastable,welltested stack
§ Portsystems— BSDPorts,portage,Macports,Homebrew,Gentoo,etc.— Minimalsupportforbuildsparameterizedbycompilers,dependencyversions.
§ VirtualMachinesandLinuxContainers(Docker)— Containersallowuserstobuildenvironmentsfordifferentapplications.— Doesnotsolvethebuildproblem(someonehastobuildtheimage)— Performance,security,andupgradeissuespreventwidespreadHPCdeployment.
Mostexistingtoolsdonotsupportcombinatorialversioning
LLNL-PRES-80606432github.com/spack/spack
§ Eachuniquedependencygraphisauniqueconfiguration.
§ Eachconfigurationinstalledinauniquedirectory.— Configurationsofthesamepackagecancoexist.
§ Hash ofentiredirectedacyclicgraph(DAG)isappendedtoeachprefix.
§ Installedpackagesautomaticallyfinddependencies— Spack embedsRPATHsinbinaries.— NoneedtousemodulesorsetLD_LIBRARY_PATH— Thingsworkthewayyoubuiltthem
Spack handlescombinatorialsoftwarecomplexity.
spack/opt/linux-x86_64/
gcc-4.7.2/mpileaks-1.1-0f54bf34cadk/
intel-14.1/hdf5-1.8.15-lkf14aq3nqiz/
bgq/xl-12.1/
hdf5-1-8.16-fqb3a15abrwx/...
mpileaks
mpi
callpath dyninst
libdwarf
libelf
InstallationLayout
DependencyDAG
Hash
LLNL-PRES-80606433github.com/spack/spack
mpileaks
mpi
callpath dyninst
libdwarf
libelf
§ Spack ensuresoneconfigurationofeachlibraryperDAG— EnsuresABIconsistency.— UserdoesnotneedtoknowDAGstructure;onlythedependencynames.
§ Spack canensurethatbuildsusethesamecompiler,oryoucanmix— WorkingonensuringABIcompatibilitywhencompilersaremixed.
Spack Specscanconstrainversionsofdependencies
$ spack install mpileaks %intel@12.1 ^libelf@0.8.12
LLNL-PRES-80606434github.com/spack/spack
Spack handlesABI-incompatible,versionedinterfaceslikeMPI
$ spack install mpileaks ^mvapich@1.9 $ spack install mpileaks ^openmpi@1.4:
$ spack install mpileaks ^mpi@2
mpileaks
mpi
callpath dyninst
libdwarf
libelf
§ mpi isavirtualdependency
§ InstallthesamepackagebuiltwithtwodifferentMPIimplementations:
§ LetSpack chooseMPIimplementation,aslongasitprovidesMPI2interface:
LLNL-PRES-80606435github.com/spack/spack
Concretizationfillsinmissingconfigurationdetailswhentheuserisnotexplicit.
mpileaks ^callpath@1.0+debug ^libelf@0.8.11 User input: abstract spec with some constraints
Concrete spec is fully constrainedand can be passed to install.
mpileaks@2.3%gcc@4.7.3=linux-ppc64
mpich@3.0.4%gcc@4.7.3=linux-ppc64
callpath@1.0%gcc@a4.7.3+debug
=linux-ppc64
dyninst@8.1.2%gcc@4.7.3=linux-ppc64
libelf@0.8.11%gcc@4.7.3=linux-ppc64
libdwarf@20130729%gcc@4.7.3=linux-ppc64
Abstract, normalized specwith some dependencies.
mpileaks
mpi
callpath@1.0+debug
dyninst
libelf@0.8.11
libdwarf
Norm
alize
Concretize Store
spec:- mpileaks:
arch: linux-x86_64compiler:name: gccversion: 4.9.2
dependencies:adept-utils: kszrtkpbzac3ss2ixcjkcorlaybnptp4callpath: bah5f4h4d2n47mgycej2mtrnrivvxy77mpich: aa4ar6ifj23yijqmdabeakpejcli72t3
hash: 33hjjhxi7p6gyzn5ptgyes7sghyprujhvariants: {} version: '1.0'
- adept-utils:arch: linux-x86_64compiler:name: gccversion: 4.9.2
dependencies:boost: teesjv7ehpe5ksspjim5dk43a7qnowlqmpich: aa4ar6ifj23yijqmdabeakpejcli72t3
hash: kszrtkpbzac3ss2ixcjkcorlaybnptp4variants: {} version: 1.0.1
- boost:arch: linux-x86_64compiler:name: gccversion: 4.9.2
dependencies: {} hash: teesjv7ehpe5ksspjim5dk43a7qnowlqvariants: {} version: 1.59.0
...
spec.yaml
Detailed provenance is storedwith the installed package
LLNL-PRES-80606436github.com/spack/spack
Use`spack spec`toseetheresultsofconcretization
$ spack spec mpileaksInput spec------------------------------
mpileaks
Normalized------------------------------
mpileaks^adept-utils
^boost@1.42:^mpi
^callpath^dyninst
^libdwarf^libelf
Concretized------------------------------
mpileaks@1.0%gcc@5.3.0 arch=darwin-elcapitan-x86_64^adept-utils@1.0.1%gcc@5.3.0 arch=darwin-elcapitan-x86_64
^boost@1.61.0%gcc@5.3.0+atomic+chrono+date_time~debug+filesystem~graph~icu_support+iostreams+locale+log+math~mpi+multithreaded+program_options~python+random +regex+serialization+shared+signals+singlethreaded+system+test+thread+timer+wave arch=darwin-elcapitan-x86_64
^bzip2@1.0.6%gcc@5.3.0 arch=darwin-elcapitan-x86_64^zlib@1.2.8%gcc@5.3.0 arch=darwin-elcapitan-x86_64
^openmpi@2.0.0%gcc@5.3.0~mxm~pmi~psm~psm2~slurm~sqlite3~thread_multiple~tm~verbs+vt arch=darwin-elcapitan-x86_64^hwloc@1.11.3%gcc@5.3.0 arch=darwin-elcapitan-x86_64
^libpciaccess@0.13.4%gcc@5.3.0 arch=darwin-elcapitan-x86_64^libtool@2.4.6%gcc@5.3.0 arch=darwin-elcapitan-x86_64
^m4@1.4.17%gcc@5.3.0+sigsegv arch=darwin-elcapitan-x86_64^libsigsegv@2.10%gcc@5.3.0 arch=darwin-elcapitan-x86_64
^callpath@1.0.2%gcc@5.3.0 arch=darwin-elcapitan-x86_64^dyninst@9.2.0%gcc@5.3.0~stat_dysect arch=darwin-elcapitan-x86_64
^libdwarf@20160507%gcc@5.3.0 arch=darwin-elcapitan-x86_64^libelf@0.8.13%gcc@5.3.0 arch=darwin-elcapitan-x86_64
LLNL-PRES-80606437github.com/spack/spack
§ Spack installspackagesinownprefix
§ Somepackagesneedtobeinstalledwithindirectorystructureofotherpackages— i.e.,Pythonmodulesinstalledin$prefix/lib/python-<version>/site-packages— Spack supportsthisviaextensions
ExtensionsandPythonSupport
class PyNumpy(Package):"""NumPy is the fundamental package for scientific computing with Python."""
homepage = "https://numpy.org"url = "https://pypi.python.org/packages/source/n/numpy/numpy-1.9.1.tar.gz"version('1.9.1', ' 78842b73560ec378142665e712ae4ad9’)
extends('python')
def install(self, spec, prefix):setup_py("install“, "--prefix={0}".format(prefix))
LLNL-PRES-80606438github.com/spack/spack
Spack extensions
spack/opt/linux-redhat6-x86_64/
gcc-4.7.2/python-2.7.12-/
python/py-numpy
py-numpy-1.10.4-oaxix36/py-numpy/
...
§ SymboliclinktoSpack installlocation
§ Automaticallyactivateforcorrectversionofdependency— ProvenanceinformationfromDAG— Activatealldependenciesthatareextensionsaswell
spack/opt/linux-redhat6-x86_64/
gcc-4.7.2/python-2.7.12-6y6vvaw/
python/py-numpy-1.10.4-oaxix36/
py-numpy/...
$ spack activate py-numpy @1.10.4
§ Examplesofextensionpackages:— pythonlibrariesareagoodexample— R,Lua,perl— Needtomaintaincombinatorialversioning
LLNL-PRES-80606439github.com/spack/spack
Extensionsmustbeactivatedintoextendee
$ spack extensions python==> python@2.7.12% gcc@4.9.3 ~tk~ucs4 arch=linux-redhat7-x86_64-i25k4oi==> 118 extensions:…py-autopep8 py-docutils py-mako py-numpy py-py2neo …
==> 3 installed:-- linux-redhat7-x86_64 / gcc@4.9.3 -----------------------------py-nose@1.3.7 py-numpy@1.11.0 py-setuptools@20.7.0
==> None activated.
$ spack load python $ python Python 2.7.12 (default, Aug 26 2016, 15:12:42)[GCC 4.9.3] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> import numpyTraceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named numpy>>>
$ spack activate py-numpy==> Activated extension py-numpy@1.11.0% gcc@4.9.3 +blas+lapackarch=linux-redhat7-x86_64-77im5ny for python@2.7.12 ~tk~ucs4 % gcc@4.9.3$ spack extensions python==> python@2.7.12% gcc@4.9.3 ~tk~ucs4 arch=linux-redhat7-x86_64-i25k4oi…==> 1 currently activated:-- linux-redhat7-x86_64 / gcc@4.9.3 -----------------------------
$ python Python 2.7.12 (default, Aug 26 2016, 15:12:42)[GCC 4.9.3] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> import numpy>>> x = numpy.zeros(10)>>>
§ Pythonunawareofnumpy installation
▪ activate symlinks entirenumpy prefixintoPythoninstallation▪ Canalternativelyloadextension
$ spack load python$ spack load py-numpy
LLNL-PRES-80606440github.com/spack/spack
mpileaks@2.3gcc@4.7.3
arch=linux-redhat6-ppc64
callpath@1.0gcc@4.7.3
arch=linux-redhat6-ppc64+debug
openmpi@2.0.0gcc@4.7.3
arch=linux-redhat6-ppc64
dyninst@8.1.2gcc@4.7.3
arch=linux-redhat6-ppc64
hwloc@1.11.3gcc@4.7.3
arch=linux-redhat6-ppc64
libpciaccess@0.13.4gcc@4.7.3
arch=linux-redhat6-ppc64
libtool@2.4.6gcc@4.7.3
arch=linux-redhat6-ppc64
m4@1.4.17gcc@4.7.3
arch=linux-redhat6-ppc64
libsigsegv@2.10gcc@4.7.3
arch=linux-redhat6-ppc64
libelf@0.8.11gcc@4.7.3
arch=linux-redhat6-ppc64
libdwarf@20130729gcc@4.7.3
arch=linux-redhat6-ppc64
Buildingagainstexternallyinstalledsoftware
mpileaks@2.3gcc@4.7.3
arch=linux-redhat6-ppc64
callpath@1.0gcc@4.7.3
arch=linux-redhat6-ppc64+debug
openmpi@2.0.0gcc@4.7.3
arch=linux-redhat6-ppc64
dyninst@8.1.2gcc@4.7.3
arch=linux-redhat6-ppc64
libelf@0.8.11gcc@4.7.3
arch=linux-redhat6-ppc64
libdwarf@20130729gcc@4.7.3
arch=linux-redhat6-ppc64
/path/to/external/gcc/openmpi-2.0.0
packages:mpi:buildable: False
openmpi:buildable: Falsepaths:openmpi@2.0.0 %gcc@4.7.3 arch=linux-redhat6-ppc64:/path/to/external/gcc/openmpi-2.0.0
openmpi@1.10.3 %gcc@4.7.3 arch=linux-redhat6-ppc64:/path/to/external/gcc/openmpi-1.10.3
openmpi@2.0.0 %intel@16.0.0 arch=linux-redhat6-ppc64:/path/to/external/intel/openmpi-2.0.0
openmpi@1.10.3 %intel@16.0.0 arch=linux-redhat6-ppc64:/path/to/external/intel/openmpi-1.10.3
...
packages.yaml
A user registers external packages with Spack.
If a node in the DAG matches an registered external package, Spack prunes the DAG at that node and replaces the node with a reference to the external package.
mpileaks ^callpath@1.0+debug ^openmpi ^libelf@0.8.11
LLNL-PRES-80606441github.com/spack/spack
$ spack repo create /path/to/my_repo$ spack repo add my_repo$ spack repo list==> 2 package repositories.my_repo /path/to/my_repobuiltin spack/var/spack/repos/builtin
§ Somepackagescannotbereleasedpublicly
§ Someusershaveusecasesthatrequirebizarre custombuilds
§ PackagingissuesshouldnotpreventusersfromupdatingSpack— Solution:separaterepositories— Arepositoryissimplyadirectoryofpackagefiles
§ Spack supportsexternalrepositoriesthatcanbelayeredontopofthebuilt-inpackages
§ Custompackagescandependonbuilt-inpackages(orpackagesinotherrepositories)
Spack packagerepositories
my_repo var/spack/repos/builtin
Proprietary packages
Pathological build cases
“Standard” packages
LLNL-PRES-80606442github.com/spack/spack
§ UserCache— Userscreatea
“mirror”oftararchivesforpackages
— Removeinternetdependence
Fetchingsourcecodeforspack
Concretize
spack install mpileaks
Recursively install dependencies
Fetch package source
Build software
Load package from repository
User Cache
Spack Cache
The Internet§ Spack Cache
— Spack automaticallycachestararchivesforpreviouslyinstalledsoftware
§ TheInternet— Spack packagescanfindsourcefilesonline
LLNL-PRES-80606443github.com/spack/spack
Addingcustomcompilerflags
$ spack install hdf5 cflags='-O3 –g –fast –fpack-struct'
§ ThisinstallsHDF5withthespecifiedflags— FlagsareinjectedviaSpack’s compilerwrappers(discussedlater).
§ Flagsarepropagatedtodependenciesautomatically— FlagsareincludedintheDAGhash— Eachbuildwithdifferentflagsisconsideredadifferentversion
§ Thisprovidesaneasyharnessfordoingparameterstudiesfortuningcodes— Previouslyworkingwithlargecodeswasverytedious.
§ Supportscflags,cxxflags,fflags,cppflags,ldflags,andldlibs— AddedfromCLIorconfig file
LLNL-PRES-80606444github.com/spack/spack
Hands-onTime:Configuration
Follow script at http://spack.rtfd.ioUnder “Tutorial: Spack 101"
LLNL-PRES-80606445github.com/spack/spack
MakingyourownSpack Packages
LLNL-PRES-80606446github.com/spack/spack
CreatingyourownSpackPackages
from spack import *
class Zlib(Package):"""A free, general-purpose, legally unencumbered lossless
data-compression library."""
homepage = "http://zlib.net"url = "http://zlib.net/zlib-1.2.8.tar.gz"
version('1.2.8', '44d667c142d7cda120332623eab69f40')
depends_on(‘cmake’, type=‘build’)
def install(self, spec, prefix):configure('--prefix={0}'.format(prefix))
make()make('install')
▪ Packageisarecipeforbuilding
▪ EachpackageisaPythonclass— Downloadinformation— Versions/Checksums— Buildoptions— Dependencies— Buildinstructions
▪ Packagecollectionsarerepos— Spackhasa“builtin”repoin
var/spack/repos/builtin
$REPO/packages/zlib/package.py
LLNL-PRES-80606447github.com/spack/spack
Spack packagesaretemplates forbuilds
def install(self, spec, prefix):config_opts=[‘--prefix=‘+prefix]
if '~shared' in self.spec:config_opts.append('--disable-shared')
else:config_opts.append('--enable-shared')
configure(config_opts)make()make('install')
▪ Eachpackagehasoneclass— zlib forIntelcompilerandzlib forGCCcompilerare
builtwiththesamerecipe.
▪ Canaddconditionallogicusingspecsyntax— Thinkofpackageastranslating aconcreteDAGto
buildinstructions.— Dependenciesarealreadybuilt— Nosearchingortesting;justdowhattheDAGsays
▪ Compilerwrappershandlemanydetailsautomatically.— Spack feedscompilerwrappersto(cc,c++,f90,…)
toautoconf,cmake,gmake,…— Wrappersselectcompilers,dependencies,and
optionsunderthehood.
package.py
LLNL-PRES-80606448github.com/spack/spack
Spackbuildseachpackageinitsowncompilationenvironment
SpackProcess
Setupenvironment
CC = spack/env/spack-cc SPACK_CC = /opt/ic-15.1/bin/iccCXX = spack/env/spack-c++ SPACK_CXX = /opt/ic-15.1/bin/icpcF77 = spack/env/spack-f77 SPACK_F77 = /opt/ic-15.1/bin/ifortFC = spack/env/spack-f90 SPACK_FC = /opt/ic-15.1/bin/ifort
PKG_CONFIG_PATH = ... PATH = spack/env:$PATHCMAKE_PREFIX_PATH = ...LIBRARY_PATH = ...
do_install()
Install dep1 Install dep2 Install package…
BuildProcess
Fork
install() configure make makeinstall
-I /dep1-prefix/include-L /dep1-prefix/lib-Wl,-rpath=/dep1-prefix/lib
Compilerwrappers(spack-cc, spack-c++, spack-f77, spack-f90)
icc icpc ifort
▪ Forkedbuildprocessisolatesenvironmentforeachbuild.Usescompilerwrappersto:— Addinclude,lib,andRPATHflags— Ensurethatdependenciesarefoundautomatically— LoadCraymodules(userightcompiler/systemdeps)
LLNL-PRES-80606449github.com/spack/spack
WritingPackages- VersionsandURLs
class Mvapich2(Package):homepage = "http://mvapich.cse.ohio-state.edu/"url = "http://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich2-2.2rc2.tar.gz"
version('2.2rc2', 'f9082ffc3b853ad1b908cf7f169aa878')version('2.2b', '5651e8b7a72d7c77ca68da48f3a5d108')version('2.2a', 'b8ceb4fc5f5a97add9b3ff1b9cbe39d2')version('2.1', '0095ceecb19bbb7fb262131cb9c2cdd6')
▪ PackagedownloadsarehashedwithMD5bydefault— AlsosupportsSHA-1,SHA-256,SHA-512— We’llbeswitchingtoSHA-256orhighersoon.
▪ DownloadURLscanbeautomaticallyextrapolatedfromURL.— ExtraoptionscanbeprovidedifSpack can’textrapolateURLs
▪ OptionscanalsobeprovidedtofetchfromVCSrepositories
$REPO/packages/mvapich/package.py
LLNL-PRES-80606450github.com/spack/spack
WritingPackages– VariantsandDependencies
class Petsc(Package):variant('mpi', default=True, description='Activates MPI support')variant('complex', default=False, description='Build with complex numbers')variant('hdf5', default=True, description='Activates support for HDF5 (only parallel)')
depends_on('blas')depends_on('python@2.6:2.7')
depends_on('mpi', when='+mpi')
▪ Variantsarenamed,havedefaultvaluesandhelptext
▪ Otherpackagescanbedependencies— whenclauseprovidesconditionaldependencies— Candependonspecificversionsorothervariants
$REPO/packages/petsc/package.py
LLNL-PRES-80606451github.com/spack/spack
WritingPackages– BuildRecipes
def install(self, spec, prefix):with working_dir("build", create=True):
cmake("..", *std_cmake_args)make()make("install")
@when('@:8.1')def install(self, spec, prefix):
configure("--prefix=" + prefix)make()make("install")
▪ Functionswrapcommonops— cmake,configure,patch,make,…— Executable andwhichfornewwrappers.
▪ Commandsexecutedincleanenvironment
▪ FullPythonfunctionality— Patchupsourcecode— Makefilesanddirectories— Calculateflags— …
$REPO/packages/dyninst/package.py
LLNL-PRES-80606452github.com/spack/spack
Createnewpackageswithspack create
class Zlib(Package):# FIXME: Add a proper url for your package's homepage here. homepage = "http://www.example.com"url = "http://zlib.net/zlib-1.2.8.tar.gz"version('1.2.8', '44d667c142d7cda120332623eab69f40')
def install(self, spec, prefix):# FIXME: Modify the cmake line to suit your build system here.
▪ spack create<url>willcreateaskeletonforapackage— Spack reasonsaboutURL,hash,version,buildrecipe.— GeneratesboilerplateforCmake,Makefile,autotools,Python,R,Waf,Perl
— Notintendedtocompletelywritethepackage,butgetsyou80%ofthewaythere.
▪ spack edit<package>forsubsequentchanges
$ spack create http://zlib.net/zlib-1.2.8.tar.gz
$REPO/packages/zlib/package.py
LLNL-PRES-80606453github.com/spack/spack
Hands-onTime:CreatingPackages
Follow script at http://spack.rtfd.ioUnder “Tutorial: Spack 101"
LLNL-PRES-80606454github.com/spack/spack
Hands-onTime:AdvancedPackaging
Follow script at http://spack.rtfd.ioUnder “Tutorial: Spack 101"
LLNL-PRES-80606455github.com/spack/spack
Hands-onTime:ModuleFiles
Follow script at http://spack.rtfd.ioUnder “Tutorial: Spack 101"
LLNL-PRES-80606456github.com/spack/spack
§ ContributingpackagestoSpack issimple— Makepackagesonyourownsystem— SendapullrequesttoSpack toletothersusethem— GitHubguidetopullrequests:
https://help.github.com/articles/creating-a-pull-request/— SeecontributorguideintheSpack repository— Spack islicensedunderLGPLv2.1
§ Wewantmorethanjustpackages— Newfeatures— Newdocumentation— Anycontributioncanhelpthecommunity!
§ Spack hasahelpfulonlinecommunity— TypicallyhappytorespondtoGitHubissues— ActivemailinglistonGoogleGroups
JointheSpack community!
@spackpm
github.com/spack/spack
We hope to make distributing & using HPC software easy!
Recommended