Embed Size (px)
Citation preview
1 The Compute Courier replaces the Cartesius newsletter with all the news related to both theCartesius and Lisa systems. Should you have a better name for this newsletter, please don't hesitateto send your suggestion to [email protected]. All bold URLs in the PDF version of this newsletterare clickable.
Table of Contentsa The GPGPU system on Cartesius is in productiona Larger projects possible with Cartesius upgradesa Lisa upgradea New remote visualization cluster ELvisa The SURFsara app!a New and updated software on Lisaa New and updated software on Cartesiusa Known issues on Cartesiusa We're hiring!a PRACE training eventsa PRACE call for preparatory access
The GPGPU system on Cartesius is in production
The production system of GPGPUs is available with 66 nodes, each with 2 Nvidia K40m GPGPUs.You can use your normal Cartesius budget both for the CPU nodes and the GPU nodes. If youwould like to use the GPU nodes, please contact [email protected].
The total peak performance of Cartesius is now 470 Tflop/s.
The nodes of the production GPU system are accounted as 48 core hours per wall clock hour andare therefore 2x more expensive than thin nodes of the current system.
These GPGPUs can also be used both for large-scale computations and large-scale visualisations incombination with the Collaboratorium. Also see the press release (in dutch only).
Larger projects possible with Cartesius upgrades
Cartesius will be upgraded significantly in late 2014, with a large expansion with the newest Intelhardware.
This translates directly into the possibility to run larger research projects on the system. The largestcomputing projects can now ask for 25 million core hours.
Lisa upgrade
We are happy to announce the replacement of the 384 old L5520 nodes with 312 new nodes in theLisa system. These nodes are financed by SURF, UvA and VU.
This installation will be finished on 2014-05-29.
The new nodes, type E5-2650 v2 @ 2.60GHz contain each 16 cores and 64 Gbyte memory and will deliver roughly a factor 3 more compute power than the old 384 nodes.
IMPORTANT Since all 8-core nodes are removed after the upgrade, the 'cores8' requirement can not be used any more. Use 'cores12' or 'cores16' or leave it out.
Due to a configuration change of the InfiniBand network, the maximum number of nodes in one jobis now 32.
New remote visualization cluster ELvis
The current Remote Visualization Service (or RVS for short) has been replaced by the new ELvis cluster. It provides similar, but more powerful, (remote) visualization capabilities. The new ELvis cluster consists of 9 render nodes and a separate login node, each with the following specs:
• 2x Intel Xeon CPU E5-2620 v2 @ 2.10GHz (6 cores per CPU)
• 64 GByte RAM
• 2x NVidia Geforce GTX 780Ti card, with 3 GByte video RAM
• OS: Debian Linux 7.4 (x86_64), kernel 3.2.0
• Roughly 800 GByte local scratch space per node
Compute Courier | number 1 | June 2014 | 2
On the ELvis cluster the Cartesius projects and scratch file systems are directly accessible, usage of data from other SURFsara HPC systems might involve (manual) data copying. Data in the Central Archive is also accessible.
Two examples of recent usage of the RVS cluster are:
Numerical simulation of stratocumulus cloudbreakup, created by J. van der Dussen (DelftUniversity of Technology) using ParaView.
http://www.youtube.com/watch?v=vZpyVWye3S4
A visualization of a Rayleigh-Bénard convectionby E.P. van der Poel & R. Ostilla Monico,University of Twente. Simulation was run onPRACE Tier-0 resources and created in LLNL'sVisIt visualization application.
This movie made it into the VisIt gallery.
The SURFsara app!
SURFsara has developed a free app for mobile devices,available for Android and iOS. Information about oursystems is always at hand with this app, even while on thego.
The app shows the availability and upcoming maintenance forall available SURFsara systems, including Cartesius and Lisa.Users can easily access their accounting information andcomputational budget (if applicable) by using the app. Forexample: to view the amount of hours that has been assigned totheir account or to check the remaining budget. Through onesimple screen containing all relevant contact information, usersof the app can directly contact SURFsara and the helpdesks ofthe various systems. To stay informed of the latestdevelopments our most recent news items can be viewed. Theseitems are linked to the SURFsara website. The possibilities ofthe app will be updated in the future to offer users theopportunity to use new features.
New and updated software on Lisa
Many software packages and libraries are pre-installed and ready for you to use on Lisa, just have alook at our software documentation. The following packages were newly installed in the last fewmonths:
julia – The Julia language.
Compute Courier | number 1 | June 2014 | 3
FSL – a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data
libSBML – library to process Systems Biology Markup Language files.
NBO – quantum chemical analysis tool for Natural Bond Orbitals.
sundials – SUite of Nonlinear and DIfferential/ALgebraic equation Solvers.
The following packages were upgraded in the last few months:
ADF 2012.1 & 2013.1 – Modelling of structure and spectroscopy of molecules.
GCTA 1.24.2 - Genome-wide Complex Trait Analysis
Molden 5.1 – a visualization program of molecular and electronic structures
QIIME 1.8.0 – Quantitative Insights Into Microbial Ecology
R 3.0.2 & 3.0.3 – Statistical computing.
Solar 7.2.5 - Sequential Oligogenic Linkage Analysis Routines
Stopos 0.93 – Manage your serial jobs.
New and updated software on Cartesius
Many software packages and libraries are pre-installed and ready for you to use on Cartesius, justhave a look at our software documentation. The following packages were newly installed since thelast Cartesius newsletter:
CUDA 5.5, 6.0 & 6.0.37 – programming environment for NVIDIA's graphic cards. This includes the OpenCL programming environment.
Go – Go compiler and runtime.
Octopus 4.1.2 – ab initio virtual experimentation. There is a non-default installation that includes the utilities, which can be loaded with 'module load octopus/4.1.2-utils'.
Mathematica – technical computing software. To use Mathematica on Cartesius, you need your own license.
PGI compilers – the PGI compilers provide support for NVIDIA GPUs and can also be used on the CPU nodes.
strace-plus – strace-plus is an improved version of strace that collects stack tracesassociated with each system call. This can be very handy if your code stops without anindication of the location in the code. E.g. to see the stack strace at the moment that yourapplication exits (not necessarily with an error):
module load straceplus strace k eexit_group ostrace.out my_application pretty_print_strace_out.py tree strace.out
The following packages were upgraded since the last Cartesius newsletter:
ADF 2012.1 & 2013.1 – Modelling of structure and spectroscopy of molecules.
automake 1.13.1, autoconf 2.69, libtool 2.4.2 – recent versions of the GNU autotools.
BSPonMPI 0.3 – Bulk Synchronous Parallel library on MPI
COMSOL 4.4 – Finite element analysis and simulation.
Gamess UK Nov '13 – General Atomic and Molecular Electronic Structure System.
Compute Courier | number 1 | June 2014 | 4
Gamess US 2013-05 r2 & r3 – General Atomic and Molecular Electronic Structure System.
Gperftools 2.1 – tool for CPU and memory profiling with call-graph view.
GROMACS 4.6.5 – A versatile package to perform Molecular Dynamics. This also includesGPU acceleration.
HDF5 1.8.12 parallel – general purpose library to store scientific data using parallel I/O.
MATLAB r2013b & MCR 8.2 – Matrix computations
Mumps 4.10.0 – Parallel solver library.
NetCDF C 4.3.1.1 parallel – general purpose library to store scientific data using parallelI/O. Note that this version only includes the C API.
NetCDF Fortran 4.2 parallel – general purpose library to store scientific data using parallelI/O. Note that this library only includes the Fortran API.
PetSc 3.4.3 – Partial Differential Equation Solvers library.
Scalasca 2.0 – Scalable performance analysis of large scale applications
SIONlib 1.4p3 – Scalable I/O library for parallel access to task-local files.
SLEPc 3.3p3 & 3.4.3 – Eigenvalue/vector Solver library.
Stopos 0.93 – Manage your serial jobs.
Totalview 8.12.0-1 – parallel debugger.
VASP 5.3.5 – molecular dynamics.
If you have any questions or need other software on Cartesius, you can send your question orrequest for installation or upgrade to [email protected].
Known issues on Cartesius
This section lists the known issues on Cartesius. It can take some time before these issues are fixed,in the meanwhile we try to provide workarounds.
Known issues on Cartesius:
• the SLURM option export=NONE can be used to ensure that environment variablesettings from your interactive session do not impact your submitted batch jobs. However,this also causes some SLURM environment variables not to be set. Most notably, the c orcpuspertask flag doesn't function anymore and your OpenMP application willonly use 1 thread. A workaround is to set the environment variable OMP_NUM_THREADSexplicitly in your job, e.g.:
#SBATCH cpuspertask=4#SBATCH export=NONEexport OMP_NUM_THREADS=4srun my_application
• the squeue u[user] command shows incorrect output for pending jobs. The 'Nodes' column shows the number of requested cores, not the number of requested nodes. The correct output can be reproduced with the command:
squeue | egrep “JOBID|[user]”
Compute Courier | number 1 | June 2014 | 5
The following problems have been fixed since the last newsletter:
• The Fortran statement INQUIRE(..., opened=flag) fails when using the Intel Fortrancompiler and more than 2.000 files are open. The error message is:
forrtl: severe (2): not implemented
Intel Fortran release 14.0.2 fixes this issue.
• Some jobs failed due to a bug in the Lustre filesystem. This could be seen from the stateNODE_FAIL when running the command 'sacct'. A new release of the Lustre filesystem hasbeen installed.
• the MPI_Init routine took very long for large task counts (⪎ 1,000) when using the IntelMPI library, increasing to over one hour when using 5,000 tasks. Please set the environmentvariable PMI_TIME to 1 in your batch job to lower this to a few minutes.
We're hiring!
As national High Performance Computing and e-Science Support Center SURFsara supportsscientists with state-of-the-art integrated services, expertise and infrastructure: High PerformanceComputing and Networking, data services, visualization and e-Science & Cloud services. TheCartesius team is looking for consultants to solve daily issues and optimize applications onCartesius, amongst others.
Jobs at SURFsara
• (Senior) Adviseur / Wetenschappelijk programmeur• Technical consultant big data• Hadoop operations engineer• (Senior) Unix system administrator (cloud)• Systeembeheerder kantoorautomatisering
Reaction formAre you looking for an internship or job in the field of High Performance Computing and Networking? Fill out the information request form or send your CV to [email protected].
PRACE training events
PRACE organizes trainings on many topics related to HPC throughout Europe. An overview of allcoming training events can be found on the website http://www.training.prace-ri.eu/. This websitealso contains presentations, videos and tutorials from previous events.
One of the coming events is the workshop on the new Cray XC30 and parallel I/O at HLRS in Stuttgart from 23 to 26 September.
The first day is targeted to HERMIT users who will continue their work on HORNET. Specialistsfrom Cray will talk about the hardware and software enhancements between the XE6 and XC30 inorder to support your migration towards this new machine.
The second day is dedicated to user parallel IO at scale.
The third and fourth day covers a 2 day introduction workshop about porting and optimizing your application for the Cray XC30. The topics of this workshop include the Cray Programming Environment, scientific libraries and profiling tools.
Compute Courier | number 1 | June 2014 | 6
PRACE call for preparatory access
PRACE, the Partnership for Advanced Computing in Europe, is a Research Infrastructure thatenables researchers from across Europe to apply for time on the PRACE resources via a peer reviewprocess. This call marks the opening of continuous preparatory access allowing researchers to applyfor code scalability testing, support for code development and optimization. Preparatory access callsare rolling calls, researchers can apply for resources all year. There are no closing dates.
The available HPC (High Performance Computing) systems available to researchers throughPRACE are:
• IBM BlueGene/Q – JUQUEEN – hosted by the Gauss Centre member site in Jülich, Germany
• BULL Bullx cluster – CURIE – hosted by CEA (funded by GENCI) in Bruyères-Le-Châtel, France
• IBM BlueGene/Q – FERMI – hosted by CINECA• IBM System X iDataPlex – MareNostrum – hosted by BSC in Barcelona, Spain• Cray XE6 – HERMIT – hosted by the Gauss Centre member site in Stuttgart, Germany• IBM System X iDataPlex – SuperMUC – hosted by the Gauss Centre member site in
Munich, Germany
There are three types of preparatory access:
A. Code scalability testing to obtain scalability data which can be used as supporting informationwhen responding to future PRACE project calls. This route provides an opportunity to ensure thescalability of the codes within the set of parameters to be used for PRACE project calls, anddocument this scalability. Assessment of applications is undertaken using a light-weight applicationprocedure with application evaluated at least every 2 months.
B. Code development and optimization by the applicant using their own personnel resources (i.e.without PRACE support). Applicants will need to describe the planning for development in detailtogether with the expert resources that are available to execute the project. Applications will beassessed at least every 2 months.
C. Code development with support from experts from PRACE. Assessment of the applicationsreceived will be carried out at least every two months.
All proposals must be submitted via the PRACE website at www.prace-ri.eu/hpc-access. Allproposals will undergo PRACE technical and scientific assessment. See also a related document onhttp://www.prace-ri.eu/IMG/pdf/prace_preparatory_access_call.pdf
Compute Courier | number 1 | June 2014 | 7
![RVS 3/M - IBS1].pdf · RVS 3/M RVS 25/CT RVS 40/CT RVS 21/SG RVS 60/CT 2 RVS liquid ring vacuum pumps are a single ... RVS 16 / SG - 09 GRANDEZZA SIZE 3÷40 VERSIONE VERSION](https://img.pdfslide.us/doc/110x75/5a794fb87f8b9a4a518cfeb3/rvs-3m-1pdfrvs-3m-rvs-25ct-rvs-40ct-rvs-21sg-rvs-60ct-2-rvs-liquid-ring.jpg)
