Sciama, the Software side

1

Agenda• Memory Models• Running Jobs, Selecting Packages and

Compiling• OpenMP• MPI• OpenMP / MPI mixed mode• Sciama Information

Memory Models

3

Basic Building Block

4

These are the building blocks of Sciama and similar commodity HPC’s.

Distributed Memory Model

5

Largest (sensible) job is 24Gbytes in this distributed

memory model

6

Shared Memory Model

Shared Memory Models Can Support very large processes

Shared Memory Model Supports OpenMP and MPI (and Hybrid)

Running Jobs, Selecting  Packages and Compiling

10

ICG‐HPC Stack

11

RedHat Clone

(CERN)

Two main Enterprise Linux players:-

RedHat & SUSE Licensed

Stability is all important.No need for latest andgreatest (Fedora 16)

Running Jobs

• Run an application already loaded in the  environment.

– Use “module avail”

to check what is installed

• Get a new application installed so you can run it.– Contact sciama.support@port.ac.uk

to get the software installed

• Compile your own application from source code.– You will need to select the compiler(s) and libraries using the “module”

command described. Request dependencies to be installed but not always 

possible.

Running Jobs• You should never run any large processes on a login server. Login servers are 

shared by others so any intense processing will slow down the responsiveness 

others experience.

• All major applications / compilations should be done on a compute node

• There are two ways to do this both using the “qsub”

command to submit a job to a 

queue :‐

– Create an interactive shell on a compute node:‐qsub

–IX

# I=Interactive

X=allow X server windowsThis will give you a command prompt in the current

shell.

– Run a batch job on a compute node(s) :‐qsub

<job‐script>This will start / queue a job according to the directives specified in the job  script.

Installed S/W

14

Compiling

15

Sciama

Traffic

Compile Options

“gdb”

GNU Debugger[burtong@login1 ~]$ gdbGNU gdb

(GDB) Red Hat Enterprise Linux (7.0.1‐23.el5_5.1)Copyright (C) 2009 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later 

<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.  Type "show 

copying"

and "show warranty" for details.This GDB was configured as "x86_64‐redhat‐linux‐gnu".For bug reporting instructions, please see:<http://www.gnu.org/software/gdb/bugs/>.(gdb) 

Intel Debugging

Vtune

Qsub –I -X Very good help

Inspector

OpenMP ( Open Multi Processing) 

22

Largest (sensible) job is 24Gbytes  in this distributed memory model 

23

Multi Threading 

• Multiple cores allow for multi treaded applications.• OpenMP

is an enabler

24

Shared memory (cooperating threads)• OpenMP• POSIX Threads• Cilk• Threaded Building Blocks• etc, etc

OpenMP

Identical instances ofparallel region spawnedAs threads

OpenMP

–

“C”

or Fortran

Compiler looks for directives and creates a multi- threaded executable.

Use –fopenmp as a compiler option

Possibly several parallel sections in one program.

Running openmp.exe

MPI ( Message Passing Interface )

28

Inter node memory sharing not  (usually ) possible 

• Gives rise to “distributed memory”

Model• Need the likes of OpenMPI

(Message Passing Interface)

MPI allows parallel programming in  distributed memory model

• MPI enables parallel computation• Message Buffers are used to pass data between processes• Standard tcp-ip network used

Buffers also usedWithin same node.

MPI Principles (1)

MPI Principles (2)

MPI Principles (3)

First of all you need the a program …

Can be “C” or

Fortran .

Select Compiler

The run script

Run the Job

“mpitest”

Output

OpenMP

/ MPI mixed mode and  memory models

39

Hybrid OpenMP

and MP programming  possible

MPI – OpenMP Mix

Free Cores

Sciama

Information

43

Live Sciama

Information

Sciama

Docs and A/c Request