Publishing applications on the web via the Easa Publishing applications on the web via the Easa Portal and integrating the Sun Portal and integrating the Sun

Grid EngineGrid Engine By Michael Griffiths

&Deniz Savas

CiCS Dept.

Sheffield University

M.Griffiths@sheffield.ac.uk D.Savas@sheffield.ac.uk

http://www.sheffield.ac.uk/wrgridSept 2007

Sheffield is in South Yorkshire, England

Sheffield University- Sheffield University- factsfacts

Established in 182870 academic departments in 7 faculties

Number of Undergraduate Students: 25,500Number of Post Graduate/Research Students: 5,600Number of International Students : 3,100

‘‘icebergiceberg’ the HPC Cluster at the Computer ’ the HPC Cluster at the Computer CentreCentre

• AMD Opteron based, supplied by Sun Microsystems.

• Processors: 320 ( 160 of these are designated to the Physics Dept. for the PP project )

• Performance: 300GFLOPs • Main Memory: 800GB • User filestore: 9TB • Temporary disk space: 10TB • Physical size: 8 racks • Power usage: 50KW

‘‘icebergiceberg’ cluster hardware components’ cluster hardware components160 general-purpose-use cpu’s; 80 of these are in dual-core configuration with 2 GBytes of memory each. ( V20 Model ) (i.e 40 boxes with 2 cpus + 4 GBytes )

80 are in quad-core configurations with 4 GBytes memory each. ( V40 Model ) ( i.e 20 boxes with 4 cpus + 16 GBytes ) These are also connected via a Myrinet Switch at 2Gbps connection speed.

IPMI Service Processors : Each box contains a service processor with separate network interface for remote monitoring and control.

Inside a V20

Iceberg Cluster ConfigurationIceberg Cluster ConfigurationH

EA

D N

OD

E

Worker node 7

Service Proc 7

Worker node 1

Service Proc 1

Worker node 6

Service Proc 6

Worker node n

Service Proc n

Worker node 56

Service Proc n

Worker node 2

Service Proc 2

Worker node n

Service Proc n

Worker node 57

Service Proc 57

Worker node 4

Service Proc 4

Worker node 9

Service Proc 9

Worker node n

Service Proc n

Worker node 59

Service Proc 59

Worker node 3

Service Proc 1

Worker node 8

Service Proc 8

Worker node n

Service Proc n

Worker node 58

Service Proc 58

Worker node 5

Service Proc 5

Worker node 10

Service Proc 10

Worker node n

Service Proc n

Worker node 60

Service Proc 60

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

Eth

0

Eth

1

nfs mounted onto

Worker nodes

Sh

ared

file sto

re

All remote access

License

server

Myranet ConnectedWorkers

White Rose GridWhite Rose Grid

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 7

Service Proc n

Workernode 7

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 1

Service Proc 1

Workernode 1

Service Proc 1

Workernode 6

Service Proc n

Workernode 6

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 56

Service Proc n

Workernode 56

Service Proc n

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 2

Service Proc 2

Workernode 2

Service Proc 2

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 57

Service Proc n

Workernode 57

Service Proc n

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 4

Service Proc 1

Workernode 4

Service Proc 1

Workernode 9

Service Proc n

Workernode 9

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 59

Service Proc n

Workernode 59

Service Proc n

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 3

Service Proc 1

Workernode 3

Service Proc 1

Workernode 8

Service Proc n

Workernode 8

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 58

Service Proc n

Workernode 58

Service Proc n

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 5

Service Proc 1

Workernode 5

Service Proc 1

Workernode 10

Service Proc n

Workernode 10

Service Proc n

Workernode n

Service Proc n

Workernode n

Service Proc n

Workernode 60

Service Proc n

Workernode 60

Service Proc n

YHMAN Network

Grid & HPC applications development toolsGrid & HPC applications development tools

• Development• Fortran77,90, C, C++, Java compilers

• MPI / MPICH-gm

• OpenMP

• Nag Mk 20, 21

• ACML

• Grid• Sun Grid Engine

• Globus 2.4.3 (via gpt 3.0)

• SRB s-client tools

Using the White Rose Grid Using the White Rose Grid

Application PortalApplication Portal

Features and CapabilitiesFeatures and Capabilities

• Web accessible management and execution of applications

• Provides a service for rapid authoring and publication of custom applications

• Easily integrate multiple heterogeneous resources

Potential benefits of an applications portalPotential benefits of an applications portal

• More efficient use of resources

• Ease of use

• Familiar GUI interface

• Capturing of expert knowledge

• Better presentation of legacy software

Potential DevelopmentPotential Development

• Building Expert Systems

• Allowing novice expert to take advantage of parallel HPC resources

• Providing HPC services over the grid

• HPC centres collaborating with each other without having to provide individual usernames, file-storage etc to remote users.

WRG – Application PortalWRG – Application Portal

• Based on EASA

• Three Usage Modes• Users

Run applications

Have storage space

Review old results

• AuthorsBuild and publish applications

• Administrators

UsingUsing

• Accessing

• Managing

• Applications

• Workspace

• Results

• Help

Using:AccessingUsing:Accessing

• Start up a web browser &

http://www.shef.ac.uk/wrgrid/easa.html

• Login using provided user name and password

Using:HelpUsing:Help

• Select Help and Support tab to register

• Apply to Admin for an account

• Apply to authors to register applications

Using:ManagingUsing:Managing

• Installing a client

• Setting password

• Setting Mode• user/author

Using:ApplicationsUsing:Applications

• View and select available applications

Running An ApplicationRunning An Application

User InterfaceUser Interface

Using:WorkspaceUsing:Workspace

• Storage for uploaded files and old job files

Using:ResultsUsing:Results

• Check results

• View job progress

• Export to spreadsheet

Using: ResultsUsing: Results

• Viewing Results

Using:HelpUsing:Help

• Documentation

• Contacts

ConclusionsConclusions

• Disadvantages• Thick Client, License costs

• Advantages• Rapid publication

• Enable virtualization of HPC resources

• Make applications available to broader community, become application focused

• Effective on a network with low bandwidth

• Make applications available to collaboration partners over the internet and outside own organisation

Demonstration Applications Developed for Demonstration Applications Developed for EASAEASA

• Demonstration of Metascheduling Across White Rose Grid

• Monitoring of usage across White Rose Grid

• Running Applications on the local cluster• Fluent

• Ansys

• Generic Matlab and Scilab applications

Metascheduler Demonstration:BackgroundMetascheduler Demonstration:Background

• Enable utilisation of resources across White Rose Grid

• Exploit use of task arrays

• Job submission is seamless

• Demonstration uses a generic scilab application that runs on any of the White Rose Grid Nodes

• Simplistic, but;• effective, manageable and sustainable

Metascheduler Demonstration: MethodMetascheduler Demonstration: Method

• Query and Compare job queues for WRG nodes• qstat –g c• Use slots available and total number of slots to generate weights for

different queues• Compare weights for all queues on different nodes and use to select

node• Use standard EASA job submission technique to submit job to

selected node• EASA does not know about clusters• Special easaqsub submits job to sge, monitors job status will remove

job if wait time exceeded, easaqsub job monitor has completed EASA knows that EASA compute task has completed

Metascheduler Demonstration: Metascheduler Demonstration: Running ScilabRunning Scilab

• User provides scilab scriptfile• Required resource file e.g.

datafiles or files for scilab library routines• Can provide zipped bundle

of scilab resources• Set job submission

information and then submit job

Metascheduler Demonstration: Job Metascheduler Demonstration: Job SubmissionSubmission

• Provide jobname and job description

• Information used for metascheduling

• Jobtime (hours)• Waittime (hours)• Number of tasks (for job

array)• Submission method

• Use metascheduling• Select a particular node

Metascheduler Demonstration : Metascheduler Demonstration : Further DevelopmentsFurther Developments

• Current method successful!• Correctly selects clusters and improves turnaround for scilab compute

tasks

• Current pattern can be extended to other EASA applications

• Provide distributed storage across White rose Grid

• Develop metascheduling strategy introduce greater dependency on user job requirements for node selection

• Exploit other metascheduling systems e.g. SGE transfer queues, CONDOR-G

THE END