Rocks ‘n’ Rolls

An Introduction to Programming Clusters

using Rocks

© 2008 UC Regents

Anoop Rajendra

Rules of the Talk

• This talk is for YOU!!• Let me know if I’m too fast• Don’t hesitate to stop me and ask

questions in the middle.

© 2008 UC Regents

What is Rocks?

• A Linux distribution based on CentOS• Cluster on a DVD ( or set of CDs)• Goals of Rocks

– Make Clusters Easy– Reduce dependence on system

administrators by reducing system administration

– Enable scientists to concentrate on science rather than on computing.

© 2008 UC Regents

How do we get Started?

• Get a bunch of x86/x64 machines• Put 2 ethernet cards on one of them

- This’ll be your front-end/head-node• Wire 'em up.• Download Rocks isos from

www.rocksclusters.org• Burn a bunch of CDs or a DVD• Get started

© 2008 UC Regents

Topology of a Rocks Cluster

© 2008 UC Regents

Frontend Node Public Ethernet

PrivateEthernet

ApplicationNetwork (Optional)

Node Node Node Node Node

Node Node Node Node Node

Pow

er Distribution

(Net addressable units as option)

Another way to look at it

© 2008 UC Regents

Installing Rocks

• Rocks provides for installation of a cluster with minimum user interaction

• Steps involved– Boot from CD or DVD– Answer a few questions– Get favorite beverage– You’re on your way to a full fledged cluster

in 2 hours or less.

© 2008 UC Regents

Innards of Automatic Installs

• Rocks uses Redhat’s Kickstart mechanism for automated installs

• Kickstart file generated and given to the installer

• Installer follows instructions in Kickstart file.

• Generation of Kickstart file done automatically by Rocks scripts

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

After Frontend Installation

© 2008 UC Regents

Add compute nodes using insert-ethers

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

© 2008 UC Regents

How does Rocks do all this?

Using a mechanism called Rolls

© 2008 UC Regents

What are Rolls?

• Software components that make up a Rocks system

• Mechanism for delivery of packages and configuration

• Rolls are how you’re going to get your software onto a Rocks cluster

© 2008 UC Regents

What are some of the available rolls?

• Sun Grid Engine Roll – Deploys SGE on all nodes and enabled job provisioning

• Bio Roll – Application Roll for Bio-informatics applications

• Xen Roll – Roll to deploy virtual machines• Viz Roll – Roll that enabled building tile

wall displays• PVFS2 Roll – Installs and Configures the

Parallel Virtual File System on your cluster© 2008 UC Regents

Viz Wall

© 2008 UC Regents

Another Viz Wall

© 2008 UC Regents

Roll Infrastructure

• Rolls are distributed as ISO images• Add a CD or leave a CD out during

installation & you control the configuration and capability of your cluster

© 2008 UC Regents

What’s inside a Roll?

• Binaries - RPM format• Configuration data• Installation Map

© 2008 UC Regents

Context of a Roll

© 2008 UC Regents

Normal Redhat Distribution

© 2008 UC Regents

Distribution based on Rolls

© 2008 UC Regents

Treasure Hunting

• The treasure you seek is a fully installed and configured cluster

• What are the things you’ll need

© 2008 UC Regents

Map

© 2008 UC Regents

Instruction set

© 2008 UC Regents

Resources

© 2008 UC Regents

Translate that to a Roll

• Graph file ⇔ Installation Map ⇔ Map• Node Files ⇔ Configuration Data ⇔

Instruction set• RPM/Binaries ⇔ Resources

© 2008 UC Regents

Bio Roll

© 2008 UC Regents

What’s inside the Bio Roll?

• Bio-informatics software like– NCBI Blast– MPI Blast– HMMER– GROMACS– EMBOSS– Phylip– TCoffee

© 2008 UC Regents

What’s inside the Roll?

© 2008 UC Regents

What’s the graph file?

<graph><edge from="compute">

<to>bio-base</to></edge>……<edge from="hpc-server">

<to>bio-head</to></edge></graph>

© 2008 UC Regents

How does the graph look?

© 2008 UC Regents

How does a node file look?<kickstart>

<package>hmmer</package><package>ncbi</package>…… <package>mpiblast</package>

<post># Shell Scripts to be run after installation<file name=.ncbirc>file contents</file>……# Shell Scripts to be run after installation</post></kickstart>

© 2008 UC Regents

Node files are the instruction sets in the roll.

© 2008 UC Regents

To Re-iterate

© 2008 UC Regents

© 2008 UC Regents

Bio-informaticsApplications

© 2008 UC Regents

ConfigurationInformation

© 2008 UC Regents

Bio-informaticsApplication

THE ROLL

ConfigurationInformation

How does it fit in?

© 2008 UC Regents

BioRoll

How does it fit in?

© 2008 UC Regents

How does all this help you?

• Roll mechanism is the recommended way of deploying software on a Rocks cluster

• It fits into the framework of Rocks• It’s reproducible• It scales

© 2008 UC Regents

Other Ways of Software Deployment

• More than one way to get your software onto the rocks cluster

• Not recommended as it is not reproducible over multiple re-installs.

© 2008 UC Regents

RPM Based Deployment

– Put your RPM in• /home/install/contrib/5.0/arch/RPM

– Create the node xml file to reflect new package• # cd /home/install/site-profiles/5.0/nodes• # cp skeleton.xml extend-compute.xml

– Inside extend-compute.xml file, put the line• <package>rpm name</package>

– Re-create the rocks distribution• # cd /home/install• # rocks-dist dist

– Re-install your compute nodes

© 2008 UC Regents

Installing in a Shared directory

• /export/apps is shared across the entire cluster as /share/apps

• Install your package into • /export/apps/package

• On all your compute nodes the package directory will be

• /share/apps/package

© 2008 UC Regents

More Information

• More Documentation available at www.rocksclusters.org

© 2008 UC Regents

Questions

© 2008 UC Regents