Parallel SystemsTools

Dr. Guy Tel-Zur

Agenda - I

• Environments– Where to execute my code?– Windows

• Native (DenioMPI, DevCPP, Visual Studio)• Virtualization (Virtual Box and VMWare Player)• PuTTY, WinSCP

– Linux• gcc• ssh• editors

Agenda - II

• Compiling and executing a MPI program• Debugging (Allinea DDT)• Profiling (Jumpshot3, IPM, Scalasca,

OpenSpeed Shop)• Eclipse PTP• Torque

Environment - Infrastructure

• Code development – Your own laptop, lab computers

• Execution – The educational cluster

Deino MPI

• Installation• The famous “cpi.c”• Profiling

http://mpi.deino.net/index.htm

Click here first

/* -*- Mode: C; c-basic-offset:4 ; -*- *//* * (C) 2006 by Deino Software. * (C) 2001 by Argonne National Laboratory. * See COPYRIGHT in top-level directory. */#include "mpi.h"#include <stdio.h>#include <math.h>

double f(double);double f(double a){ return (4.0 / (1.0 + a*a));}int main(int argc,char *argv[]){ int n, myid, numprocs, i; double PI25DT = 3.141592653589793238462643; double mypi, pi, h, sum, x; double startwtime = 0.0, endwtime; int namelen; char processor_name[MPI_MAX_PROCESSOR_NAME];

MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); MPI_Get_processor_name(processor_name,&namelen);

fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name);

fflush(stdout);

n = 10000; /* default # of rectangles */ if (myid == 0)

startwtime = MPI_Wtime(); MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD);

h = 1.0 / (double) n; sum = 0.0; /* A slightly better approach starts from large i and works back */ for (i = myid + 1; i <= n; i += numprocs) {

x = h * ((double)i - 0.5);sum += f(x);

} mypi = h * sum;

MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD);

if (myid == 0) {endwtime = MPI_Wtime();printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT));printf("wall clock time = %f\n", endwtime-startwtime); fflush(stdout);

} MPI_Finalize(); return 0;}

Jumpshot: clog to slog conversion

Profiling

Code Development under Windows

• Open Source/Free– DevC++– Cygwin– Virtualization

• Commercial– Visual Studio

DevC++

• http://www.bloodshed.net/devcpp.html

Working with DevC++

Select the path to the installed MPI library

Adding the link to the MPI library

Compiler options - includes

Select the path to the right include directory

Code Development under Linux

• Connecting from Windows to Linux– PuTTY (+ Xming)– Remote Desktop– VNC

PuTTY

PuTTY

Putty + X windows

Install xming and xming fonts

Start first Xming Server

IPMIntegrated Performance Monitoring

Read instructions at:http://ipm-hpc.sourceforge.net/userguide.html

The Wrong Way and the Correct Way to Execute Jobs

Batch Queuing System

TorqueFrequently Used Shell Commands Basic Usage Example

qsub submit a pbs job qsub [script] $ qsub job.pbs

qstatshow status of pbs batch jobs qstat [job_id] $ qstat 44

qdeldelete pbs batch job qdel [job_id] $ qdel 44

qholdhold pbs batch jobs qhold [job_id] $ qhold 44

qrlsrelease hold on pbs batch jobs qrls [job_id] $ qrls 44

Reference: http://www.rcc.uh.edu/hpc-docs/49-using-torque-to-submit-and-monitor-jobs.html

TorqueShell Commands to Check Queue and Job Status

qstat -q list all queuesqstat -a list all jobs

qstat -au userid list jobs for userid

qstat -r list running jobs

qstat -f job_id list full information about job_id

qstat -Qf queue list full information about queue

qstat -B list summary status of the job server

pbsnodes list status of all compute nodes

Torque

Basic #PBS commands#PBS -N myjob Set the job name

#PBS -m ae Mail status when the job completes

#PBS -M your@email.address Mail to this address

#PBS -l nodes=4 Allocate specified number of nodes

#PBS -l walltime=1:00:00Inform the PBS scheduler of the expected runtime

#PBS in your Job ScriptThe job script is any shell script you normally run to execute your programs. The #PBS commands appear to be comments to the shell but when your script is submitted to the PBS job scheduler

Next slide from: http://www.democritos.it/activities/IT-MC/documentation/newinterface/pages/runningcodes.html

Option Description

#PBS -N myJob Assigns a job name. The default is the name of PBS job script.

#PBS -l nodes=4:ppn=2 The number of nodes and processors per node.

#PBS -q queuename Assigns the queue your job will use.

#PBS -l walltime=01:00:00 The maximum wall-clock time during which this job can run.

#PBS -o mypath/my.out The path and file name for standard output.

#PBS -e mypath/my.err The path and file name for standard error.

#PBS -j oeJoin option that merges the standard error stream with the standard output stream of the job.

#PBS -W stagein=file_list Copies the file onto the execution host before the job starts. (*)

#PBS -W stageout=file_list Copies the file from the execution host after the job completes. (*)

#PBS -m b Sends mail to the user when the job begins.

#PBS -m e Sends mail to the user when the job ends.

#PBS -m a Sends mail to the user when job aborts (with an error).

#PBS -m baAllows a user to have more than one command with the same flag by grouping the messages together on one line, else only the last command gets executed.

#PBS -r n Indicates that a job should not rerun if it fails.

#PBS -V Exports all environment variables to the job.

Torque and MPI

#!/bin/bash#! Name of the job ('MPI' in this example): #PBS -N MPICMD="mpirun -tune -ppn $ppn -np $np $application $options“#! Or this:#CMD="mpirun -np $np $application”eval $CMD

Another example# The name of the script is myjob#PBS -N myjob

# Only 1 hour wall-clock time will be given to this job#PBS -l walltime=1:00:00

# Number of cores to be allocated is 288.# always ask for complete nodes #PBS -l mppwidth=288

#PBS -e error_file.e#PBS -o output_file.o

# Change to the work directorycd $PBS_O_WORKDIR

# Run the executable named myexe # and write the output into my_output_filempirun -n 288 ./myexe > my_output_file 2>&1

Parallel Debugger – Allinea DDT

Not available at the BGU

Allinea DDT

The Allinea DDT Parallel debuggerTutorials, Webinars