CSE 160 – Lecture 5

Introduction to PVM

PVM

• How PVM is structured

• Sending and receiving messages

• Getting started with the programming assignment

PVM Programming Model• Dynamic collection of serial and parallel computers

appear as single distributed memory Virtual Machine.

• Tasks can be dynamically spawned and killed by any other task. • Any PVM task can send a message to any other.• No limit to size or number of messages.

• Model supports fault tolerance, resource control, process control, heterogeneous networks and hosts.

host (one per IP address)pvmd - one PVM daemon per host

pvmd

pvmd

pvmd

How PVM is Designed

libpvm - task linked to PVM library

pvmds fully connected using UDP

task task task

Unix Domain Socketsinner host messages

OS network interface

task task task

Shared Memory

shared memory multiprocessor

P0 P1 P2

task task task

distributed memory MPP

task task tasktask task task

internal interconnect

tcpdirect connect

Step 1 – Enrolling in PVM

• PVM is a user-level library and tasks must be enrolled to be addressed.

myid = pvm_mytid();printf(“ -> %x”, myid);-> 0x40001

• First call to any PVM function will enroll your program PVM.

• Above construct is the standard practice

Task ID = TID

• PVM TIDs are very strange looking numbers.– Give host information– Node information– Type of message (unicast or multicast)– System bit for messages destined to PVMDs (as

opposed to regular tasks).

• PVM uses tid to identify pvmd, tasks, groups

• Fits into 32-bit integer

• S bit addresses pvmd, G bit forms mcast address

• Local part defined by each pvmd - eg. for PGON

Task ID (tid)

18 bits12 bitsS G host ID local part

12 bitsS G host ID process node ID

11 bits7 bits

4096 hosts 2048 nodeseach with

Popular Distributed Programming Schemes

• Master/SlaveMaster task starts all slave tasks

and coordinates their work and I/O

• SPMDSame program executes on different pieces

of the problem

• FunctionalSeveral programs are written. Each performs

a different function in the application

Sending and receiving messages

• Send a messageerr = pvm_send( dest, tag);Dest = TID of destination;Tag = user-defined distinguishing integer;

• Receive a messageErr = pvm_recv( src, tag);Src = TID of senderTag = distinguished integer

Receive Wildcards

• A receiver may wildcard the sending tid or the matching tag. –1 is the wildcard.

Any sender, specific tag

pvm_recv(-1, tag);

Specific sender, any message

pvm_recv(src, -1);

Any sender, any message

pvm_recv(-1, -1);

PVM Examples in Package

Supplied examples include:hello hello_other PVM equivalent to hello worldmaster slave Master/slave examplespmd SPMD examplegexample Group and collective ops.timing timing_slave Timing example - comm perfhitc hitc_slave Dynamic load balanceinheritb Communication contextimbi gmbi Persistent messages templatemhf_server mhf_tickle Message handlers

Examples illustrate PVM usage and serve as templatesto build your own applications.

Examples include Makefile.aimk and both C and Fortran versions.

Starting PVM

pvm [-nhostname] [hostfile]

PVM console. Starts PVM or just attaches to running PVM. Recommended way to start and stop PVM 3.

xpvm [hostfile]

PVM Graphical Interface includes console functions, and more. Starts PVM or attaches to running PVM.

pvmd [-nhostname] [hostfile]direct startup, seldom used except for debugging. Option -n useful if host has multiple network cards eg. ATM, ethernet

Three ways to start PVM

Virtual Machine Construction via the PVM Console

• When you run pvm from the console– If no existing VM for this user, start a new one

– If an existing VM, enroll in PVM and wait for commands.

• Remember! The collection of communicating PVMDs define a VM.

• The pvm console is “just another pvm program”. – Anything that can done from the console, can be done

from your program.

VM Construction Cont’d

• VMs with one physical machine are OK, but not very interesting .– Other machines must be added from the first machine you

started (eg. From whip)whip% pvmpvm> add fraphost frap added

• A hostfile can be used to list all the hosts you want in VM.whip% pvm hostfile(start pvm on whip and add all hosts to VM listed in hostfile)

Some Common Startup Problems

• PVM needs to be to remote shell to a host to add a pvmd and extend the machine– Make sure you rsh to a machine without

needing a password

• Old PVMD was running, but didn’t clean up properly (Pvmd already running? Error)– Look for /tmp/pvmd.uid and remove it

PVM Console

add hostname

alias

conf

delete hostname

halt

help [command]

kill tid

ps -a

quit

reset

sig

spawn

trace

version

pvm> Available commands and their meaning

Add host(s) to virtual machine (can list several)

define/list command alias

list hosts in the virtual machine

delete hosts from virtual machine

shut down PVM and tasks

print information about commands and options

kill a task

list all running tasks

exit console - leave PVM and tasks running

kill all tasks and reset PVM

send signal to a task

spawn tasks (many options)

set/display trace events

print PVM version

Creating a Hostfile

lo= login name for this host

so=pw ask for passwd this host

dx= location of pvmd

ep= path to user’s executables

wd= working directory to spawn tasks

sp= relative speed rating

bx= location of desired debugger

so=ms manual pvm startup this host

Simplest form is just a list of hostnames

Same as initial host

Same as initial host

$PVM_ROOT/lib/pvmd

~/pvm3/bin/$PVM_ARCH

$HOME

1000 (not used by PVM)

$PVM_ROOT/lib/debugger

use to bridge firewalls

Hostfile options Default values

Example: # Beowulf hostfile (comment lines start with #)* ep=/stuff/myapp wd=/stuff/myapphost1 host2 lo=tom so=pw

Starting tasks

• Use spawn in the pvm consolepvm> spawn -> /bin/date pvm decides where to start task

pvm> spawn -> -whip /bin/datestart on host whip

pvm> spawn -> –4 /bin/hostnamestart 4 copies

• Use the pvm_spawn() command to start tasks from your own program

PVM_ROOT

• PVM_ROOT – Where PVM looks for various files

• PVM_ARCH – architecture code for the machine– eg. LINUX, SUN4SOL2, SGI5

• If you do a “spawn -> gexample”, where does it look for the executable?– $PVM_ROOT/bin/$PVM_ARCH– $HOME/bin/$PVM_ARCHBoth of these paths are relative to the remote machine

Capturing Output

• When tasks are spawned, there output goes to /tmp/pvml.uid on the local machine?– Output shows up in many different files

• PVM answer: pvm_catchout(stdout)– Capture standard output/error from spawned (children)

tasks and copy to your standard output.

• The ‘->’ in the console spawn command tells PVM to capture output of all tasks and display

• Remote output is buffered. Data will show up when buffer is full or when remote program ends.

First Part of Homework Assignment

• Write a launcher, that uses pvm_spawn, captures output, and responds to errors by killing tasks.

• Spawning on a particular hostchar *remotehost;

int remtid;

numt = pvm_spawn(task, task_argv, PvmTaskHost, remotehost, 1, &remtid);

• If spawn fails, numt != number of tasks (1);– Errors are sent back in remtid (array)

Pseudo code - LauncherRead nprocs and hostfileRead program name, and argsRead numhosts from hostfile into a hosts arraymytid = pvm_mytid(); /* enroll */pvm_catchout(stdout);for (i = 0; i < nprocs; i++){

numt = pvm_spawn(prog, progargv, PvmTaskHost, &hosts[numhosts % i], &tids[i]);if (numt != 1)

print spawning error code;for (j = 0; j < i; j ++)

pvm_kill(tids[j]);pvm_exit();exit();

}pvm_exit();exit();

Messages

• Messages serve TWO purposes– Exchange data

– Synchronize processes

• PVM uses the concept of message buffers– Buffers may have heterogenous and typed information

in them.

– Message contents are up to the user• Unpacking/packing order are defined by your program. Types

of packed data not included in message itself.

Example Packed Message buffer

nitems Ao A1 A2 Anitems

message length

floatsint

• By default, data is packed in a universal format – XDR (External Data Representation), so that the data can be read by any architecture.

Initializing A Message Buffer

• pvm_initsend(PvmDataDefault)– Initialize the current send buffer

– The buffer itself is hidden.• This causes no end to confusion !

• Suppose you want to send a message formatted as follows:– First entry: integer K, number of strings to follow

– Next K entries: string data

Example – Packing, sending

#define TEST 101pvm_initsend(PvmDataDefault);pvm_pkint(&K, 1, 1);/* pack one integer, note & */

for (i = 0; i < K; i ++)pvm_pkstr(&strings[I]);

pvm_send(dest, TEST);Sends the packed message to dest with tag TEST;

Receiving the Message and Unpacking

#define TEST 101

pvm_recv(-1, TEST);

recv’s the packed message from anybody with tag TEST; Set the current receive buffer for unpacking.

pvm_upkint(&K, 1, 1);

/* find out how many strings in msg */

for (i = 0; i < K; i ++)

pvm_upkstr(&strings[I]);

TIDs revisited

• TIDs have a strange assignment that makes it hard to write SPMD-style programs.

• PVM uses Groups to ease the mapping of TIDs to a 0 – N (for N+1, members).– Each member of a group is called an instance– Algorithms often are based on 0 – N membership

• PVM uses user-defined group names– look at example where N is given as a command arg

• PVM send/recv use TIDs as dest/src NOT instance numbers

Groups

#define GNAME “myGroup”Read nprocs from command linemytid = pvm_mytid();myinst = pvm_joingroup(GNAME);pvm_freezegroup(GNAME, nprocs);pvm_initsend();pvm _pkxxx();pvm_send(pvm_gettid(GNAME, oinst), TAG);

pvm_recv(pvm_gettid(GNAME, ainst), TAG);pvm_upkxxx();pvm_lvgroup(GNAME);pvm_exit();

Ring Gather - Overview

3

2

1

0

send(2, tag)

1

send(0, tag)

3

send(1, tag)

2 time step

4

Print Results

Ring Gather – Pseudo Codeint ndata = 0;myinst = pvm_joingroup(GNAME);if (myinst != nprocs – 1)

pvm_recv(pvm_gettid(GNAME, myinst+1), TAG);pvm_upkint(&ndata,1,1);for (i = 1; i <= ndata; I++)

pvm_upkxxx(&Data[i][0], length, 1); fill data[0][] with local information

if (myinst != 0)ndata ++;pvm_pkint(&ndata,1, 1);for (i = 0; i <= ndata; i++)

pvm_pkxxx(&data[i][0], length, 1); pvm_send(pvm_gettid(GNAME, myinst – 1), TAG);

else /* instance 0 */print out data entries 0 to ndata

Tree Gather - Overview

3

4

5

6

1

2

0

time step

send(1,tag)

send(1,tag)

send(2,tag)

send(2,tag)1

send(0,tag)

send(0,tag)

2

Print Results

3

Tree Gather – Pseudo Codeint ndata = 0; int index = 1; myinst = pvm_joingroup(GNAME);if (2*myinst + 1 < nprocs) /* “left” subtree */

pvm_recv(pvm_gettid(GNAME, 2*myinst+1), TAG);pvm_upkint(&ndata,1,1);for (i = index; i <= ndata; i++)

pvm_upkxxx(&Data[i][0], length, 1);index = ndata + 1;

if (2*myinst + 2 < nprocs) /* “right” subtree */pvm_recv(pvm_gettid(GNAME, 2*myinst+2), TAG);pvm_upkint(&tmpdata,1,1);ndata += tmpdata;for (i = index; i <= ndata; i++)

pvm_upkxxx(&Data[i][0], length, 1);

fill data[0][] with local information

Tree Gather Pseudo Code Part 2if (myinst != 0)

ndata ++;

pvm_pkint(&ndata,1, 1);

for (i = 0; i <= ndata; i++)

pvm_pkxxx(&data[i][0], length, 1);

dst = (myinst – 1)/2;

pvm_send(pvm_gettid(GNAME, dst), TAG);

else /* instance 0 */

print out data entries 0 to ndata

Next Time

• Start on parallel algorithms and parallel programming paradigms