Inter-Process Communication (IPC)

Dr. Yingwu Zhu

Purposes for IPC

• Data Transfer

• Sharing Data

• Event notification

• Resource Sharing and Synchronization

• Process Control

IPC from the OS Point of View

OS address space

Process A Process B

Private address space

Private address space

Fundamental IPC Problem for the OS

• Each process has a private address space

• Normally, no process can write to another process’s space

• How to get important data from process A to process B?

OS Solutions to IPC Problem

• Fundamentally, two options

1. Support some form of shared address space

– Shared memory

2. Use OS mechanisms to transport data from one address space to another

– Message passing: Files, messages, pipes, sockets, RPC, etc.

Fundamental Differences in OS Treatment of IPC Solutions

• Shared memory

–OS has job of setting it up

–And perhaps synchronizing

–But not transporting data

• Message passing

–OS involved in every IPC

–Os transports data

IPC vs. Synchronization

• Synchronization is a major concern for IPC

– Allowing sender to indicate when data is transmitted

– Allowing receiver to know when data is ready

– Allowing both to know when more IPC is possible

Message Passing• In a Message passing system there are no shared variables.

IPC facility provides two operations for fixed or variable sized message:– send(message)– receive(message)

• If processes P and Q wish to communicate, they need to:– establish a communication link– exchange messages via send and receive

• Implementation of communication link– physical (e.g., shared memory, hardware bus, network links)– logical (e.g., syntax and semantics, abstractions)

Message passing: implementation questions

• How are links established?

• Can a link be associated with more than two processes?

• How are links made known to processes?

• How many links can there be between every pair/group of communicating processes?

• What is the capacity of a link?

• Is the size of a message that the link can accommodate fixed or variable?

• Is a link unidirectional or bi-directional?

Message Passing Systems

•Exchange messages over a communication link

•Methods for implementing the communication link and primitives (send/receive):

1.Direct or Indirect communications (Naming)

2.Symmetric or Asymmetric communications

3.Automatic or Explicit buffering

4.Send-by-copy or send-by-reference

5.fixed or variable sized messages

• Processes must name each other explicitly:– Symmetric Addressing

• send (P, message) – send to process P

• receive(Q, message) – receive from Q

– Asymmetric Addressing• send (P, message) – send to process P

• receive(id, message) – rx from any; system sets id = sender

• Properties of communication link– Links established automatically between pairs

– processes must know each others ID

– Exactly one link per pair of communicating processes

• Disadvantage: a process must know the name or ID of the process(es) it wishes to communicate with

Direct Communication – Internet and Sockets

Indirect Communication - Pipes

• Messages are sent to or received from mailboxes(also referred to as ports).– Each mailbox has a unique id.– Processes can communicate only if they share a mailbox.

• Properties of communication link– Link established only if processes share a common mailbox– A link may be associated with more than 2 processes.– Each pair of processes may share several communication links.

• Ownership:– process owns (i.e. mailbox is implemented in user space): only

the owner may receive messages through this mailbox. Other processes may only send. When process terminates any “owned” mailboxes are destroyed.

– system owns – then mechanisms provided to create, delete, send and receive through mailboxes. Process that creates mailbox owns it (and so may receive through it) but may transfer ownership to another process.

Indirect Communication

• Mailbox (a named area) sharing:– P1, P2, and P3 share mailbox A.

– P1, sends; P2 and P3 receive.

– Who gets the message?

• Solutions– Allow a link to be associated with at most two processes.

– Allow only one process at a time to execute a receive operation.

– Allow the system to select arbitrarily the receiver. Sender is notified who the receiver was

Synchronizing Message Flow

• Message passing may be either blocking or non-blocking.– blocking send: sender blocked until message received by

mailbox or process

– nonblocking send: sender resumes operation immediately after sending

– blocking receive: receiver blocks until a message is available

– nonblocking receive: receiver returns immediately with either a valid or null message.

Buffering

• All messaging system require framework to temporarily buffer messages. These queues are implemented in one of three ways:

1. Zero capacity – No messages may be queued within the link, requires sender to block until receives retrieves message.

2. Bounded capacity – Link has finite number of message buffers. If no buffers are available then sender must block until one is freed up.

3. Unbounded capacity – Link has unlimited buffer space, consequently send never needs to block.

Acknowledgements

• A message back to sender indicating that original message was received correctly

• May be sent piggy-back on another message

– Implicit or explicit

• May be synchronous or asynchronous

• May be positive or negative

Message Passing issues

• Scrambled messages (checksum)

• Lost messages (acknowledgements)

• Lost acknowledgements (sequence no.)

• Destination unreachable (down, terminates)– Mailbox full

• Naming

• Authentication

• Performance (copying, message building)

Typical UNIX IPC Mechanisms

• Different versions of UNIX introduced different IPC mechanisms

– Pipes

– Message queues

– Semaphores

– Shared memory

– Sockets

– RPC

Pipes

• Only IPC mechanism in early UNIX systems (other than files)– Uni-directional (one process writes and the other reads )

– Unformatted

– Uninterpreted

– Interprocess byte streams

– blocking communication (no data blocks receiver, no capacity blocks writer)

• Accessed in file-like way

• Only used for parent-child or sib process pairs– Named pipes allow more flexibility

Pipes

• The input of one process is made the input of another. We have seen examples of this from the UNIX command line using |

• UNIX allows two ways of opening a pipe– popen(), pclose(), read/write

– pipe(), close(), read/write

popen() -- Formatted Piping

• FILE *popen(char *command, char *type) – opens a pipe for I/O where the command is the process that will be connected

to the calling process thus creating the pipe. The type is either ``r'' - for reading, or ``w'' for writing

• popen() returns is a stream pointer or NULL for any errors

• A pipe opened by popen() should always be closed by pclose(FILE *stream)

• We use write() and read() to communicate with the pipe's stream

Reading Output From an External Program

#include <unistd.h>#include <stdlib.h>#include <stdio.h>#include <string.h>

int main(){

FILE *read_fp;char buffer[BUFSIZ + 1];int chars_read;memset(buffer,’\0’,sizeof(buffer));read_fp = popen(“uname –a”, “r”);if (read_fp !=NULL){

chars_read = fread(buffer, sizeof(char), BUFSIZ, read_fp);if (chars_read > 0){

printf(“Output was:-\n%s\n”,buffer);}pclose(read_fp);exit(EXIT_SUCCESS);

}exit(EXIT_FAILURE);

}

Sending Output to popen

#include <unistd.h>#include <stdlib.h>#include <stdio.h>int main(){

FILE *write_fp;char buffer[BUFSIZ + 1];sprintf(buffer, “Once upon a time, there was …\n”);write_fp = popen (“od –c”, “w”);if (write_fp != NULL){

fwrite( buffer , sizeof(char), strlen(buffer), write_fp);pclose(write_fp);exit(EXIT_SUCCESS);

}exit(EXIT_FAILURE);

}

pipe() -- Low level Piping

• int pipe(int fd[2])– creates a pipe and returns two file descriptors, fd[0], fd[1]

– fd[0] is opened for reading, fd[1] for writing

– pipe() returns 0 on success, -1 on failure and sets errno accordingly

• The standard programming model is that after the pipe has been set up, two (or more) cooperative processes will be created by a fork and data will be passed using read() and write()

• Pipes opened with pipe() should be closed with close(int fd )

#include <unistd.h>

#include <stdlib.h>

#include <stdio.h>

#include <string.h>

int pdes[2];

char string[128];

char string2[] = "This msg is ent to child process";

main(){

pipe(pdes);

if(fork()==0){

close(pdes[1]);

read(pdes[0],string,128);

printf("The msg recieved from parent process:\n");

printf("%s\n",string);

}

else{

close(pdes[0]);

write(pdes[1],string2,strlen(string2));

}

}

Named Pipes: FIFOs

• Make unrelated processes to be able to exchange data

• A named pipe is a special type of file that exists as a name in the file system, but behaves like the unnamed pipes– We can create named pipes from the command line$ mkfifo filename

E.g.

$ mkfifo fpipe

$ grep “.c” < fpipe &

$ls > fpipe

– From inside a program, we can use the following call:

#include <sys/types.h>

#include <sys/stat.h>

Int mkfifo(const char *filename, mode_t mode);

Message Queues

• A message queue is a queue onto which messages can be placed

• A message is composed of a message type (which is a number), and message data

• A message queue can be either private, or public

– If it is private, it can be accessed only by its creating process or child processes of that creator

Creating a Message Queue - msgget()

• This system call accepts two parameters - a queue key, and flags

– The key may be one of: • IPC_PRIVATE - used to create a private message queue.

• a positive integer - used to create (or access) a publicly-accessible message queue

• The second parameter contains flags that control how the system call is to be processed

– It may contain flags like IPC_CREAT or IPC_EXCL

– The lowest 9 bits of the flags are used to define access permission (3 groups, 3 bits each group)

The Message Structure - struct msgbuf

struct msgbuf

{

long mtype; /* message type, a positive

number (cannot be zero). */

char mtext[1]; /* message body array.

usually larger than one byte. */

};

Writing Messages Onto A Queue -msgsnd()

• It takes the following parameters: – int msqid - id of message queue, as returned from the

msgget() call. – struct msgbuf* msg - a pointer to a properly initializes

message structure– int msgsz - the size of the data part (mtext) of the

message, in bytes. – int msgflg - flags specifying how to send the message.

Reading a Message From The Queue -msgrcv()

• This system call accepts the following list of parameters: – int msqid - id of the queue, as returned from msgget(). – struct msgbuf* msg - a pointer to a pre-allocated msgbuf structure. It

should generally be large enough to contain a message with some arbitrary data.

– int msgsz - size of largest message text we wish to receive. Must NOT be larger then the amount of space we allocated for the message text in 'msg'.

– int msgtyp - Type of message we wish to read. may be one of: • 0 - The first message on the queue will be returned. • a positive integer - the first message on the queue whose type (mtype)

equals this integer (unless a certain flag is set in msgflg, see below). • a negative integer - the first message on the queue whose type is less

than or equal to the absolute value of this integer.

Message flag

• int msgflg - a logical 'or' combination of any of the following flags: – IPC_NOWAIT - if there is no message on the queue matching what we

want to read, return -1

– MSG_EXCEPT - if the message type parameter is a positive integer, then return the first message whose type is NOT equal to the given integer.

– MSG_NOERROR - If a message with a text part larger than 'msgsz' matches what we want to read, then truncate the text when copying the message to our msgbuf structure. If this flag is not set and the message text is too large, the system call returns '-1'

Example code

/* prepare a message structure large enough to read our "hello world". */ struct msgbuf* recv_msg = (struct msgbuf*)malloc(sizeof(struct msgbuf)+strlen("hello world")); /* use msgrcv() to read the message. We agree to get any type, and thus */ /* use '0' in the message type parameter, and use no flags (0). */ int rc = msgrcv(queue_id, recv_msg, strlen("hello world")+1, 0, 0); if (rc == -1) {

perror("msgrcv"); exit(1);

}

IPC – signals

• Similar to hardware interrupt.

– Processes interrupt each other

– Non-process activities interrupt processes

• Asynchronous! Stops execution then restarts

– Keyboard driven – e.g. ctrl-C

– An alarm scheduled by the process expires • Unix: SIGALRM from alarm() or settimer()

– resource limit exceeded (disk quota, CPU time...)

– programming errors: invalid data, divide by zero, etc.

Signals Overview• Divided into asynchronous (CTL-C) and synchronous

(illegal address)

• Three phases to processing signals:

– generation: event occurs requiring process notification

– delivery: process recognizes and takes appropriate action

– pending: between generation and delivery

• SVR4 and 4.4BSD define 31 signals, original had 15. Some commercial system support > 32.

• Signal to integer mappings differ between BSD and System V implementations

Actions• Handling, default actions

– Abort: terminate process, generate core dump– Exit: terminate without generating core dump– Ignore: ignore signal– Stop: suspend process– Continue: resume process

• User specified actions– Default action, – Ignore signal, – Catch signal - invoke user specified signal handler

• User may– not ignore, catch or block SIGKILL and SIGSTOP– change action at any time– block signal: signal remains pending until unblocked

Signal Generation• Exceptions - kernel notifies process with signal

• Other Process - using kill or sigsend.

• Terminal interrupts - stty allows binding of signals to specific keys, sent to foreground process

• Job control - background processes attempt to read/write to terminal. Process terminate or suspends, kernels sends signal to parent

• Quotas - exceeding limits

• Notifications - event notification (device ready)

• Alarms - process notified of alarm via signal reception

Reliable Signals - BSD

• Persistent handlers

• Masking signals

– signals masked (blocked) temporarily

– user can specify mask set for each signal

– current signal is masked when handler invoked

• Interruptible sleeps

• Restartable system calls

• Allocate separate stack for handling signals

Signals - A Few Details• Any process or interrupt can post a signal

– set bit in pending signal bit mask

– perform default action or setup for delivery

• Signal typically delivered in context of receiving process.– exception is sending SIGSTOP, kernel may perform

action directly

– Pending signals are checked before returning to user mode and just before/after certain sleep calls.

– Produce core dump or invoke signal handler

Shared Memory

• Simplest, fastest, but local communication

• Allow two or more processes to access the same memory (as if they called malloc and were returned pointer to the same memory address)– Let multiple processes attach a segment of physical memory to their

virtual address spaces

• One changes the memory, then all others see the change!

Caution on Shared Memory

• Sometimes, you may need to synchronize accesses to the shared memory

– Synchronization issue!

Overview

• Creation: shmget()

• Access control: shmctl()

• Attached to addr. space: shmat()

• Detached from addr. space : shmdt()

• #include <sys/types.h>

• #include <sys/ipc.h>

• #include <sys/shm.h>

Notes

• A process creates a shared memory segment using shmget().

– The original owner of a shared memory segment can assign ownership to another user with shmctl(). It can also revoke this assignment.

• Other processes with proper permission can perform various control functions on the shared memory segment using shmctl().

• Once created, a shared segment can be attached to a process address space using shmat(). It can be detached using shmdt().

– The attaching process must have the appropriate permissions for shmat(). Once attached, the process can read or write to the segment, as allowed by the permission requested in the attach operation.

– A shared memory segment is described by a control structure with a unique ID that points to an area of physical memory. The identifier of the segment is called the shmid. The structure definition for the shared memory segment control structures and prototypes can be found in <sys/shm.h>.

Create a Share Memory Segment

• int shmget(key_t key, size_t size, int shmflg) key: integer ID

processes can access the same seg. by using the same key; IPC_PRIVATE as the key guarantee a new seg. is created

size: number of bytes, actual bytes rounded up to multiple of page size

shmflg: access permissions flags and creation control flags specify IPC_CREAT, if a segment for the key does not exist, it is created

If you specify IPC_CREAT | IPC_EXCL, the key must not exist, otherwise fails. If the key exists and the ICP_EXCL is not given, the existing seg. is returned!

Permission flag to indicate permissions granted to owner, group and world, see <sys/stat.h> for constants, or simply bits

#define PERM_UREAD 0400 #define PERM_UWRITE 0200 #define PERM_GREAD 0040 #define PERM_GWRITE 0020 #define PERM_OREAD 0004 #define PERM_OWRITE 0002

return: shared memory segment ID on success int segment_id = shmget (shm_key, getpagesize (),IPC_CREAT | S_IRUSR | S_IWUSER);

int segment_id = shmget (shm_key, 1000,IPC_CREAT | 0666);

Attach a Shared Memory Segment

• Must attach it before using!• void *shmat (int shmid, const void*shmaddr, int shmflg);

– returns a pointer to the head of the shared segment associated with a valid shmid

– #2 para: specifies where in your process’s address space you want to map the shared memory; if you specify NULL, Linux will choose an available address

– #3 para:• SHM_RND indicates that the address specified for the second parameter should be

rounded down to a multiple of the page size. If you don’t specify this flag, you must page-align the second argument to shmat yourself.

• SHM_RDONLY indicates that the segment will be only read, not written.

Detaching a Share Mem. Seg.

• Never forget to detach it!

• int shmdt(const void* shmaddr);– Detaches the shared memory segment located at the address

indicated by shmaddr

Controlling a Shared Mem. Seg.

• int shmctl(int shmid, int cmd, struct shmid_ds *buf);

• cmd SHM_LOCK -- Lock the specified shared memory segment in memory. The

process must have the effective ID of superuser to perform this command.

SHM_UNLOCK -- Unlock the shared memory segment. The process must have the effective ID of superuser to perform this command.

IPC_STAT -- Return the status information contained in the control structure and place it in the buffer pointed to by buf. The process must have read permission on the segment to perform this command.

IPC_SET -- Set the effective user and group identification and access permissions. The process must have an effective ID of owner, creator or superuser to perform this command.

IPC_RMID -- Remove the shared memory segment.

The buf is a structure of type struct shmid_ds which is defined in <sys/shm.h>

Deleting Segment

• When you detach from the segment, it isn't destroyed. Nor is it removed when everyone detaches from it. You have to specifically destroy it using a call to shmctl(), similar to the control calls for the other System V IPC functions:

shmctl(shmid, IPC_RMID, NULL);

Examples: server.c

main() {char c; int shmid;key_t key; char *shm, *s; /* * We'll name our shared memory segment * "5678". */ key = 5678;/* * Create the segment. */ if ((shmid = shmget(key, SHMSZ, IPC_CREAT | 0666)) < 0) {

perror("shmget"); exit(1); }

/* * Now we attach the segment to our data space. */ if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {

perror("shmat"); exit(1); } /* * Now put some things into the memory for the * other process to read. */ s = (char*)shm; for (c = 'a'; c <= 'z'; c++)

*s++ = c; *s = NULL; /* * Finally, we wait until the other process * changes the first character of our memory * to '*', indicating that it has read what * we put there. */while (*shm != '*')

sleep(1); exit(0);

}

client.cmain() {

int shmid; key_t key; char *shm, *s; /* * We need to get the segment named * "5678", created by the server. */

key = 5678;/* * Locate the segment. */ if ((shmid = shmget(key, SHMSZ, 0666)) < 0) {

perror("shmget"); exit(1); } /* * Now we attach the segment to our data space. */ if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {

perror("shmat"); exit(1); } /* * Now read what the server put in the memory. */ for (s = shm; *s != NULL; s++)

putchar(*s); putchar('\n'); /* * Finally, change the first character of the * segment to '*', indicating we have read * the segment. */ *shm = '*'; exit(0);

}

server.c: does this have problem?

main() {char c; int shmid;key_t key; char *shm, *s; /* * We'll name our shared memory segment * "5678". */ key = 5678;/* * Create the segment. */ if (shmid = shmget(key, SHMSZ, IPC_CREAT | 0666) < 0) {

perror("shmget"); exit(1); }

/* * Now we attach the segment to our data space. */ if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {

perror("shmat"); exit(1); } /* * Now put some things into the memory for the * other process to read. */ s = (char*)shm; for (c = 'a'; c <= 'z'; c++)

*s++ = c; *s = NULL; /* * Finally, we wait until the other process * changes the first character of our memory * to '*', indicating that it has read what * we put there. */while (*shm != '*')

sleep(1); exit(0);

}

Fork and shmat

• After fork(), the child inherits the attached shared memory segments

Debugging

• ipcs –m

– Get info about shared memory------ Shared Memory Segments --------

key shmid owner perms bytes nattch status

0x00000000 1627649 user 640 25600 0

• ipcrm shm 1627649– Remove the shared memory was erroneously left behind by a program