Concurrent Programming - cs.cmu.edu file¢ Many aspects of concurrent programming are beyond the scope of our course

Carnegie Mellon

1 BryantandO’Hallaron,ComputerSystems:AProgrammer’sPerspec;ve,ThirdEdi;on

ConcurrentProgramming15-213:Introduc;ontoComputerSystems23rdLecture,Nov.17,2015

Instructors:RandalE.BryantandDavidR.O’Hallaron

Carnegie Mellon


ConcurrentProgrammingisHard!

¢  Thehumanmindtendstobesequen9al

¢  Theno9onof9meiso<enmisleading

¢  Thinkingaboutallpossiblesequencesofeventsinacomputersystemisatleasterrorproneandfrequentlyimpossible

Carnegie Mellon


ConcurrentProgrammingisHard!

¢  Classicalproblemclassesofconcurrentprograms:§  Races:outcomedependsonarbitraryschedulingdecisionselsewhereinthesystem§  Example:whogetsthelastseatontheairplane?

§  Deadlock:improperresourcealloca;onpreventsforwardprogress§  Example:trafficgridlock

§  Livelock/Starva4on/Fairness:externaleventsand/orsystemschedulingdecisionscanpreventsub-taskprogress§  Example:peoplealwaysjumpinfrontofyouinline

¢  Manyaspectsofconcurrentprogrammingarebeyondthescopeofourcourse..§  but,notallJ§  We’llcoversomeoftheseaspectsinthenextfewlectures.

Carnegie Mellon


Itera9veServers

¢  Itera9veserversprocessonerequestata9me

Client 1 Server Client 2 connect

accept connect

write read

call read

close

accept

write

read

close WaitforservertofinishwithClient1

call read

write

ret read

write ret read read

Carnegie Mellon


WhereDoesSecondClientBlock?

¢  SecondclientaKemptstoconnecttoitera9veserver

¢  Calltoconnectreturns§  Eventhoughconnec;onnot

yetaccepted§  ServersideTCPmanager

queuesrequest§  Featureknownas“TCP

listenbacklog”

¢  Calltorio_writenreturns§  ServersideTCPmanager

buffersinputdata

¢  Calltorio_readlinebblocks§  Serverhasn’twri]en

anythingforittoreadyet.

Clientsocket

rio_readlineb

rio_writen

Connec9onrequest

open_clientfd

connect

Carnegie Mellon


FundamentalFlawofItera9veServers

¢  Solu9on:useconcurrentserversinstead§  Concurrentserversusemul;pleconcurrentflowstoservemul;ple

clientsatthesame;me

User goes out to lunch

Client 1 blocks waiting for user to type in data

Client 2 blocks waiting to read from server

Server blocks waiting for data from Client 1

Client 1 Server Client 2 connect

accept connect

write call read

call read write

call read write ret read

call read

Carnegie Mellon


ApproachesforWri9ngConcurrentServersAllowservertohandlemul;pleclientsconcurrently

1.Process-based§  Kernelautoma;callyinterleavesmul;plelogicalflows§  Eachflowhasitsownprivateaddressspace

2.Event-based §  Programmermanuallyinterleavesmul;plelogicalflows§  Allflowssharethesameaddressspace§  UsestechniquecalledI/Omul(plexing.

3.Thread-based§  Kernelautoma;callyinterleavesmul;plelogicalflows§  Eachflowsharesthesameaddressspace§  Hybridofofprocess-basedandevent-based.

Carnegie Mellon


Approach#1:Process-basedServers¢  Spawnseparateprocessforeachclient

client 1 server client 2

call connect call accept

call read

ret accept call connect

call fgets fork child 1

User goes out to lunch Client 1 blocks waiting for user to type in data

call accept ret accept

call fgets

write fork

call read

child 2

write

call read

ret read close

close

...

Child blocks waiting for data from Client 1

Carnegie Mellon


int main(int argc, char **argv) !{ ! int listenfd, connfd; ! socklen_t clientlen; ! struct sockaddr_storage clientaddr; !! Signal(SIGCHLD, sigchld_handler); ! listenfd = Open_listenfd(argv[1]); ! while (1) { ! clientlen = sizeof(struct sockaddr_storage); ! connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); ! if (Fork() == 0) { ! Close(listenfd); /* Child closes its listening socket */! echo(connfd); /* Child services client */! Close(connfd); /* Child closes connection with client */! exit(0); /* Child exits */! } ! Close(connfd); /* Parent closes connected socket (important!) */! } !} !

Process-BasedConcurrentEchoServer

echoserverp.c

Carnegie Mellon


Process-BasedConcurrentEchoServer(cont)

void sigchld_handler(int sig) !{ ! while (waitpid(-1, 0, WNOHANG) > 0) ! ; ! return; !}

§  Reapallzombiechildren

echoserverp.c

Carnegie Mellon


ConcurrentServer:acceptIllustratedlistenfd(3)

Client1.Serverblocksinaccept,wai4ngforconnec4onrequestonlisteningdescriptorlistenfd

clientfd

Server

listenfd(3)

Client

clientfd

Server2.Clientmakesconnec4onrequestbycallingconnect

Connec9onrequest

listenfd(3)

Client

clientfd

Server3.Serverreturnsconnfdfromaccept.Forkschildtohandleclient.Connec4onisnowestablishedbetweenclientfdandconnfd

ServerChild

connfd(4)

Carnegie Mellon


Client 2 data

Process-basedServerExecu9onModel

§  Eachclienthandledbyindependentchildprocess§  Nosharedstatebetweenthem§  Bothparent&childhavecopiesoflistenfdandconnfd

§  Parentmustcloseconnfd §  Childshouldcloselistenfd

Client 1 server

process

Client 2 server

process

Listening server

process

Connection requests

Client 1 data

Carnegie Mellon


IssueswithProcess-basedServers

¢  Listeningserverprocessmustreapzombiechildren§  toavoidfatalmemoryleak

¢  Parentprocessmustcloseitscopyofconnfd§  Kernelkeepsreferencecountforeachsocket/openfile§  Aderfork,refcnt(connfd) = 2§  Connec;onwillnotbeclosedun;lrefcnt(connfd) = 0

Carnegie Mellon


ProsandConsofProcess-basedServers

¢  +Handlemul9pleconnec9onsconcurrently¢  +Cleansharingmodel

§  descriptors(no)§  filetables(yes)§  globalvariables(no)

¢  +SimpleandstraighZorward¢  –Addi9onaloverheadforprocesscontrol¢  –Nontrivialtosharedatabetweenprocesses

§  RequiresIPC(interprocesscommunica;on)mechanisms§  FIFO’s(namedpipes),SystemVsharedmemoryandsemaphores

Carnegie Mellon


Approach#2:Event-basedServers

¢  Servermaintainssetofac9veconnec9ons§  Arrayofconnfd’s

¢  Repeat:§  Determinewhichdescriptors(connfd’sorlistenfd)havependinginputs

§  e.g.,usingselectorepollfunc;ons§  arrivalofpendinginputisanevent

§  Iflistenfdhasinput,thenacceptconnec;on§  andaddnewconnfdtoarray

§  Serviceallconnfd’swithpendinginputs

¢  Detailsforselect-basedserverinbook

Carnegie Mellon


I/OMul9plexedEventProcessing

10

connfd’s

7 4 -1 -1 12 5 -1 -1 -1

0 1 2 3 4 5 6 7 8 9

Active

Inactive

Active

Never Used

listenfd = 3

10

connfd’s

7 4 -1 -1 12 5 -1 -1 -1

listenfd = 3 Ac9veDescriptors PendingInputs

Readandservice

Carnegie Mellon


ProsandConsofEvent-basedServers

¢  +Onelogicalcontrolflowandaddressspace.¢  +Cansingle-stepwithadebugger.¢  +Noprocessorthreadcontroloverhead.

§  Designofchoiceforhigh-performanceWebserversandsearchengines.e.g.,Node.js,nginx,Tornado

¢  –Significantlymorecomplextocodethanprocess-orthread-

baseddesigns.¢  –Hardtoprovidefine-grainedconcurrency

§  E.g.,howtodealwithpar;alHTTPrequestheaders¢  – Cannottakeadvantageofmul9-core

§  Singlethreadofcontrol

Carnegie Mellon


Approach#3:Thread-basedServers

¢  Verysimilartoapproach#1(process-based)§  …butusingthreadsinsteadofprocesses

Carnegie Mellon


Tradi9onalViewofaProcess

¢  Process=processcontext+code,data,andstack

Shared libraries

Run-time heap

0

Read/write data

Program context: Data registers Condition codes Stack pointer (SP) Program counter (PC)

Code, data, and stack

Read-only code/data

Stack SP

PC

brk

Process context

Kernel context: VM structures Descriptor table brk pointer

Carnegie Mellon


AlternateViewofaProcess

¢  Process=thread+code,data,andkernelcontext

Shared libraries

Run-time heap

0

Read/write data Thread context: Data registers Condition codes Stack pointer (SP) Program counter (PC)

Code, data, and kernel context

Read-only code/data

Stack SP

PC

brk

Thread (main thread)


Carnegie Mellon


AProcessWithMul9pleThreads¢  Mul9plethreadscanbeassociatedwithaprocess

§  Eachthreadhasitsownlogicalcontrolflow§  Eachthreadsharesthesamecode,data,andkernelcontext§  Eachthreadhasitsownstackforlocalvariables

§  butnotprotectedfromotherthreads§  Eachthreadhasitsownthreadid(TID)

Thread 1 context: Data registers Condition codes SP1 PC1

stack 1

Thread 1 (main thread)

shared libraries

run-time heap

0

read/write data

Shared code and data

read-only code/data


Thread 2 context: Data registers Condition codes SP2 PC2

stack 2

Thread 2 (peer thread)

Carnegie Mellon


LogicalViewofThreads

¢  Threadsassociatedwithprocessformapoolofpeers§  Unlikeprocesseswhichformatreehierarchy

P0

P1

sh sh sh

foo

bar

T1

Process hierarchy Threads associated with process foo

T2 T4

T5 T3

shared code, data and kernel context

Carnegie Mellon


ConcurrentThreads

¢  Twothreadsareconcurrentiftheirflowsoverlapin9me

¢  Otherwise,theyaresequen9al

¢  Examples:§  Concurrent:A&B,A&C§  Sequen;al:B&C

Time

Thread A Thread B Thread C

Carnegie Mellon


ConcurrentThreadExecu9on

¢  SingleCoreProcessor§  Simulateparallelismby;meslicing

¢  Mul9-CoreProcessor§  Canhavetrueparallelism

Time

Thread A Thread B Thread C Thread A Thread B Thread C

Run3threadson2cores

Carnegie Mellon


Threadsvs.Processes¢  Howthreadsandprocessesaresimilar

§  Eachhasitsownlogicalcontrolflow§  Eachcanrunconcurrentlywithothers(possiblyondifferentcores)§  Eachiscontextswitched

¢  Howthreadsandprocessesaredifferent§  Threadsshareallcodeanddata(exceptlocalstacks)

§  Processes(typically)donot§  Threadsaresomewhatlessexpensivethanprocesses

§  Processcontrol(crea;ngandreaping)twiceasexpensiveasthreadcontrol

§  Linuxnumbers:–  ~20Kcyclestocreateandreapaprocess–  ~10Kcycles(orless)tocreateandreapathread

Carnegie Mellon


PosixThreads(Pthreads)Interface¢  Pthreads:Standardinterfacefor~60func9onsthat

manipulatethreadsfromCprograms§  Crea;ngandreapingthreads

§  pthread_create()

§  pthread_join()

§  DeterminingyourthreadID§  pthread_self()

§  Termina;ngthreads§  pthread_cancel()

§  pthread_exit()§  exit()[terminatesallthreads],RET [terminatescurrentthread]

§  Synchronizingaccesstosharedvariables§  pthread_mutex_init

§  pthread_mutex_[un]lock

Carnegie Mellon


void *thread(void *vargp) /* thread routine */!{ ! printf("Hello, world!\n"); ! return NULL; !}

ThePthreads"hello,world"Program/* ! * hello.c - Pthreads "hello, world" program ! */!#include "csapp.h" !void *thread(void *vargp); !!int main() !{ ! pthread_t tid; ! Pthread_create(&tid, NULL, thread, NULL); ! Pthread_join(tid, NULL); ! exit(0); !} !

Thread attributes (usually NULL)

Thread arguments (void *p)

Return value (void **p)

hello.c

Thread ID

Thread routine

hello.c

Carnegie Mellon


Execu9onofThreaded“hello,world”Main thread

Peer thread

return NULL; Main thread waits for peer thread to terminate

exit() Terminates

main thread and any peer threads

call Pthread_create()

call Pthread_join()

Pthread_join() returns

printf()

Peer thread terminates

Pthread_create() returns

Carnegie Mellon


Thread-BasedConcurrentEchoServerint main(int argc, char **argv) !{ ! int listenfd, *connfdp; ! socklen_t clientlen; ! struct sockaddr_storage clientaddr; ! pthread_t tid; !! listenfd = Open_listenfd(argv[1]); ! while (1) { !

clientlen=sizeof(struct sockaddr_storage); !connfdp = Malloc(sizeof(int)); !*connfdp = Accept(listenfd, !

(SA *) &clientaddr, &clientlen); !Pthread_create(&tid, NULL, thread, connfdp); !

} !} ! echoservert.c

§  mallocofconnecteddescriptornecessarytoavoiddeadlyrace(later)

Carnegie Mellon


Thread-BasedConcurrentServer(cont)

/* Thread routine */!void *thread(void *vargp) !{ ! int connfd = *((int *)vargp); ! Pthread_detach(pthread_self()); ! Free(vargp); ! echo(connfd); ! Close(connfd); ! return NULL; !}

§  Runthreadin“detached”mode.§  Runsindependentlyofotherthreads§  Reapedautoma;cally(bykernel)whenitterminates

§  Freestorageallocatedtoholdconnfd.§  Closeconnfd(important!)

echoservert.c

Carnegie Mellon


Thread-basedServerExecu9onModel

§  Eachclienthandledbyindividualpeerthread§  ThreadsshareallprocessstateexceptTID§  Eachthreadhasaseparatestackforlocalvariables

Client 1 server peer

thread

Client 2 server peer

thread

Listening server

main thread

Connection requests

Client 1 data Client 2 data

Carnegie Mellon


IssuesWithThread-BasedServers

¢  Mustrun“detached”toavoidmemoryleak§  Atanypointin;me,athreadiseitherjoinableordetached§  Joinablethreadcanbereapedandkilledbyotherthreads

§  mustbereaped(withpthread_join)tofreememoryresources§  Detachedthreadcannotbereapedorkilledbyotherthreads

§  resourcesareautoma;callyreapedontermina;on§  Defaultstateisjoinable

§  usepthread_detach(pthread_self())tomakedetached

¢  Mustbecarefultoavoidunintendedsharing§  Forexample,passingpointertomainthread’sstack

§  Pthread_create(&tid, NULL, thread, (void *)&connfd);

¢  Allfunc9onscalledbyathreadmustbethread-safe§  (nextlecture)

Carnegie Mellon


ProsandConsofThread-BasedDesigns

¢  +Easytosharedatastructuresbetweenthreads§  e.g.,logginginforma;on,filecache

¢  +Threadsaremoreefficientthanprocesses

¢  –Uninten9onalsharingcanintroducesubtleandhard-to-reproduceerrors!§  Theeasewithwhichdatacanbesharedisboththegreateststrengthandthegreatestweaknessofthreads

§  Hardtoknowwhichdatashared&whichprivate§  Hardtodetectbytes;ng

§  Probabilityofbadraceoutcomeverylow§  Butnonzero!

§  Futurelectures

Carnegie Mellon


Summary:ApproachestoConcurrency

¢  Process-based§  Hardtoshareresources:Easytoavoidunintendedsharing§  Highoverheadinadding/removingclients

¢  Event-based§  Tediousandlowlevel§  Totalcontroloverscheduling§  Verylowoverhead§  Cannotcreateasfinegrainedalevelofconcurrency§  Doesnotmakeuseofmul;-core

¢  Thread-based§  Easytoshareresources:Perhapstooeasy§  Mediumoverhead§  Notmuchcontroloverschedulingpolicies§  Difficulttodebug

§  Eventorderingsnotrepeatable

Documents

Concurrent Programming - cs.cmu.edu file¢ Many aspects of concurrent programming are beyond the scope of our course