Concurrent Programming - cs.princeton.edu · • CGI programming – When HTTP server receives HTTP request: • HTTP server forks a child process • Child process execs (overwrites

Concurrent Programming

Copyright © 2017 by Robert M. Dondero, Ph.D.

Princeton University

1

Objectives •  You will learn/review:

– What a process is – How to fork and wait for processes – What a thread is – How to spawn and join threads – How to handle threads that access shared objects

•  Race conditions and prevention of them •  Deadlocks and prevention of them

– How processes can communicate – How threads can communicate

2

Agenda

1.  Process-level concurrency (C) 2.  Thread-level concurrency (Java/Python) 3.  Race conditions (Java/Python) 4.  Deadlock (Java/Python) 5.  Inter-process comm (C) 6.  Inter-thread comm (Java/Python)

3

Processes

•  Program – Executable code

•  Process – An instance of a program in execution – A process has its own distinct context

4

Process Context

•  Context consists of: – Process id – Address space: TEXT, RODATA, DATA, BSS,

HEAP, STACK – Processor state: RIP, EFLAGS, RAX, RBX,

etc. registers

5

Concurrency Options

•  To implement concurrency... •  Option 1: Multiple processes

– Process P1 forks process P2 – P1 and P2 execute concurrently – P1 and P2 do not share objects

•  P1 and P2 have (initially identical but) distinct memory spaces

–  (Relatively) expensive context switching

6

STACK HEAP

DATA

BSS

STACK HEAP

DATA

BSS

RODATA

TEXT

PROCESS 1 PROCESS 2

IP REG IP REG

Concurrent Processes Running Same Program

Concurrent Processes

7

Process-Level Concurrency

•  At system boot-up... – Several processes are created automatically – They run concurrently

•  When you login... – ssh process forks a child process – Child process execs (overwrites itself with)

your login shell (e.g. bash) – ssh process and your login shell process run

concurrently

8

Process-Level Concurrency •  When you execute your program as a

command... – Bash forks a child process – Child process execs (overwrites itself with) your

program – Bash process and your process run concurrently

•  Your process can fork/exec additional processes as desired – Your process and child processes run

concurrently

9

Process-Level Concurrency

•  COS 333 example… •  CGI programming

– When HTTP server receives HTTP request: •  HTTP server forks a child process •  Child process execs (overwrites itself with) the

specified CGI program •  Parent HTTP server process and CGI process run

concurrently

10

Forking Processes

•  How to fork child processes? •  See forking.c

– Parent process forks “blue” child process and “red” child process

– Three processes execute concurrently – Parent process may exit before child

processes •  Malformed!!! •  A parent should wait for its children to exit •  A parent should reap its children

11

Waiting for Processes

•  See waiting.c – Parent waits for (reaps) its children – Parent process is “blocked” until both children

exit – Proper pattern

12

Agenda

1.  Process-level concurrency (C) 2.  Thread-level concurrency (Java/

Python) 3.  Race conditions (Java/Python) 4.  Deadlock (Java/Python) 5.  Inter-process comm (C) 6.  Inter-thread comm (Java/Python)

13

Threads

•  Thread – A flow of control within a process – A process contains one or more threads – Within a process, all threads execute

concurrently

14

Concurrency Options

•  To implement concurrency… •  Option 2: Multiple threads

– Within P1, thread T1 spawns thread T2 – T1 and T2 execute concurrently – T1 and T2 may share some of P1’s objects –  (Relatively) inexpensive context switching

15

STACK

HEAP

DATA

BSS

RODATA

TEXT

THREAD 1 THREAD 2

IP REG

Concurrent Threads within Same Process

STACK

IP REG

Concurrent Threads

16

Thread-Level Concurrency

•  COS 333 examples… •  Java virtual machine process

– JVM process must execute given program – JVM process also must collect garbage – Main thread and garbage collector thread run

concurrently

17

Thread-Level Concurrency •  COS 333 examples… •  Assignment 2 server process

– Server process must communicate with client process for (relatively) long time periods

– Server process also must respond to other client processes

– Server process spawns distinct thread for each client request

– Main thread and “client handler” thread run concurrently

18

Thread-Level Concurrency

•  COS 333 examples… •  Assignment 4 browser process

– Browser process must fetch data – Browser process also must respond to user

input – Via AJAX, Browser main (UI) thread spawns a

child thread to fetch data – Browser main (UI) thread and “fetch data”

thread run concurrently

19

Spawning Threads

•  The “main” thread runs at process startup – Other threads may run at process startup too

•  The main thread can spawn other threads •  Note terminology:

– One process “forks” another – One thread “spawns” another

20

Spawning Threads in Java

•  See Spawning.java – To spawn a thread:

•  Define a subclass of Thread •  Override run() method •  Create an object of that class

– To begin execution of a thread: •  Call object’s start() method •  start() calls run() •  Don’t call run() directly

21


•  See RunnableInterface.java – Alternative way to spawn a thread:

•  Define class that implements Runnable •  Must define a run() method •  Create a Thread object specifying Runnable

object as argument

22


– To begin execution: •  Call Runnable object’s start() method •  start() calls run() •  Don’t call run() directly

– Useful when class extends some class other than Thread

23

Spawning Threads in Python

•  See spawning.py –  (Much the same as Java) – No need for “Runnable interface” approach

•  Python supports multiple inheritance

24

Joining Threads

•  Main thread can join a child thread – Main thread can block until child thread

terminates •  Terminology

– A parent process can “wait” for a child process

– A parent thread can “join” a child thread

25

Joining Threads in Java

•  See Joining.java – thread.join()

•  Blocks current thread until thread terminates •  May throw InterruptedException •  (Explained in Appendix)

26

Joining Threads in Python

•  See joining.py –  (Much the same)

27

Agenda


28

Race Conditions

•  Problem: – Threads can share objects – Danger if multiple threads update same object – Race condition

•  Outcome depends upon thread scheduling

•  See RaceCondition.java •  See racecondition.py

29

Locking

•  Solution: Locking – Current thread gets lock on shared object

•  Thereby current thread has exclusive use of shared object

– Other threads cannot obtain lock on shared object until current thread releases lock

– Adds lots of overhead

30

User-Level Locking

•  Approach 1: Locking in user of shared object

31

•  See LockInUser.java

– Current thread: “Give me the lock on sharedobj”

– Other threads cannot obtain the lock on sharedobj during execution of block

synchronized(sharedobj) { stmt; stmt; …}

User-Level Locking in Java

32

User-Level Locking in Python

•  See lockinuser.py – self._lock = Lock()

•  In BankAcct constructor •  Creates a lock for the newly instantiated object

– self._bankAcct.getLock().acquire() •  In each thread •  “Give me the lock on the bankAcct object”

– self._bankAcct.getLock().release() •  In each thread •  “Release the lock on the bankAcct object”

33

Resource-Level Locking

•  Approach 2: Locking in shared resource/object itself

34

•  See LockInResource.java

•  Is the same as:

•  Each BankAcct object thus is thread-safe

public synchronized void method() {…}

public void method() { synchronized(this) {…} }

Java Resource-Level Locking

35

Python Resource-Level Locking

•  See lockinresource.py – Same, except... – Locking is performed by BankAcct object

rather than by thread objects – Each BankAcct object thus is thread-safe

36

Locking Strategies

•  Which locking approach is better? •  Resource-level locking

– Generally safer; shared object is thread-safe •  User-level locking

– Generally faster; avoids locking when unnecessary

– Maybe easier to understand?

37

Aside: Thread-Safe Collections

38

Vector myVector = new Vector(); Hashmap myMap = new Hashmap(); List<E> myList = new Collections.synchronizedList(new ArrayList<E>()); Map<K,V> myMap = new Collections.synchronizedMap(new HashMap<K,V>()); ConcurrentLinkedQueue<E> myQueue = new ConcurrentLinkedQueue<E>(); ConcurrentSkipListSet<E> mySet = new ConcurrentSkipListSet<E>(); ConcurrentHashMap<K,V> myMap = new ConcurrentHashMap<K,V>(); ConcurrentSkipListMap<K,V> myMap = new ConcurrentSkipListMap<K,V>();

Thread-safe collections in Java:

Aside: Thread-Safe Collections

39

from Queue import Queue myQueue = Queue()

Thread-safe collections in Python:

Conditions

•  Problem: – Thread may need to wait for some condition

on a locked object to become true – Example:

•  Withdraw thread must wait for bank account balance to be sufficiently large

•  Solution: Conditions

40

Conditions

•  Called from within locked code… •  object.wait()

– Blocks current thread until it is notified – Releases the lock

•  object.notifyAll() – Notifies all waiting threads

•  That some significant event has occurred

– Thread then should re-check condition

41

consumer() while (! condition) wait(); // Do what should be done when // condition is true. producer() // Change condition. notifyAll();

Thread conditions pattern:

Conditions Pattern

42

Java Conditions

•  See Conditions.java – Consumer method calls object.wait()

repeatedly until condition is true – Producer method calls object.notifyAll()

– Could handle conditions in users rather than in shared resource

43

Python Conditions

•  See conditions.py – Condition wraps around lock – Could handle conditions in users rather than

in shared resource

44

Agenda


45

Deadlock

•  Problem: deadlock – Two threads: t1 and t2 – Two shared objects: o1 and o2 –  t1 has lock on o1; needs lock on o2 –  t2 has lock on o2; needs lock on o1 – Both threads block forever

46

(1) run() (2) run()

(3) aliceAcct.transferOneTo(bobAcct) (4) bobAcct.transferOneTo(aliceAcct)

(5) bobAcct.depositOne() (6) aliceAcct.depositOne()

Deadlock

transferThread1 transferThread2

aliceAcct bobAcct

Java Deadlock Example See Deadlock.java

47

Python Deadlock Example

•  See deadlock.py –  (Same pattern)

48

Preventing Deadlock

•  Conditions for deadlock – Mutual exclusion – Hold and wait – No preemption – Circular chain of events

•  Solution – OS level: Negate one of the four conditions – App level: Negate “circular chain of events”…

49

(1) run() (2) run()

(4) alictAcct.transferOneTo(bobAcct) (8) bobAcct.transferOneTo(aliceAcct) (5) bobAcct.withdrawOne() (9) aliceAcct.withdrawOne()

(3) get lock (6) release lock

NoDeadlock

Mutex

transferThread1 transferThread2

Preventing Deadlock

50

(7) get lock (10) release lock

aliceAcct bobAcct

Java Preventing Deadlock

•  See NoDeadlock1.java – Mutex object

•  See NoDeadlock2.java – Static fields and static methods – Mutex object as a static field in BankAcct class

•  See NoDeadlock3.java – Really no need for Mutex object! – Any object will suffice

•  E.g., the object which represents the BankAcct class!

51

Python Preventing Deadlock

•  See nodeadlock1.py – Mutex object

•  See nodeadlock2.py – Static fields and static methods – Mutex object as a static field in BankAcct

class

52

Agenda


53

Inter-Process Comm

•  Facts: – Processes on different computers can

communicate via sockets •  See Network lecture

– Processes on the same computer can communicate via pipes

•  A Unix pipe is a queue for inter-process comm

54

Inter-Process Comm

•  See childtochild.c – Parent process creates “producer” and

“consumer” child processes – Producer child process writes data to pipe – Consumer child process reads data from pipe

55

Inter-Process Comm

•  See sortmore.c – Parent process forks two child processes – First child executes sort somefile

•  With stdout redirected to pipe – Second child executes more

•  With stdin redirected to pipe

56

Inter-Process Comm

•  Generalization – Unix users routinely command the shell to fork

communicating concurrent processes •  E.g. sort somefile | more

– sortmore.c is a hardcoded shell

57

Agenda


58

Inter-Thread Comm

•  Facts: – Threads can communicate via shared objects – Communication via shared objects is

dangerous (race conditions, deadlock) – Often threads are in a producer/consumer

relationship •  Is there a safer way for producer/

consumer threads to communicate?

59

Java Inter-Thread Comm

•  See ProdConStream.java – PipedOutputStream and PipedInputStream •  Allow producer thread to send data to consumer

thread – See also java.io.PipedWriter and PipedReader for sending Unicode characters

60

Python Inter-Thread Comm

•  See prodconstream.py – No “stream” mechanism

•  Can’t implement Unix pipes model •  Can’t implement Java stream model •  Instead...

– Queue class •  Methods are “synchronized” •  Producer thread “puts” to queue object •  Consumer thread “gets” from queue object

61

Concurrency Commentary

•  Process-level concurrency is: – Essential (Unix relies on it) – Safe (distinct processes share no data)

•  Thread-level concurrency is: – Essential (garbage collection, network

pgmming, web pgmming, …) – Dangerous!!! (distinct threads can share

objects)

62

Concurrency Commentary

•  Should methods be “synchronized” by default?

•  When using thread-level concurrency, should we avoid shared objects?

•  Should we use process-level concurrency instead of thread-level concurrency whenever possible?

•  In the long run, is thread-level concurrency a passing phase?

63

Summary •  We have covered:

– What a process is – How to fork and wait for processes – What a thread is – How to spawn and join threads – How to handle threads that access shared objects

•  Race conditions and prevention of them •  Deadlocks and prevention of them

– How processes can communicate – How threads can communicate

64

Appendix 1: Interrupting Threads

65

Interrupting Threads

•  Bad idea to end a thread abnormally – May leave shared objects in inconsistent

states •  How to force normal thread termination?

– That is, how to force normal return from run()?

66


•  Forcing normal thread termination – Define interrupted flag in thread object

•  Initially set to false

– Define public thread.interrupt() method •  Called (by main thread?) to set interrupted flag

to true – run() checks interrupted flag periodically

•  true => return from run()

67


•  Problem –  If thread is joining/waiting/sleeping, then… – Thread might not check flag for a long time

•  Solution –  Interrupting threads…

68

Java Interrupting Threads •  b = thread.interrupted()

– Returns the value (True or False) of thread’s interrupted flag

•  thread.interrupt() – Sets thread’s interrupted flag –  If thread is joining, waiting, or sleeping...

•  join(), wait(), or sleep() throws InterruptedException

•  Thread.interrupted() – Checks interrupted flag of current thread

69

Java Interrupting Threads

•  Suppose… – Main thread spawns thread t1 – Main thread later calls t1.interrupt() – What happens?

70

if t1 is running, then t1.interrupt() sets t1’s interrupted flag to True t1.run() calls b = t1.interrupted() at leading edge of its loop b is True => t1.run() returns, and thread terminates else // t1 is blocked (for joining, waiting, or sleeping) join(), wait(), or sleep() throws an InterruptedException t1 catches the InterruptedException Catch clause re-calls t1.interrupt()


71

try { Thread.sleep(1000); } catch (InterruptedException e) { interrupt(); }

Pattern in thread object:

try { Thread.sleep(1000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }

Pattern in non-thread object:

public void run() { … while (…) { if (interrupted()) return; … } }

Pattern in thread object:

•  See Interrupting.java


72

Python Interrupting Threads

•  Not supported!!! •  Threads cannot be controlled “from the

outside” •  (Java is particularly good for multi-

threaded programming)

73

Appendix 2: Daemon Threads

74

Daemon Threads

•  User thread (as seen so far) – Example: thread that calls main() – Process exits when all user threads have

terminated •  Daemon thread

– Example: garbage collector thread – Exists only to service other threads – Process can terminate with running daemon

threads

75

Java Daemon Threads

•  thread.setDaemon(b) – Sets daemon status of thread to b (true or false)

•  b = thread.isDaemon() – Returns daemon status of thread

•  See Daemons.java – Daemon threads end when main thread ends

76

Python Daemon Threads

•  thread.setDaemon(b) – Sets daemon status of thread to b (True or False)

•  b = thread.isDaemon() – Returns daemon status of thread

•  See daemons.py – Daemon threads end when main thread ends

77

Appendix 3: Threads in C

78

Threads in C

•  C language does not support threads •  C standard library does not support

threads •  Use non-standard “pthreads” library

– Not object-oriented; uses function pointers – Specify “-pthread” option to gcc command

79

Threads in C

•  An error-handling module – See mypthread.h – See mypthread.c

•  Spawning threads – See spawning.c

80

Threads in C

•  Joining threads – See joining.c

•  Locking – See lockinuser.c – See lockinresource.c

•  Conditions – See conditions.c

81

Threads in C

•  Deadlock – See deadlock.c – See nodeadlock.c

•  Interrupting –  (Not available in C)

•  Inter-thread comm –  (Omitted)

82

Documents

Concurrent Programming - cs.princeton.edu · • CGI programming – When HTTP server receives HTTP request: • HTTP server forks a child process • Child process execs (overwrites