ParallelJava 2012 8 ConditionDeadlockWR · Serialversusconcurrent Sequential correctness is mostly concerned with safety properties: – ensuing that a program transforms each before-state

Deadlock, Reader-‐Writer problem and Condi6on synchroniza6on

Michelle Ku>el

Serial versus concurrent

Sequential correctness is mostly concerned with safety properties: –  ensuing that a program transforms each before-state to the

correct after-state.

Concurrent correctness is also concerned with safety, but the problem is much, much harder: –  safety must be ensured despite the vast number of ways

steps of concurrent threads can be be interleaved.

Also,concurrent correctness encompasses a variety of liveness properties that have no counterparts in the sequential world.

Concurrent correctness

There are two types of correctness proper6es:

Safety proper+es

The property must always be true.

Liveness proper+es

The property must eventually become true.

Java Deadlocks

We use locking to ensure safety •  but locks are inherently vulnerable to deadlock

•  indiscriminate locking can cause lock-‐ordering deadlocks

Dining philosophers

Classic problem used to illustrate deadlock –  proposed by Dijkstra in 1965

•  a table with five silent philosophers, five plates, five forks (or chops6cks) and a big bowl of spagheP (or rice).

•  Each philosopher must alternately think and eat.

•  Ea6ng is not limited by the amount of spagheP leR: assume an infinite supply.

•  However, a philosophers need two forks to eat

•  A fork is placed between each pair of adjacent philosophers.

unrealis6c, unsanitary and interes6ng

Dining philosophers

•  Basic philosopher loop: while True:! think()! get_forks()! eat()! put_forks()!

The problem is how to design a concurrent algorithm such that each philosopher won't starve, i.e. can forever con6nue to alternate between ea6ng and thinking.

•  Some algorithms result in some or all of the philosophers dying of hunger…. deadlock

Dining philosophers in Java class Philosopher extends Thread {!!int identity;!!Chopstick left; Chopstick right;!!Philosopher(Chopstick left,Chopstick right){!! !this.left = left; this.right = right;!!}!!public void run() {!! !while (true) {!! ! !try {!! ! ! !sleep(…); // thinking!! ! ! !right.get(); left.get(); // hungry!! ! ! !sleep(…) ; // eating!! ! ! !right.put(); left.put();!! ! !} catch (InterruptedException e) {}!! !}!!}!

}!

poten+al for deadlock

Chops6ck Monitor class Chopstick {!!Boolean taken=false;!!synchronized void put() {!! !taken=false;!! !notify();!!}!!synchronized void get() throws!! ! ! ! !InterruptedException!!{!! !while (taken) wait();!! !taken=true;!!}!

}!

Applet for diners

for (int i =0; i<N; ++I) // create Chopsticks!!stick[i] = new Chopstick();!

for (int i =0; i<N; ++i){ !// create Philosophers!!phil[i]=new Philosopher(stick[(i-1+N%N],stick[i]);!!phil[i].start();!

}!

Dining philosophers cont.

We can avoid deadlock by: •  controlling the number of philosophers (HOW?)

•  change the order in which the philosophers pick up forks. (HOW?)

Mo6va6ng Deadlock Issues Consider a method to transfer money between bank accounts

11 Sophomoric Parallelism &

Concurrency, Lecture 6

class BankAccount { … synchronized void withdraw(int amt) {…} synchronized void deposit(int amt) {…} synchronized void transferTo(int amt, BankAccount a) { this.withdraw(amt); a.deposit(amt); } }

No6ce during call to a.deposit, thread holds 2 locks

–  Need to inves6gate when this may be a problem

The Deadlock



acquire lock for x do withdraw from x

block on lock for y

acquire lock for y do withdraw from y

block on lock for x

Thread 1: x.transferTo(1,y)

Time

For simplicity, suppose x and y are sta6c fields holding accounts

Thread 2: y.transferTo(1,x)

Deadly embrace

Simplest form of deadlock: •  Thread A holds lock L while trying to acquire lock M, while thread B holds lock M while trying to acquire lock L.

Deadlock, in general

A deadlock occurs when there are threads T1, …, Tn such that:

•  For i=1,..,n-‐1, Ti is wai6ng for a resource held by T(i+1) •  Tn is wai6ng for a resource held by T1

In other words, there is a cycle of wai6ng –  Can formalize as a graph of dependencies

Deadlock avoidance in programming amounts to employing techniques to ensure a cycle can never arise

14 slide adapted from: Sophomoric Parallelism &


Deadlocks in Java

Java applica6ons do not recover from deadlocks: •  when a set of Java threads deadlock, they are permanently out of commission

•  applica6on may stall completely, a subsystem may stall, performance may suffer – …. all not good!

•  If there is poten6al for deadlock it may actually never happen, but usually does under worst possible condi6ons

so we need to ensure that it can’t happen

Back to our example Op6ons for deadlock-‐proof transfer:

1.  Make a smaller cri6cal sec6on: transferTo not synchronized –  Exposes intermediate state aRer withdraw before deposit –  May be okay, but exposes wrong total amount in bank

2.  Coarsen lock granularity: one lock for all accounts allowing transfers between them –  Works, but sacrifices concurrent deposits/withdrawals

3.  Give every bank-‐account a unique number and always acquire locks in the same order –  En/re program should obey this order to avoid cycles –  Code acquiring only one lock is fine



Ordering locks



class BankAccount { … private int acctNumber; // must be unique void transferTo(int amt, BankAccount a) { if(this.acctNumber < a.acctNumber) synchronized(this) { synchronized(a) { this.withdraw(amt); a.deposit(amt); }} else synchronized(a) { synchronized(this) { this.withdraw(amt); a.deposit(amt); }} } }

Lock-‐ordering deadlocks

•  occur when two threads a>empt to acquire the same locks in a different order

•  A program will be free of lock-‐ordering deadlocks if all threads acquire the locks they need in a fixed global order –  requires global analysis of your programs locking behaviour

•  A program than never acquires more than one lock at a +me will also never deadlock, but oRen imprac6cal

Another example From the Java standard library



class StringBuffer { private int count; private char[] value; … synchronized append(StringBuffer sb) { int len = sb.length(); if(this.count + len > this.value.length) this.expand(…); sb.getChars(0,len,this.value,this.count); } synchronized getChars(int x, int, y, char[] a, int z) { “copy this.value[x..y] into a starting at z” } }

Two problems Problem #1: The lock for sb is not held between calls to

sb.length and sb.getChars –  So sb could get longer –  Would cause append to throw an ArrayBoundsException

Problem #2: Deadlock poten6al if two threads try to append in opposite direc6ons, just like in the bank-‐account first example

Not easy to fix both problems without extra copying: –  Do not want unique ids on every StringBuffer –  Do not want one lock for all StringBuffer objects

Actual Java library: fixed neither (leR code as is; changed javadoc) –  Up to clients to avoid such situa6ons with own protocols



Perspec6ve

•  Code like account-‐transfer and string-‐buffer append are difficult to deal with for deadlock

•  Easier case: different types of objects –  Can document a fixed order among types –  Example: “When moving an item from the hashtable to the

work queue, never try to acquire the queue lock while holding the hashtable lock”

•  Easier case: objects are in an acyclic structure –  Can use the data structure to determine a fixed order –  Example: “If holding a tree node’s lock, do not acquire other

tree nodes’ locks unless they are children in the tree”



Why are Thread.suspend and Thread.resume deprecated?

Thread.suspend is inherently deadlock-‐prone. •  If the target thread holds a lock on the monitor protec6ng a cri6cal system resource when it is suspended, no thread can access this resource un6l the target thread is resumed.

•  If the thread that would resume the target thread a>empts to lock this monitor prior to calling resume, deadlock results.

•  Such deadlocks typically manifest themselves as "frozen" processes.

Checkpoint •  The BirdsSpo>ed2 class is thread safe. Is it also deadlock free?

public final class BirdsSpotted2 { private long CapeStarling = 0; private long SacredIbis = 0; private long CapeRobinChat = 0;

public synchronized long getStarling() { returnCapeStarling;} public synchronized long getIbis() { returnSacredIbis;} public synchronized long getRobin() { returnCapeRobinChat;}

public synchronized long spottedStarling() {return ++CapeStarling;} public synchronized long spottedIbis() { return ++SacredIbis;} public synchronized long spottedRobin() { return ++CapeRobinChat;} }!

Checkpoint

•  why can we have 2 separate locks here?

•  why is it desirable?

public class MsLunch { private long orc = 0; private long dragon = 0; private Object orcLock = new Object(); private Object dragonLock = new Object();

public void inc1() { synchronized(orcLock) { orc++; } }

public void inc2() { synchronized(dragonLock) { dragon++; } } }

Checkpoint

•  why can we have 2 separate locks here?

•  why is it desirable?

public class MsLunch { private long orc = 0; private long dragon = 0; private Object orcLock = new Object(); private Object dragonLock = new Object();

public void inc1() { synchronized(orcLock) { orc++; } }

public void inc2() { synchronized(dragonLock) { dragon++; } } }

Advantage of this using private lock: lock is encapsulated so client code

cannot acquire it –  clients incorrectly using lock can

cause liveness problems –  verifying that a publically

accessible lock is used properly requires examining the en6re program, compared to a single class for a private one

Progress Condi6ons

•  Deadlock-‐free: some thread trying to acquire the lock eventually succeeds.

•  Starva/on-‐free: every thread trying to acquire the lock eventually succeeds.

Art of Mul6processor Programming 26

Starva+on

•  much less common a problem than deadlock

•  situa6on where a thread is unable to gain regular access to shared resources and is unable to make progress. –  most commonly starved resource is CPU cycles

•  happens when shared resources are made unavailable for long periods by "greedy" threads.

•  For example: –  suppose an object provides a synchronized method that oRen

takes a long 6me to return. –  If one thread invokes this method frequently, other threads that

also need frequent synchronized access to the same object will oRen be blocked.

Starva6on

•  In Java can be caused by inappropriate use of thread priori6es

•  or indefinite loops or resource waits that do not terminate where a lock is held

Livelock •  A thread oRen acts in response to the ac6on of another thread.

–  If the other thread's ac6on is also a response to the ac6on of another thread, then livelock may result.

–  As with deadlock, livelocked threads are unable to make further progress.

•  Process is in a livelock if it is spinning while wai6ng for a condi6on that will never become true (busy wait deadlock)

•  comparable to two people a>emp6ng to pass each other in a corridor: Alphonse moves to his leR to let Gaston pass, while Gaston moves to his right to let Alphonse pass.

•  Seeing that they are s6ll blocking each other, Alphone moves to his right, while Gaston moves to his leR. They're s6ll blocking each other, so...

Readers/writer locks

30

Reading vs. wri6ng

Recall: –  Mul6ple concurrent reads of same memory: Not a problem –  Mul6ple concurrent writes of same memory: Problem –  Mul6ple concurrent read & write of same memory: Problem

So far: –  If concurrent write/write or read/write might occur, use

synchroniza6on to ensure one-‐thread-‐at-‐a-‐6me

But this is unnecessarily conserva6ve: –  Could s6ll allow mul6ple simultaneous readers!



Readers and writers problem

variant of the mutual exclusion problem where there are two classes of processes:

•  writers which need exclusive access to resources

•  readers which need not exclude each other

Readers/Writers •  Easy to solve with mutual exclusion •  But mutual exclusion requires wai6ng

– One waits for the other – Everyone executes sequen6ally

•  Performance hit!

Example

Consider a hashtable with one coarse-‐grained lock –  So only one thread can perform opera6ons at a 6me

But suppose: –  There are many simultaneous lookup opera6ons – insert opera6ons are very rare

Note: Important that lookup doesn’t actually mutate shared memory, like a move-‐to-‐front list opera6on would



Readers/writer locks A new synchroniza6on ADT: The readers/writer lock

•  A lock’s states fall into three categories: –  “not held” –  “held for wri6ng” by one thread –  “held for reading” by one or more threads

•  new: make a new lock, ini6ally “not held” •  acquire_write: block if currently “held for reading” or

“held for wri6ng”, else make “held for wri6ng” •  release_write: make “not held” •  acquire_read: block if currently “held for wri6ng”, else

make/keep “held for reading” and increment readers count •  release_read: decrement readers count, if 0, make “not

held”



0 ≤ writers ≤ 1 0 ≤ readers writers*readers==0

Pseudocode example (not Java)



class Hashtable<K,V> { … // coarse-grained, one lock for table RWLock lk = new RWLock(); V lookup(K key) { int bucket = hasher(key); lk.acquire_read(); … read array[bucket] … lk.release_read(); } void insert(K key, V val) { int bucket = hasher(key); lk.acquire_write(); … write array[bucket] …

lk.release_write(); } }

Readers/writer lock details

•  A readers/writer lock implementa6on (“not our problem”) usually gives priority to writers: –  Once a writer blocks, no readers arriving later will get the lock

before the writer –  Otherwise an insert could starve

•  Re-‐entrant? Mostly an orthogonal issue –  But some libraries support upgrading from reader to writer

•  Why not use readers/writer locks with more fine-‐grained locking, like on each bucket? –  Not wrong, but likely not worth it due to low conten6on



In Java

Java’s synchronized statement does not support readers/writer

Instead, library java.util.concurrent.locks.ReentrantReadWriteL

ock

•  Different interface: methods readLock and writeLock return objects that themselves have lock and unlock methods

•  Does not have writer priority or reader-‐to-‐writer upgrading –  Always read the documenta6on



Condi6on variables

39

Condi6on variables: Producer-‐Consumer synchroniza6on problem

In mul6threaded programs there is oRen a division of labor between threads.

•  In one common pa>ern, some threads are producers and some are consumers. – Producers create items of some kind and add them to a data structure;

– consumers remove the items and process them

•  a new coordina6on problem: Producer-‐Consumer

Producer-‐Consumer

canonical example of a bounded buffer for sharing work among threads

Bounded buffer: A queue with a fixed size –  (Unbounded s6ll needs a condi6on variable, but 1 instead of 2)

For sharing work – think an assembly line: –  Producer thread(s) do some work and enqueue result objects –  Consumer thread(s) dequeue objects and do next stage –  Must synchronize access to the queue



f e d c buffer

back front

producer(s) enqueue

consumer(s) dequeue

The little book of semaphores, by Downey 42

Producer-‐consumer problem

Event-‐driven programs are a good example. •  Whenever an event occurs, a producer thread creates an event object and adds it to the event buffer.

•  Concurrently, consumer threads take events out of the buffer and process them.

43

Producer-‐consumer problem

For this to work correctly: •  Producers must not produce when the buffer is full – must wait 6ll there is a gap.

•  Consumers must not consume when the buffer is empty – must wait 6ll it is filled.

Code, a>empt 1



class Buffer<E> { E[] array = (E[])new Object[SIZE]; … // front, back fields, isEmpty, isFull methods synchronized void enqueue(E elt) { if(isFull()) ??? else … add to array and adjust back … } synchronized E dequeue() if(isEmpty()) ??? else … take from array and adjust front … } }

Wai6ng •  enqueue to a full buffer should not raise an excep6on

–  Wait un6l there is room

•  dequeue from an empty buffer should not raise an excep6on –  Wait un6l there is data

Bad approach is to spin (wasted work and keep grabbing lock)



void enqueue(E elt) { while(true) { synchronized(this) { if(isFull()) continue; … add to array and adjust back … return; }}} // dequeue similar

What we want •  Be>er would be for a thread to wait un6l it can proceed

–  Be no/fied when it should try again –  In the mean6me, let other threads run

•  Like locks, not something you can implement on your own –  Language or library gives it to you, typically implemented with

opera6ng-‐system support

•  An ADT that supports this: condi6on variable –  Informs waiter(s) when the condi/on that causes it/them to wait

has varied

•  Terminology not completely standard; will mostly s6ck with Java



Java approach: not quite right



class Buffer<E> { … synchronized void enqueue(E elt) { if(isFull()) this.wait(); // releases lock and waits add to array and adjust back if(buffer was empty) this.notify(); // wake somebody up } synchronized E dequeue() { if(isEmpty()) this.wait(); // releases lock and waits take from array and adjust front if(buffer was full) this.notify(); // wake somebody up } }

Key ideas •  Java weirdness: every object “is” a condi6on variable (and

a lock) –  other languages/libraries oRen make them separate

•  wait: –  “register” running thread as interested in being woken up –  then atomically: release the lock and block –  when execu6on resumes, thread again holds the lock

•  notify: –  pick one wai6ng thread and wake it up –  no guarantee woken up thread runs next, just that it is no

longer blocked on the condi/on – now wai6ng for the lock –  if no thread is wai6ng, then do nothing



Bug #1

Between the 6me a thread is no6fied and it re-‐acquires the lock, the condi6on can become false again!



synchronized void enqueue(E elt){ if(isFull()) this.wait(); add to array and adjust back … }

if(isFull()) this.wait();

add to array

Time

Thread 2 (dequeue) Thread 1 (enqueue)

take from array if(was full)

this.notify();

make full again

Thread 3 (enqueue)

Bug fix #1

Guideline: Always re-‐check the condi6on aRer re-‐gaining the lock –  In fact, for obscure reasons, Java is technically allowed to no6fy a

thread spuriously (i.e., for no reason)



synchronized void enqueue(E elt) { while(isFull()) this.wait(); … } synchronized E dequeue() { while(isEmpty()) this.wait(); … }

Bug #2 •  If mul6ple threads are wai6ng, we wake up only one

–  Sure only one can do work now, but can’t forget the others!



while(isFull()) this.wait();

…

Time

Thread 2 (enqueue) Thread 1 (enqueue)

// dequeue #1 if(buffer was full) this.notify();

// dequeue #2 if(buffer was full) this.notify();

Thread 3 (dequeues) while(isFull()) this.wait();

…

Bug fix #2

notifyAll wakes up all current waiters on the condi6on variable

Guideline: If in any doubt, use notifyAll –  Wasteful waking is be>er than never waking up

•  So why does notify exist? –  Well, it is faster when correct…



synchronized void enqueue(E elt) { … if(buffer was empty) this.notifyAll(); // wake everybody up } synchronized E dequeue() { … if(buffer was full) this.notifyAll(); // wake everybody up }

A new liveness hazard: missed signals

•  A missed signal occurs when a thread must wait for a specific condi6on that is already true, but fails to check before wai6ng

•  no6fyAll is almost always be>er than no6fy, because it is less prone to missed signals

Alternate approach •  An alterna6ve is to call notify (not notifyAll) on every enqueue / dequeue, not just when the buffer was empty / full –  Easy: just remove the if statement

•  Alas, makes our code subtly wrong since it’s technically possible that an enqueue and a dequeue are both wai6ng. –  See notes for the step-‐by-‐step details of how this can happen

•  Works fine if buffer is unbounded since then only dequeuers wait



Alternate approach fixed •  The alternate approach works if the enqueuers and dequeuers wait

on different condi6on variables –  But for mutual exclusion both condi6on variables must be associated

with the same lock

•  Java’s “everything is a lock / condi6on variable” doesn’t support this: each condi6on variable is associated with itself

•  Instead, Java has classes in java.util.concurrent.locks for when you want mul6ple condi6ons with one lock –  class ReentrantLock has a method newCondition that

returns a new Condition object associate with the lock –  See the documenta6on if curious



Last condi6on-‐variable comments

•  notify/notifyAll oRen called signal/broadcast, also called pulse/pulseAll

•  Condi6on variables are subtle and harder to use than locks •  But when you need them, you need them

–  Spinning and other work-‐arounds don’t work well

•  Fortunately, like most things in a data-‐structures course, the common use-‐cases are provided in libraries wri>en by experts –  Example: java.util.concurrent.ArrayBlockingQueue<E>

–  All uses of condi6on variables hidden in the library; client just calls put and take



Condi6on synchroniza6on

Java has built-‐in mechanisms for wai6ng for a condi6on to become true: wait() and notify()!

They are tightly bound to intrinsic locking and can be difficult to use correctly

Often easier to use existing synchronizer classes: •  coordinate control flow of cooperating threads

e.g. BlockingQueue and Semaphore!

Java Blocking queues and the producer-‐consumer design paGern

•  BlockingQueue extends Queue with blocking inser6on and retrieval opera6ons – put and take methods – 6med equivalents: offer and poll!

•  If queue is empty, a retrieval (take) blocks un6l an element is available

•  If queue is full (for bounded queues), inser6on (put) blocks un6l there is space available

Producer-‐consumer design pa>ern

separates iden6fica6on of work to be done from execu6on of that work

•  work items are placed on “to do” list for later processing

•  removes code dependencies between producers and consumers

Most common design is a thread pool coupled with a work queue

Several implementa6ons of blocking queue

•  LinkedBlockingQueue, ArrayBlockingQueue: – FIFO queues

•  Priority blocking queue •  SynchronousQueue:

– queued THREADS

The Executor Framework and Thread Pools

•  usually the easiest way to implement a producer-‐consumer design is to use a thread pool implementa6on as part of the Executor framework public interface Executor {!!void execute(Runnable command);!}!

An Executor object typically creates and manages a group of threads called a thread pool

•  threads execute the Runnable objects passed to the execute method

Concurrency summary •  Access to shared resources introduces new kinds of

bugs –  Data races –  Cri6cal sec6ons too small –  Cri6cal sec6ons use wrong locks –  Deadlocks

•  Requires synchroniza6on –  Locks for mutual exclusion (common, various flavors) –  Condi6on variables for signaling others (less common)

•  Guidelines for correct use help avoid common pi{alls

•  Not clear shared-‐memory is worth the pain –  But other models (e.g., message passing) not a panacea



Java synchronizers

A synchronizer is any object that coordinates the control flow of threads based on its state. Java has: – Blocking Queues – Semphores – Barriers – Latches

Java synchronizers

All synchronizers: •  determine whether arriving threads should be allowed to pass or be forced to wait based on encapsulated state

•  provide methods to manipulate state

•  provide methods to wait efficiently for the synchronizers to enter the desired state

Latches

Acts as a gate: no thread can pass un6l the gate opens, and then all can pass

•  delays progress of threads un6l it enters terminal state

•  cannot then change state again (open forever) For example, can be used to wait un6l all par6es involved in an ac6vity are ready to proceed: –  like all players in a mul6-‐player game

CountDownLatch!

CountDownLatch allows one or more threads to wait for a set of events to occur

Latch state is a counter ini6alized to a posi6ve number, represen6ng number of elements to wait for

Semaphores

Coun+ng semaphores are used to control the number of ac6vi6es that can access a certain resource or perform a given ac6on at the same 6me

•  like a set of virtual permits – ac6vi6es can acquire permits and release then when they are done with them

•  Useful for implemen6ng resource pools, such as database connec6on pools.

Barriers

Similar to latches – block a group of threads un6l an event has occurred – but:

•  latches wait for events •  barriers wait for other threads

CyclicBarrier

Allows a fixed number of par6es to rendezvous repeatedly at a barrier point

Threads call await when they reach the barrier point and await blocks un6l all threads have reached the barrier point.

Once all threads are there, the barrier is passed, all threads are released and the barrier is reset.

CyclicBarrier

Useful in parallel itera6ve algorithms that break down a problem into a fixed number of independent subproblems:

•  In many simula6ons, the work done in one step can be done in parallel, but all work in one step must be completed before the next step begins…

Conway’s game of life Conway’s game of life is a cellular automaton first proposed by the

Bri6sh mathema6cian John Horton Conway in 1970. The game is a simula6on on a two-‐dimensional grid of cells. Each cell

starts off as either alive or dead. The state of the cell changes depending on the state of its 7 neighbours in the grid. At each 6me-‐step, we update the state of each cell according to the following four rules.

•  A live cell with fewer than two live neighbors dies due to underpopula6on.

•  A live cell with more than three live neighbors dies due to overpopula6on.

•  A live cell with two or three live neighbors survives to the next genera6on.

•  A dead cell with exactly three live neighbors becomes a live cell due to breeding.

Mul6threaded Conway’s game of life

Parallel program generates threads equal to the number of cells, •  or, be>er, a part of the grid -‐ and updates the status of each cell independently. •  Before proceeding to the next 6me step, it is necessary that all the

grids have been updated. •  This requirement can be ensured by using a global barrier for all

threads.

Causes of Efficiency Problems in Java

Too much locking • Cost of using synchronized • Cost of blocking wai6ng for locks • Cost of thread cache flushes and reloads Too many threads • Cost of star6ng up new threads • Cost of context switching and scheduling • Cost of inter-‐CPU communica6on, cache misses Too much coordina+on • Cost of guarded waits and no6fica6on messages • Cost of layered concurrency control Too many objects • Cost of using objects to represent state, messages, etc

Documents

ParallelJava 2012 8 ConditionDeadlockWR · Serial*versus*concurrent Sequential correctness is mostly concerned with safety properties: – ensuing that a program transforms each before-state

ParallelJava 2012 8 ConditionDeadlockWR · Serialversusconcurrent Sequential correctness is mostly concerned with safety properties: – ensuing that a program transforms each before-state