Upload
britton-perry
View
220
Download
2
Embed Size (px)
Citation preview
Concurrency in JavaConcurrency in Java
(Shooting yourself in the foot)n(Shooting yourself in the foot)n
AcknowledgmentAcknowledgment
This lecture is derived almost exclusively from
Java Concurrency in Practice by Goetz, et. al.http://
www.javaconcurrencyinpractice.com
Additional notes from Dr. Dan Wallach
This lecture is derived almost exclusively from
Java Concurrency in Practice by Goetz, et. al.http://
www.javaconcurrencyinpractice.com
Additional notes from Dr. Dan Wallach
OutlineOutline
BackgroundBasic Thread SafetyConcurrent Object
ManagementConcurrent Library
ComponentsTask Execution and
Shutdown
BackgroundBasic Thread SafetyConcurrent Object
ManagementConcurrent Library
ComponentsTask Execution and
Shutdown
Next TimeNext Time
Thread PoolsGUI ApplicationsSafety and PerformanceDocumentation and TestingAdv. Topics
Explicit LocksCustom SynchronizersNonblocking Synchronization
Thread PoolsGUI ApplicationsSafety and PerformanceDocumentation and TestingAdv. Topics
Explicit LocksCustom SynchronizersNonblocking Synchronization
BackgroundBackground
Why concurrency?Resource utilization - why
wait?Fairness - why wait?Convenience - why wait?
First concurrency: processes
Later: threads
Why concurrency?Resource utilization - why
wait?Fairness - why wait?Convenience - why wait?
First concurrency: processes
Later: threads
Benefits of ThreadsBenefits of Threads
Responsiveness (esp. GUIs)
Exploiting multi-processorsSimplicity of modelingSimplified handling of
asynchronous events
Responsiveness (esp. GUIs)
Exploiting multi-processorsSimplicity of modelingSimplified handling of
asynchronous events
Risks of ThreadsRisks of Threads
Java’s “built-in” threads means that concurrency is NOT an advanced topic
Safety hazards (correctness)Liveness hazards (progress)Performance hazards
(happiness)
Java’s “built-in” threads means that concurrency is NOT an advanced topic
Safety hazards (correctness)Liveness hazards (progress)Performance hazards
(happiness)
Threads are EverywhereThreads are Everywhere
Frameworks use threadsTimerServlets and JSPsRMISwing and AWT
You will use this one!
Frameworks use threadsTimerServlets and JSPsRMISwing and AWT
You will use this one!
Starting a Java Thread
Starting a Java Thread
public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new Thread(new HelloRunnable())).start(); }}public class HelloThread extends Thread { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new HelloThread()).start(); }}
OutlineOutline
BackgroundBasic Thread SafetyConcurrent Object
ManagementConcurrent Library
ComponentsTask Execution and Shutdown
BackgroundBasic Thread SafetyConcurrent Object
ManagementConcurrent Library
ComponentsTask Execution and Shutdown
SynchronizationSynchronization
Given a mutable variable vIf multiple threads access vAND if one can modify vALL must synchronize accessThis includes read access
Never attempt to ignore thisIf sync is broken, so is the
code (even if it passes tests)
Given a mutable variable vIf multiple threads access vAND if one can modify vALL must synchronize accessThis includes read access
Never attempt to ignore thisIf sync is broken, so is the
code (even if it passes tests)
Concurrent CorrectnessConcurrent Correctness
Fixing a broken “shared” varDon’t Share! OrMake the var immutable! OrSynchronize access
Better yet, design it rightEncapsulationImmutabilityClear specifications of
invariants
Fixing a broken “shared” varDon’t Share! OrMake the var immutable! OrSynchronize access
Better yet, design it rightEncapsulationImmutabilityClear specifications of
invariants
What is “Thread-Safe”?
What is “Thread-Safe”?
Definitions are vague and varyProblem: what is correctness?Thread Safe:
“Correct” in multi-threaded envOR, no more broken in a mult-
threaded environment then in a single-threaded one
Thread-safe classes encapsulate all synchronization
Definitions are vague and varyProblem: what is correctness?Thread Safe:
“Correct” in multi-threaded envOR, no more broken in a mult-
threaded environment then in a single-threaded one
Thread-safe classes encapsulate all synchronization
Example: Unsafe Counter
Example: Unsafe Counter
@NoThreadSafepublic class UnsafeSequence{ private int value; public int getNext() { return value++; }}
@NoThreadSafepublic class UnsafeSequence{ private int value; public int getNext() { return value++; }}
What’s Wrong with That?
What’s Wrong with That?
Invariant:getNext must return a
sequenceUnlucky execution timing:
Invariant:getNext must return a
sequenceUnlucky execution timing:
Value->9 9+1->10 Value=10
Value->9 9+1->10 Value=10
Race ConditionsRace Conditions
The problem is a race conditionDef: r.c. occurs when the
correctness of a computation depends on the relative timing of multiple threads by the runtime.
Often confused for data race
The problem is a race conditionDef: r.c. occurs when the
correctness of a computation depends on the relative timing of multiple threads by the runtime.
Often confused for data race
Achieving SafetyAchieving Safety
Go stateless!Use atomic operationsUse locking
Go stateless!Use atomic operationsUse locking
Using Atomics!(It’s not a treaty violation)
Using Atomics!(It’s not a treaty violation)
@ThreadSafepublic class UnsafeSequence{ private final AtomicLong value = new
AtomicLong(0); public long getNext() { return value.incrementAndGet(); }}
@ThreadSafepublic class UnsafeSequence{ private final AtomicLong value = new
AtomicLong(0); public long getNext() { return value.incrementAndGet(); }}
LockingLocking
Solved one mutable variable by making the var atomic
What if we have 2? Can we just make them both atomic?
NOT if they are dependentWe have to lock any
combined operations
Solved one mutable variable by making the var atomic
What if we have 2? Can we just make them both atomic?
NOT if they are dependentWe have to lock any
combined operations
ExampleExample
Suppose we have a dictionary that stores the last (key,value) pair
Setting the (k,v) pair must be locked. It is not enough to use an atomic var for k, and another one for v
Suppose we have a dictionary that stores the last (key,value) pair
Setting the (k,v) pair must be locked. It is not enough to use an atomic var for k, and another one for v
Intrinsic LocksIntrinsic Locks
Java provides built-in locksEach object is a lock, so is
each classMarked by synchronizedEnforces
AtomicityMemory visibility (more later)
Java provides built-in locksEach object is a lock, so is
each classMarked by synchronizedEnforces
AtomicityMemory visibility (more later)
ReentrancyReentrancy
Requesting a lock held by another causes a block
Intrinsic locks are reentrant
A thread that owns a lock, that requests the same lock, will succeed (good for inheritance!)
Requesting a lock held by another causes a block
Intrinsic locks are reentrant
A thread that owns a lock, that requests the same lock, will succeed (good for inheritance!)
Example LockingExample Locking
@ThreadSafePublic class LockSafe<K,V> { @GuardedBy(“this”) private K key; @GuardedBy(“this”) private V value;
public synchronized setKvPair(K k, V v) { this.key = k; this.value = v; }}
@ThreadSafePublic class LockSafe<K,V> { @GuardedBy(“this”) private K key; @GuardedBy(“this”) private V value;
public synchronized setKvPair(K k, V v) { this.key = k; this.value = v; }}
Locks: “Guarded By”Locks: “Guarded By”
“For each mutable state variable that may be accessed by more than one thread, all accesses to that variable must be performed with the same lock held. In this case, we say that the variable is guarded by that lock” (JCP 28)
“For each mutable state variable that may be accessed by more than one thread, all accesses to that variable must be performed with the same lock held. In this case, we say that the variable is guarded by that lock” (JCP 28)
Locks: Documentation
Locks: Documentation
“Every shared, mutable variable should be guarded by exactly one lock. Make it clear to maintainers which lock that is” - (JCP p28)
“Every shared, mutable variable should be guarded by exactly one lock. Make it clear to maintainers which lock that is” - (JCP p28)
Locks: Across Invariants
Locks: Across Invariants
“For every invariant that involves more that one variable, all the variables involved in that invariant must be guarded by the same lock” - (JCP p29)
“For every invariant that involves more that one variable, all the variables involved in that invariant must be guarded by the same lock” - (JCP p29)
Critical ConceptCritical Concept
Any mutable state that can be concurrently accessed, must be synchronized EVERYWHERE else in the program!
EXAMPLE: TimerTask (JCP p29)
Any mutable state that can be concurrently accessed, must be synchronized EVERYWHERE else in the program!
EXAMPLE: TimerTask (JCP p29)
Why Not Lock Everything?
Why Not Lock Everything?
Even if every method were synchronized, it doesn’t solve actions that combine those methods! (example: vector)
Also, possible liveness and/or performance problems
Poor concurrency example (JCP p30)
Even if every method were synchronized, it doesn’t solve actions that combine those methods! (example: vector)
Also, possible liveness and/or performance problems
Poor concurrency example (JCP p30)
Lock AdviceLock Advice
“Avoid holding locks during lengthy computations or operations at risk of not completing quickly such as network or console I/O” - (JCP p32)
“Avoid holding locks during lengthy computations or operations at risk of not completing quickly such as network or console I/O” - (JCP p32)
More SyntaxMore Syntax
Other use of synchronized
public somemethod() { synchronized(this) { // do stuff }}
Other use of synchronized
public somemethod() { synchronized(this) { // do stuff }}
OutlineOutline
BackgroundBasic Thread Safety
Concurrent Object Management
Concurrent Library ComponentsTask Execution and Shutdown
BackgroundBasic Thread Safety
Concurrent Object Management
Concurrent Library ComponentsTask Execution and Shutdown
Sharing ObjectsSharing Objects
Memory VisibilityShared object modification
should be visible to other threads
Without synchronization, this may not happen
Memory VisibilityShared object modification
should be visible to other threads
Without synchronization, this may not happen
More about VisibilityMore about Visibility
“In general, there is no guarantee that the reading thread will see a value written by another thread on a timely basis, or even at all” - (JCP p33)
“In general, there is no guarantee that the reading thread will see a value written by another thread on a timely basis, or even at all” - (JCP p33)
Public class NoVisibility { private static boolean ready; private static int number;
private static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield() System.out.println(number) } }
public static void main(String[] args) { new ReaderThread().start(); number = 42; ready = true; }}
NoVisibility :(NoVisibility :(
NoVisibility could loop forever!
NoVisibility could write 0 through reordering
Other badness and sadness
Don’t do this!
NoVisibility could loop forever!
NoVisibility could write 0 through reordering
Other badness and sadness
Don’t do this!
Synchronization Helps
Synchronization Helps
NoVisibility demonstrates stale data
Synchronization must be used on all shared variables, even just for reads
Enforces memory visibility
NoVisibility demonstrates stale data
Synchronization must be used on all shared variables, even just for reads
Enforces memory visibility
A Word about 64bitA Word about 64bit
When a thread reads a var w/o synchronization, at worst it is stale, but not random
Out-of-thin-air safetyOne exception: 64 bit
numbers (double and long)Read/write can be 2 32 bit
operations!
When a thread reads a var w/o synchronization, at worst it is stale, but not random
Out-of-thin-air safetyOne exception: 64 bit
numbers (double and long)Read/write can be 2 32 bit
operations!
Volatile VariablesVolatile Variables
Can declare a java var volatileVolatile ensures visibility, but not
locking.Volatile use can be fragile. Many
times, you shouldn’t use itGood uses of volatile
Ensure visibility of their own stateThat of the object they refer toOr indicate that an important event
has occurred (e.g. shutdown)
Can declare a java var volatileVolatile ensures visibility, but not
locking.Volatile use can be fragile. Many
times, you shouldn’t use itGood uses of volatile
Ensure visibility of their own stateThat of the object they refer toOr indicate that an important event
has occurred (e.g. shutdown)
Use Volatile ONLY when
Use Volatile ONLY when
Writes to the var do not depend on its current value or are only written by one thread
The var does not participate with invariants with other state vars
Locking is not required for any other reason when the var is being accessed
Writes to the var do not depend on its current value or are only written by one thread
The var does not participate with invariants with other state vars
Locking is not required for any other reason when the var is being accessed
Publication and Escape
Publication and Escape
Publishing an object - makes it available outside current scope
Many times, objects should not be published at allBreaks encapsulationNot fully constructed (!)
Object published when it should not have been is said to have escaped!
Publishing an object - makes it available outside current scope
Many times, objects should not be published at allBreaks encapsulationNot fully constructed (!)
Object published when it should not have been is said to have escaped!
Publication MethodsPublication Methods
Public static fieldsChained publication
Special Case: published inner class
Passing to an alien method
Public static fieldsChained publication
Special Case: published inner class
Passing to an alien method
How Escape Happens
How Escape Happens
All linked to poor designSemantic escapes:
Return ref instead of copyInadvertent chained publication
Syntactic escapes:Escaped this during construction
“Object not properly constructed”
All linked to poor designSemantic escapes:
Return ref instead of copyInadvertent chained publication
Syntactic escapes:Escaped this during construction
“Object not properly constructed”
This Escape ExampleThis Escape Example
Public class ThisEscape { // JCP p41 public ThisEscape(Eventsource source) { source.registerListener( new EventListener() { public void onEvent(Event e) { doSomething(e); } }); }}
Preventing EscapePreventing Escape
Thread ConfinementImmutabilitySafe Publication
Thread ConfinementImmutabilitySafe Publication
Thread ConfinementThread Confinement
Keep mutable vars confined to a single thread
Ad-hoc = enforced by impl.Stack Confinement
Local varsViolated by publication
ThreadLocalPer thread value holding
object
Keep mutable vars confined to a single thread
Ad-hoc = enforced by impl.Stack Confinement
Local varsViolated by publication
ThreadLocalPer thread value holding
object
Immutable ObjectsImmutable Objects
Always thread safeCan’t escape after
constructionDefinition
It’s state cannot be modified after construction
All fields are final*Properly constructed
Can have mutable variables
Always thread safeCan’t escape after
constructionDefinition
It’s state cannot be modified after construction
All fields are final*Properly constructed
Can have mutable variables
Final FieldsFinal Fields
Can’t be modifiedAND, have special semantics
in the Java Memory model (initialization safety)
Make all fields final by default“Mostly Immutable” is better
than mutable
Can’t be modifiedAND, have special semantics
in the Java Memory model (initialization safety)
Make all fields final by default“Mostly Immutable” is better
than mutable
Using Volatile Again!Using Volatile Again!
Volatile can be used to publish immutable objects
Still not locked, but can be safe depending on semantics
Example - JCP p49-50
Volatile can be used to publish immutable objects
Still not locked, but can be safe depending on semantics
Example - JCP p49-50
Improper PublicationImproper Publication
If synchronization is not used to publish a mutable object, the object is not properly published
Without proper publication, there are serious problems with stale data
If synchronization is not used to publish a mutable object, the object is not properly published
Without proper publication, there are serious problems with stale data
Safe PublicationSafe Publication
Object reference and object state must become visible at the same time!
Idioms:Initializing from static initializerStoring ref in volatile or
AtomicReferenceStoring ref in final field of
properly constructed objectStoring ref in a lock-guarded field
Object reference and object state must become visible at the same time!
Idioms:Initializing from static initializerStoring ref in volatile or
AtomicReferenceStoring ref in final field of
properly constructed objectStoring ref in a lock-guarded field
Good News!Good News!
Java Collections safe publication examples:Key or value in Hashtable,
synchronizedMap, or ConcurrentMapInsert in Vector,
CopyOnWriteArrayList, CopyOnWriteArraySet, synchronizedList, or synchronizedSet
Insert in BlockingQueue or ConcurrentLinkedQueue
Java Collections safe publication examples:Key or value in Hashtable,
synchronizedMap, or ConcurrentMapInsert in Vector,
CopyOnWriteArrayList, CopyOnWriteArraySet, synchronizedList, or synchronizedSet
Insert in BlockingQueue or ConcurrentLinkedQueue
More Good NewsMore Good News
Effectively Immutable Objects
Objects that are used as if they were immutable
Example: Date
Effectively Immutable Objects
Objects that are used as if they were immutable
Example: Date
Modification Vs Visibility
Modification Vs Visibility
Safe publication ensures visibility
Synchronization is required if the object can be modified post-publication
Safe publication ensures visibility
Synchronization is required if the object can be modified post-publication
Rules of Engagement
Rules of Engagement
Thread-Confined: no sharingShared read-only:
(effectively) immutable objectsShared thread-safe: internal
synchronizationGuarded: only access with the
associated guard
Thread-Confined: no sharingShared read-only:
(effectively) immutable objectsShared thread-safe: internal
synchronizationGuarded: only access with the
associated guard
Designing Thread-Safe Classes
Designing Thread-Safe Classes
Identify the variables that form the object’s state
Identify the invariants that contain the state variables
Establish a policy for managing concurrent access to the object’s state
Identify the variables that form the object’s state
Identify the invariants that contain the state variables
Establish a policy for managing concurrent access to the object’s state
Java Monitor PatternJava Monitor Pattern
Principle of instance confinement
Encapsulate all mutable state and guard it with intrinsic lock
Good coarse-grained locking
Principle of instance confinement
Encapsulate all mutable state and guard it with intrinsic lock
Good coarse-grained locking
Delegating Thread Safety
Delegating Thread Safety
Safety can often be delgated to an internally-used thread-safe object
Collections are especially useful
Only works for 1 state var, or multiple independent vars
Safety can often be delgated to an internally-used thread-safe object
Collections are especially useful
Only works for 1 state var, or multiple independent vars
Extending Thread-safe Classes
Extending Thread-safe Classes
BE CAREFUL! Extending means that the synchronization policy is distributed over multiple separately maintained classes!
Inheritance requires the base class to publish enough state
Wrappers require that the same lock is used
Composition is often less fragile
BE CAREFUL! Extending means that the synchronization policy is distributed over multiple separately maintained classes!
Inheritance requires the base class to publish enough state
Wrappers require that the same lock is used
Composition is often less fragile
Extension Examples:Extension Examples:
Inheritance (JCP p72)Wrapping (JCP p72-73)Composition (JCP p74)
Inheritance (JCP p72)Wrapping (JCP p72-73)Composition (JCP p74)
OutlineOutline
BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library
ComponentsTask Execution and Shutdown
BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library
ComponentsTask Execution and Shutdown
Synchronized Collections
Synchronized Collections
Are thread safe, butComposite actions require
additional locking for semantic correctness
Careless use of locking leads to poor performance (think iteration)
Are thread safe, butComposite actions require
additional locking for semantic correctness
Careless use of locking leads to poor performance (think iteration)
IteratorsIterators
The synchronized collections provide iterators
These iterators are fail-fastThrows
ConcurrentModificationException
The synchronized collections provide iterators
These iterators are fail-fastThrows
ConcurrentModificationException
Concurrent CollectionsConcurrent Collections
Designed for concurrent access
Improved performance with a few minor trade-offs:
Your assignment: go look up the docs on ConcurrentHashMap and CopyOnWriteArrayList and try them out!
Designed for concurrent access
Improved performance with a few minor trade-offs:
Your assignment: go look up the docs on ConcurrentHashMap and CopyOnWriteArrayList and try them out!
Blocking QueuesBlocking Queues
Support the producer-consumer pattern
Can be bounded or unbounded
Serial thread confinementTry it out!
Support the producer-consumer pattern
Can be bounded or unbounded
Serial thread confinementTry it out!
BlockingBlocking
Threads may blockWait for I/OWait for LockWait for Thread.sleepWait for computation
put and get from BlockingQueue block
Threads may blockWait for I/OWait for LockWait for Thread.sleepWait for computation
put and get from BlockingQueue block
InterruptionInterruption
interrupt() methodRequest a blocking thread
stopA blocking method should
throw an InterruptedExceptionIf you catch this
Propagate the exception ORRestore the interrupt
interrupt() methodRequest a blocking thread
stopA blocking method should
throw an InterruptedExceptionIf you catch this
Propagate the exception ORRestore the interrupt
Example:Example:
Try{someBlockingMethod();
} catch (InterruptedException) {
Thread.currentThread().interrupt();
}
Try{someBlockingMethod();
} catch (InterruptedException) {
Thread.currentThread().interrupt();
}
SynchronizersSynchronizers
Latches: one-time triggere.g. CountDownLatch
FutureTask: long-running resultHas a get() methodReturns value if doneBlocks until it is done if not
Latches: one-time triggere.g. CountDownLatch
FutureTask: long-running resultHas a get() methodReturns value if doneBlocks until it is done if not
More SynchronizersMore Synchronizers
Semaphore: virtual permitsNow a library class!
Barriers: wait for enough threadsUseful for threaded steppingExchanger 2-party barrier
Semaphore: virtual permitsNow a library class!
Barriers: wait for enough threadsUseful for threaded steppingExchanger 2-party barrier
OutlineOutline
BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library ComponentsTask Execution and
Shutdown
BackgroundBasic Thread SafetyConcurrent Object ManagementConcurrent Library ComponentsTask Execution and
Shutdown
Advanced ExecutionAdvanced Execution
The Executor FrameworkDecouples task submission
and task executionSimple interface:
void execute(Runnable command)
Easiest way to implement a producer-consumer design in an application
The Executor FrameworkDecouples task submission
and task executionSimple interface:
void execute(Runnable command)
Easiest way to implement a producer-consumer design in an application
Execution PoliciesExecution Policies
AnswersIn what thread will tasks be
runWhat order should they be runHow many concurrently?How many queued?“Victim” selectionSetup and Teardown
AnswersIn what thread will tasks be
runWhat order should they be runHow many concurrently?How many queued?“Victim” selectionSetup and Teardown
Policy ExamplesPolicy Examples
Single-thread (JCP p119)1-thread-per-client (JCP
p118)Thread Pools: some built in
newFixedThreadPoolnewCachedThreadPoolnewSingleThreadExecutornewScheduleThreadPool
Single-thread (JCP p119)1-thread-per-client (JCP
p118)Thread Pools: some built in
newFixedThreadPoolnewCachedThreadPoolnewSingleThreadExecutornewScheduleThreadPool
Cancellation & Shutdown
Cancellation & Shutdown
Reasons for cancellationUser-requestedTime-limitedEventsErrorsShutdown
Reasons for cancellationUser-requestedTime-limitedEventsErrorsShutdown
Cancellation PolicyCancellation Policy
“how” “when” and “what” of cancellation
Not enforced by Java, but generally, use interruption for cancellation
E.g. have a cancel() method call the interrupt
“how” “when” and “what” of cancellation
Not enforced by Java, but generally, use interruption for cancellation
E.g. have a cancel() method call the interrupt
Interruption PolicyInterruption Policy
Determines how a thread interprets an interruption request
Tasks are “guests” in a threadPreserve interruption status
Only call interrupt, when you know the interruption policy
Determines how a thread interprets an interruption request
Tasks are “guests” in a threadPreserve interruption status
Only call interrupt, when you know the interruption policy
Detecting InterruptsDetecting Interrupts
Polling the thread’s interrupted status
Detecting an InterruptedException
Polling the thread’s interrupted status
Detecting an InterruptedException
Non-interruptible Blocks
Non-interruptible Blocks
Synchronous socket I/O:Close the socket
Asynchronous I/O with Selector:Call wakeup
Lock Acquisition:Use the explicit Lock classHas a lockInterruptibly() method
Synchronous socket I/O:Close the socket
Asynchronous I/O with Selector:Call wakeup
Lock Acquisition:Use the explicit Lock classHas a lockInterruptibly() method
Stopping a Thread-Based Service
Stopping a Thread-Based Service
ExceutorService extends Executor
Provides “Lifecycle” optionsshutdown()awaitTermination(timeout, unit)
Any thread based service should provide similar methods
ExceutorService extends Executor
Provides “Lifecycle” optionsshutdown()awaitTermination(timeout, unit)
Any thread based service should provide similar methods
Waiting for ThreadsWaiting for Threads
In the “raw” case, use join()A number of Java library
classes have advanced “waits” like ExecutionService’s awaitTermination()
Obviously, if you implement your own ExecutionService, you’ll have to use the raw stuff
In the “raw” case, use join()A number of Java library
classes have advanced “waits” like ExecutionService’s awaitTermination()
Obviously, if you implement your own ExecutionService, you’ll have to use the raw stuff