1

Thread Synchronization (Un)Fairness in the .NET

Common Language Runtime

If you are building an application that absolutely requires fair thread synchronization, you should not use the .NET Framework at all.“

Jeffrey Richter, In his article "Thread Synchronization Fairness in the .NET CLR”, published 6 February, 2003)

2

What? ... Why?The CLR manages memory via garbage collection. When the CLR wants to start a garbage collection, … the CLR will suspend the threads executing managed code.

When Windows suspends a thread, it stops the thread from waiting for any thread synchronization object. Later, when the thread is resumed, all the suspended threads race back to wait on the object that it was waiting on before it got suspended.

This means that threads are not guaranteed to gain ownership of an object on a first-in-first-out basis.

Jeffrey Richter, In his article "Thread Synchronization Fairness in the .NET CLR”, published 6 February, 2003)

3

So what is this presentation about?

Thread Synchronization Garbage Collection in .NET managed

application. How these seemingly unrelated areas

conspire to cause unfair thread synchronization in the .NET CLR.

4

First, Thread Synchronization:

Thread Synch support exposed in the underlying OS (and used by the .NET CLR)

OS Kernel Support for Thread Synch Thread Synch in the CLR

5

Managed execution in the common language runtime (CLR)

Figure 2-1, Introducing Microsoft .NET, Second Edition by David S. Platt

6

Thread Synch Support in Win2k User Mode

User Mode – No transition to Kernel Mode* Interlocked Functions – Set simple 32 bit values

atomically. Critical Sections – Data structure owned by one

thread at a time to synchronize access to common code. Can be used to manipulate complex data structures in a thread safe manner.

* When entering a critical section, the thread stays in user mode only as long as there is no contention. If the critical section is owned by another thread, the thread will transition to kernel mode and enter a wait state.

7

Thread Synch Support in Win2k Kernel Mode

Provided Primarily through the Win32 subsystem Wait functions (i.e. WaitForSingleObject)

A thread calls a Wait function to place itself in a wait state until a kernel object is signaled.

8

Waiting on a dispatcher object Figure 3-19, Inside Microsoft Windows 2000, Third Edition by David

A. Solomon and Mark E. Russinovich 

9

Object Type Set to Signaled State When Effect on Waiting Threads

Mutex Thread releases the mutex One thread released

Semaphore Semaphore count drops by 1 One thread released

Event (synchronization type)

Thread sets the event One thread released; event object reset

Event (notification type)

Thread sets the event All released

Timer (synchronization type)

Set time arrives or time interval expires

One thread released

Timer (notification type)

Set time arrives or time interval expires

All released

Process Last thread terminates All released

Thread Thread terminates All released

File I/O operation completes All released

Queue Item is placed on queue One thread released

Definitions of the Signaled State Table 3-9: Inside Microsoft Windows 2000, Third Edition by David A. Solomon and Mark E.

Russinovich 

10

Running in the CLR

Loading and initializing the CLR Figure 1-3 , Applied Microsoft .NET Framework Programming by Jeffrey Richter 

11

Threads in a Managed Process

Each application thread in a managed application has 1:1 correspondence with an OS thread.

The CLR will create additional threads for its own purposes: Finalizer, Debug, Threadpool, Asynch Ops (Timers, Delegates)

All thread creation, dispatch and execution is done in the OS. The CLR itself provides none of this functionality.

12

Thread Synch Classes in .NETClass Purpose OS Parallel

I nterlocked Provides atomic operations for variables that are shared by multiple threads.

Interlocked Functions

Monitor Provides a mechanism that synchronizes access to objects. Visual Basic .NET applications call SyncLock to use monitor objects.

Critical Section

WaitHandle Encapsulates operating system-specific objects

that wait for exclusive access to shared resources. Dispatch Object

AutoResetEvent A wait handle that notifies one or more waiting threads that an event has occurred. AutoResetEvent automatically changes status to signaled when a waiting thread is released.

Event (synchronization type)

ManualResetEvent A wait handle that notifies one or more waiting threads that an event has occurred. The state of a manually reset event remains signaled until the Reset method sets it to the nonsignaled state. Similarly, the state remains nonsignaled until the Set method sets it to the signaled state. Any number of waiting threads, or threads that subsequently begin wait operations for the specified event object by calling one of the wait functions, can be released while the object's state is signaled.

Event (notification type)

Mutex A wait handle that can be used for interprocess synchronization.

Mutex

Adaptation of Table in Robert Burns, Multithreaded Programming with Visual Basic .NET, MSDN Online Library, February 2002

13

The other piece of the puzzle …

Automatic Memory Management…

a.k.a. Garbage Collection

What is it and why do we use it?

14

Traditional Manual Memory Management (e.g. C-Runtime)

Memory Allocation (malloc): User code requests a certain number of bytes. CRT walks linked list to find large enough memory

block then updates list.

Memory Deallocation (free): CRT updates linked list to free up the memory block.

15

Automatic Memory Management (e.g. Garbage Collection)

CLR reserves contiguous block of memory when process initialized, the managed heap. Pointer maintained to next free location.

Memory Allocation (newobj IL instruction): CLR determines memory required based on object metadata. If enough memory available, the CLR returns a pointer to next

free location, and updates the pointer to the next free location. If not enough memory in the heap, a garbage collection (details

later).

Memory Deallocation: Memory is never explicitly deallocated, and is not freed until the

next garbage collection is done.

16

Memory Management ComparisonManual Memory Mgt Automatic Memory Mgt

Adv More power to programmer Simple and fast allocation

Predictable time delay Good Object Locality

Deterministic Finalization

Disadv More power to programmer Not predictable – usually short / sometimes (unpredictably) long

Introduces global dependencies Non-deterministic finalization

Most allocations slower User has less power

Less object locality

Memory leaks

Early deallocation – memory corruption

Fragile Code (static allocations)

Application specific GC awkward & buggy

17

Garbage Collection in the .NET CLR

The .NET CLR uses a Mark-Compact type garbage collector that utilizes generations and a separate Large Object Heap …

18

Garbage Collection = Detection + ReclamationMark-Compact type garbage collection

Mark – garbage detection GC creates a list of live objects by starting at a set of

roots and walking all referenced objects, marking them as reachable. Anything not marked is reclaimable garbage.

Other types of marking: Reference counting – Number of object references maintained. Copying – copy live objects as they are found to another space.

19

Garbage Collection = Detection + ReclamationMark-Compact type garbage collection

Compact – garbage reclamation GC walks linearly through the heap looking for contiguous

garbage (now free) blocks that it can move live objects down to. In this way live objects are moved to the front of the heap and free space is ‘squeezed’ to the end.

Pointers to moved objects are updated.

Other types of reclamation: Copying: copy live objects as they are found to another space. Sweep: Garbage marked as free, but memory left fragmented.

20

Mark-Compact GC

Mark-Compact Type Garbage Collection

Adv Provides contiguous section of memory for easy future memory allocation.

Object ordering maintained / locality somewhat maintained

Circular references not a problem (e.g. garbage objects that reference each other – keeping themselves alive)

Disadv Memory cannot be accessed during garbage collection.

Several Passes over data required – marking, compute new locations, update pointers, actual object move. Time consuming if a lot of live data left. Generations used to help this situation …

21

Generations Most objects live a very short time.

“80-98% of new objects die within a few million CPU instructions.” Paul R. Wilson, Uniprocessor Garbage Collection Techniques, 1992 International Workshop on Memory Management, St. Malo, France, September 1992.

A small percent live much longer. Using generations capitilizes on these statistics:

Each time an object survives a GC, it is graduated to a higher generation (.NET currently supports generations 0,1, and 2).

The garbage collector will normally do a generation 0 collection. This will lessen the amount of long lived live objects that must be marked and copied during a GC.

22

Large Object Heap

Any objects that are 85,000 bytes or more are allocated on a separate heap – the Large Object Heap.

The Large Object Heap is never compacted. Done to alleviate the need to copy large

objects during the compact phase of the garbage reclamation.

23

How does this all tie together?

Lets reexamine one of the disadvantages listed for the Mark-Compact type GC:

Memory cannot be accessed during garbage collection.

In the case we are examining, the .NET CLR on a uniprocessor system, this means that all threads that are running managed code in the process must be suspended while the GC moves memory and fixes up pointers.

24

The problem restated

A request is made to instantiate a new object, if there is not enough room in generation 0 for this object, then a garbage collection is run.

During a garbage collection, all threads that are running managed code are suspended.

“When Windows suspends a thread, it stops the thread from waiting for any thread synchronization object.

Later, when the thread is resumed, all the suspended threads race back to wait on the object that it was waiting on before it got suspended.

This means that threads are not guaranteed to gain ownership of an object on a first-in-first-out basis.

Since a garbage collection can start at any time (and cannot be prevented), the architecture of the CLR just doesn't support fair thread synchronization — period. “ Jeffrey Richter, In his article "Thread Synchronization Fairness in the .NET CLR”, published 6 February, 2003)

25

References

General:Jeffrey Richter, Thread Synchronization Fairness in the .NET CLR.

http://www.codeguru.com/net_general/ThreadSynchFair.html, published 6 February, 2003

.NETFrancesco Balena, Programming Microsoft Visual Basic .NET, Microsoft Press © 2002.Aaron Barth, Jackie Richards, Production Debugging for .NET Framework Applications,

Microsoft Patterns and Practices, MSDN Online Library, November 2002, (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/DBGrm.asp )

Robert Burns, Multithreaded Programming with Visual Basic .NET, MSDN Online Library, February 2002 (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vbtchasyncprocvb.asp )

Jeffrey Richter, Applied Microsoft .NET Framework Programming. Microsoft Press © 2002.Jeffrey Richter, Thread Synchronization Fairness in the .NET CLR.

http://www.codeguru.com/net_general/ThreadSynchFair.html, published 6 February, 2003

WindowsDavid A. Solomon and Mark E. Russinovich, Inside Microsoft Windows 2000, Third Edition.

Microsoft Press © 2000. Jeffrey Richter, Programming Applications for Microsoft Windows, Fourth Edition. Microsoft

Press © 1999.

Garbage Collection Paul R. Wilson, Uniprocessor Garbage Collection Techniques, 1992 International Workshop

on Memory Management, St. Malo, France, September 1992

26

Questions?

As clear as mud?