The Pillars Of Concurrency

Responsiveness and

Isolation

Throughput and

ScalabilityConsistency

Pillar I – Responsiveness and Isolation

UI

BackgroundThread

BackgroundThread

Running expensive work asynchronously via background threads (or thread pool threads) to avoid blocking ‘time sensitive’ threads such as the UI Thread.

  The background threads communicate with the ‘time sensitive’ threads via asynchronous messages

Thread-PoolThread

Msg

Msg

Msg

Msg

How? Using BackgroundWorker

1. private void btoStart_Click(object sender, EventArgs e)  2.         {  3.             backgroundWorker1.RunWorkerAsync();  4.         }  

5.  private void backgroundWorker1_DoWork(…)  2.         {  14.             backgroundWorker1.ReportProgress(…)  15.                   21.             if (backgroundWorker1.CancellationPending)  22.             {  23.               e.Cancel = true;   24.              return;  25.             }                    36.         }  

37. //This event is raised on the main thread.  3. private void backgroundWorker1_ProgressChanged(…)  5.         {  6.             progressBar1.Value = e.ProgressPercentage;   2.         }  

Pillar II -Throughput and Scalability

Distribute the work between threads in order to put as many cores to work, to maximize throughput

1KB and more

instructions

1KB and more

instructions1KB and

more instructions

1KB and more

instructions

1KB and more

instructions

1KB and more

instructions

Work Load

OpenMP TPL

Parallel Studio

1KB and more

instructions

Why?The number of

transistors never stopped

climbing

The Free Lunch is Over However,

clock speed stopped

somewhere near 3GHz

The Solution

Re-Enable the Free Lunch

Use the Thread-Pool to execute

your work asynchronously

Add a concurrency control mechanism which will adjust

the amount of work items thrown into the pool according

to the workload and the machine architecture, in order to put the maximum number of cores to work with minimum

contentions

How many callbacks to put in the

pool?

How to separate the

work?

The Future Lock-Free Thread-Pool

Instead of using a linked list, use the array-style, lock-free, GC-friendly

ConcurrentQueue<T> class

The increasing number of

work items and worker threads

result in a problematic

contention on the pool.

The Future Work Stealing Queues

When work is queued from a

non-pool thread, it goes into the global queue.

Each worker thread in the

pool has its own private WSQ.

When work is queued from a

pool worker thread, the work

goes into its WSQ, most of the time, avoiding all

locking.

WSQ has two ends, it allows

lock-free pushes and pops from

one end (“private”), but

requires synchronization from the other end (“public”)

Worker thread is being

created/assigned to grab work from the global

queueThe worker thread grab

work from its WSQ in a LIFO

fashion, avoiding all locking.

Worker threads steal work from other WSQs in a

FIFO fashion, synchronization

is required.

The Future Work Stealing Queues (cont)

When threads are looking for work, they can have a preferred search order: Check the local WSQ.  Work here can be dequeued without locks. Check the global queue.  Work here must be dequeued using locks. Check other threads’ WSQs.  This is called “stealing”, and requires locks.

The Future Task Parallel Library

Aims to lower the cost of fine-grained parallelism by executing the asynchronous work (tasks) in a way that fit the number of available cores, and providing developers with more control over the way in which the tasks get scheduled and executed.

The Future Task Parallel Library

Exposes rich set of APIs that enable operations such as

Waiting for tasks Canceling tasks Optimizing the fairness when scheduling tasks Marking tasks known to be long-running to help the

scheduler execute efficiently Creating tasks as attached or detached to the parent

task Scheduling continuations that run if a task threw an

exception or got cancelled Running tasks synchronously and more.

Pillar III -Consistency 

With the addition of concurrency comes the responsibility for ensuring safety.  We need to deal with shared memory without causing either corruption or deadlock

What’s the Problem? 

Race conditions - when one or more threads are writing a piece of data while one or more threads are also reading that piece of data. 

What can we do? 

In the mean while – learn to love locks! Use ‘out of the box’ lock-free data

structures  Move forwards functional languages

Avoid sharing altogether Communicate through message passing Use immutable objects Good luck with that! In F#, types are immutable by default

What’s the problem with Locks?

Races – even when using a single lock

Dead Locks – when multiple locks are being acquired in the wrong order

Composability – locks do not compose

Modularity - locks do not support modular programming

And More

What the experts suggest?

Isolation first Immutability second Synchronization last

The Future Transactional memory

Wrapping all the code that access shared memory in a transaction and let the runtime execute it atomically and in isolation by doing the appropriate synchronization behind the scenes. By that, maintaining a single lock illusion.

With STM we can write code that looks sequential, can be reasoned about sequentially, and yet run concurrently in a scalable fashion.

1. atmic   2. {  

//Safely access shared memory  

3. }  

The Future Transactional memory (cont)

STM generates a single lock illusion and still maintain great scalability by utilizing an optimistic concurrency control technique.

Store reads version

Shadow copy writes

Execute transaction (instead of writing into master locations –

write into shadow copies)

Re-execute transaction

(Consider switching to pessimistic

concurrency control)

Copy shadow copies into their master location

Update writes versions

Conflict

Validate

readsSubmit