Parallel programming in modern world .NET TECHNICS

parallelism vs concurrency Concurrency existence of multiple threads of execution goal of concurrency is to prevent thread starvation concurrency is required operationally Parallelism concurrent threads execute at the same time on multiple cores parallelism is only about throughput It is an optimization, not a functional requirement

Limitations to linear speedup of parallel code Serial code Overhead from parallelization Synchronization Sequential input/output

Parallel Speedup Calculation Amdahls Law: Speedup = 1 1 P + (P/N) Gustafsons Law: Speedup = S + N (1 S) S + (1 S) - 0n

Phases of parallel development Finding Concurrency Task Decomposition pattern Data Decomposition pattern Group Tasks Pattern Order Tasks Pattern Data Sharing pattern Algorithm Structures Support Structures Implementation Mechanisms

The Algorithm Structure Pattern Task Parallelism Pattern Divide and Conquer Pattern Geometric Decomposition Pattern Recursive Data Pattern Pipeline Pattern

The Supporting Structures Pattern SPMD (Single Program/Multiple Data) Master/Worker Loop Parallelism Fork/Join

Data Parallelism Search for Loops Unroll Sequential Loops Evaluating Performance Considerations: Conduct performance benchmarks to confirm potential performance improvements When there is minimal or no performance gain, one solution is to change the chunk size Parallel.For and Parallel.ForEach ParallelLoopState for breaking

Reduction/Aggregation

Variations of Reduce Scan pattern - each iteration of a loop depends on data computed in the previous iteration. Pack pattern - uses a parallel loop to select elements to retain or discard => The result is a subset of the original input. Map Reduce

MapReduce Pattern Elements: 1) input - a collection of key and value pairs; 2) intermediate collection - a non- unique collection of key and value pairs; 3) third collection - a reduction of the non-unique keys from the intermediate collection.

MapReduce Example Counting Words across multiple documents

Map/reduce via PLINQ

Futures Future is a stand-in for a computational result that is initially unknown but becomes available at a later time. A future in .NET is a Task that returns a value. A .NET continuation task is a task that automatically starts when other tasks, known as its antecedents, complete.

Futures example

Dynamic Task Parallelism Dynamic Tasks (decomposition or divide and conquer) - tasks that are dynamically added to the work queue as the computation proceeds. Most known instance recursion.

Dynamic Task example

Pipelines Each task implements a stage of the pipeline, and the queues act as buffers that allow the stages of the pipeline to execute concurrently, even though the values are processed in order. The buffers BlockingCollection

Pipeline example

C# 5 : async and await asynchronous pattern, event-based asynchronous pattern, task-based asynchronous pattern (TAP) : async&await!

Asynchronous Pattern

Event-Based Asynchronous Pattern

Task-Based Asynchronous Pattern

Using Multiple Asynchronous Methods Using Multiple Asynchronous Methods: vs Using Combinators:

Converting the Asynchronous Pattern The TaskFactory class defines the FromAsync method that allows converting methods using the asynchronous pattern to the TAP.

ERROR HANDLING

Multiple tasks error handling

CANCELLATION Cancellation with Framework Features

CANCELLATION Cancellation with custom tasks

Literature Concurrent Programming on Windows MapReduce: Simplified Data Processing on Large Clusters by Jeffrey Dean and Sanjay Ghemawat. 2004 Parallel Programming with Microsoft Visual Studio 2010 Step by Step Parallel Programming with Microsoft.NET: Design Patterns for Decomposition and Coordination on Multicore Architectures Pro .NET 4 Parallel Programming in C# [Adam_Freeman] Professional Parallel Programming with C# [Gaston_Hillar] Professional Parallel Programming with C# Master Parallel Extensions with NET 4 .NET 4.5 Parallel Extensions Cookbook | Packt Publishing

Questions? Parallel programming in modern world .NET technics

Thanks!