The Concurrency And Coordination Runtime And Decentralized Software Services Toolkit

George ChrysanthakopoulosSoftware ArchitectMicrosoft Corporation

Announcing CCR And DSS Toolkit 2008

Enables developers to more easily create loosely-coupled concurrent and distributed applications

Allows early adopters to use the technologies today for building commercial and non-commercial applications

Provides early adopters access to select technologies today; transitioning to Microsoft’s .NET Framework in future

Robots are complex and heavily depend on coordination between software components

The application (robot solving a problem) is a composition of hardware and software, developed by different people

Performance, determinism are critical Concurrency, isolation

Re-use enables progress

CCR/DSS Initially applied in robotics

CCR/DSS Toolkit At A Glance

Runtime• Coordination and

Concurrency runtime (CCR)

• Decentralized Software Services (DSS)

Authoring Tools• Visual

Programming Language

• Visual Configuration and Deployment

• Visual Studio templates

Services• Samples and

tutorials

• Infrastructure services

CCR/DSS Enterprise CustomersBeyond robotics!

Event processing for security systems

Mail sorting systemASP.NET page handling, IO coordination

Concurrency And Coordination Runtime

(CCR)

Why CCR?

Concurrency Process many tasks (load-balance across cores) Scalability, Responsiveness Exploit latency

Coordination Exercise control without blocking threads Orchestrate asynchronous operations New mechanism to handle failure for

concurrent, asynchronous code Runtime

Advanced scheduler with fairness, throttling Extensible primitives

Siemens Infrastructure Logistics Inc.

customer

Siemens postal automation handles majority of worldwide mail volume CCR is now basis of next generation

blackboard system AI agents use OCR and database lookup

to determine mail source/destination Publication/subscription systems that

requires high throughput, isolation between components

New version is scalable, readable, extensible

Quick adoption – Took few days, one dev, to get CCR based solution up and running

Customer Use CaseSiemens Automation

“We deal in milliseconds and microseconds, and there are not very many commercial products that we can take off the shelf and integrate into our product that can meet our demanding performance criteria…”

“We dropped the CCR code into our application and within a few hours we were able to establish the basic mechanics and flow for our AI agents to work with the Blackboard Framework”

Hamid Salemizadeh, Director of Engineering, Reading & Coding, Siemens Infrastructure Logistics Inc.

David Hudspeth, Software Engineer, Siemens Infrastructure Logistics Inc.

Customer Use CaseSiemens case study quotes

Tyco Software House

customer

Tyco event management system used from small stores to the White House Next generation now uses CCR at its core

Fast – CCR solution uses only 1 thread per core and is twice as fast

Responsive – Multiple dispatcher queues and fair scheduling prevent starvation of low frequency events

Robust – Causalities simplify failure handling, increase robustness

Quick adoption: Within a week CCR was integrated

Customer Use CaseTyco

“With CCR, we see linear scaling. If you double the number of processors, you will see a doubling in performance. That is impressive”

“I came back from the Microsoft conference, and two days later I had removed our thread pool code and we were running with CCR. You can’t beat that”

Stephen Tarmey, Architect, Tyco International’s Software House

Customer Use CaseTyco case study quotes

Asynchronous in-process message passing No explicit threads, locks, semaphores!

Task scheduled based on message availability Data-dependency scheduler Models concurrency

Coordination primitives (join, choice, …) Composition of data-driven components

Iterative tasks Express sequential control flow of asynch. tasks

CCR Programming Model

DispatcherPort

DispatcherQueues

ThreadsArbiter

HandlerArbiter is attached to port

Arbiter is activated on queue

Dispatcher schedules items from its queues round-robin to run in its threadsPost places a message

on the port

Arbiter checks whether it can consume the message

Creates work item frommessage and handler

Enqueue work item

Thread calls handler with message as arg

Scheduler picks next work item to execute

HandlerHandler

DispatcherPort

DispatcherQueues

ThreadsArbiter

Handler

There can be many of everything

CCR Coordination PrimitivesExposed via Arbiter methods

Single-Port Primitives

Code-Scheduling(non port-specific)

Multi-Port Primitives

Multi-Port Receiver

Choice (Logical OR)

Join (Logical AND)

Multi-Item Receiver

Single Item ReceiverFromHandler

FromIterator Handler

Interleave(Reader/Writer)

Hello, World!

var dispatcher = new Dispatcher(0,”default”);var queue = new DispatcherQueue(dispatcher);var port = new Port<string>(); port.Post("Hello, World!"); Arbiter.Activate(queue, port.Receive(message => Console.WriteLine(message) ));

Port: Building block for sending and receiving messages

Post: Queues a message

Task: Delegate that handles the message (message is consumed)

Receive coordination primitive

Dispatcher uses fixed number of threads, schedules from queues in round-robin

Port on which to receive the message

Hello, World!Iterator version

void Start(){ var queue = new DispatcherQueue(); var port = new Port<string>(); port.Post("Hello"); port.Post(“World”); Arbiter.Activate(queue, new IterativeTask(() => Hello(port) ) );}IEnumerator<ITask> Hello(Port<string> port){ for(var i=0; i<5; i++) { yield return port.Receive(); var message = (string)port; Console.WriteLine(message); }

Schedule iterative task

Lambda captures arguments for iterator

Yield execution until receive is satisfied

Item retrieved after yield

Loops around asynchronous operations are easy

IEnumerator<ITask> CcrReadFileAsync(string file){ var result = new Port<IAsyncResult>(); using (var fs = new FileStream(file,…,FileOptions.Asynchronous)) { var buf = new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, result.Post, null); yield return result.Receive(); var ar = (IAsyncResult)resultPort.Test(); try { fs.EndRead(ar); ProcessData(buf); } catch { // handle exception } }}

CCR InteropAsynchronous Pattern (Begin/End)

Instead of passing a delegate, stream will post AR to port

Retrieve AR result from port

Complete operation

CCR InteropUsing adapters for common APIs

static IEnumerator<ITask> CopyStream(Stream source, Stream dest){ var buffer = new byte[4096]; int read = 0; do { PortSet<int,Exception> readResult = StreamAdapter.Read( source, buffer, 0, buffer.Length); yield return readResult.Choice(); var exception = (Exception)readResult; if (exception != null) yield break; read = (int)readResult; var writeResult = StreamAdapter.Write(buffer,0,read); yield return writeResult.Choice(); } while (…)}

CCR Adapter for Stream class

Yield to result outcomes

Retrieve success outcome (bytes read)

Proceed to asynchronous write

CCR Concurrent Processing

demo

Decentralized Software Services

(DSS)

Robust Deep isolation (data and execution) Contain and manage failure Uniform concurrency model

Composable Protocol and runtime support to create, manage,

deploy, data driven applications Runtime and tool support for service bindings Publication/subscription integrated with structured

state manipulation Observable

Service is a living document addressable through URIs Consistent mechanism for configuring, managing and

controlling access

Why DSS?

DSSP Protocol

Microsoft Open Specification Promise

DSSP

HTTP

Service State – A Live Document<DriveState   <Connected>true</Connected>  <DistanceBetweenWheels>0.112</DistanceBetweenWheels>  <LeftWheel> …  </LeftWheel>  <RightWheel> …  </RightWheel>  <PollingFrequencyMs>80</PollingFrequencyMs>  <TimeStamp>2007-10-10T13:07:45.5195866-07:00</TimeStamp></DriveState>

Flexible UI

Service Orchestration

Service Properties Identity (URI) Structured State Composition through

partnering Uniform Behavior

Deep isolation in execution and data

State retrieval and manipulation

Service creation and Termination

Notifications are coupled to state changes

Service As A Unit Of Composition

Six Steps To A DSS ServiceDefine Data Types•DATA AND MESSAGE CONTRACTS

Define Service Types•MESSAGE-OPERATIONS•PARTNERSHIPS•POLICIES

Implement Service•MESSAGE HANDLERS•STATE AND PUB-SUB•STATE NOTIFICATIONS

Build Service•VPL TOOL•PROXY GENERATION

Compose Application•DECLARE•DRAW•CODE

Deploy And Run Application•VPL TOOLS•DSS-HOSTED•SELF-HOSTED

Define Data TypesExample state and operation types

[DataContract]class State{ [DataMember] public List<Record> {get; set;}}[DataContract]class Record{ [DataMember] public string Name {get; set;} [DataMember] public int Age{get; set;}}// operationclass Query : Query<Record,Record>{}// operation portclass OperationsPort : PortSet<Query,Get,Update> {}

Service ImplementationDeclare partnerships, operation port

[Contract(Contract.Identifier)]class RecordKeeperService : DsspServiceBase{ [Partner(“PeopleLookup”, Policy = PartnerCreationPolicy.UseExistingOrCreate, Optional = false)] peopleLookup.OperationsPort _peopleLookupPort = peopleLookup.OperationsPort();

[ServicePort(“/recordkeeper”,AllowMultipleInstances = true] OperationsPort _mainPort;

protected override void Start() { base.Start(); }}

Declarative, dynamic composition, annotations picked up by visual editor

Attach handlers to operation port, publish instance URI in service directory

Implement ServiceTypical service handlers

[ServiceHandler(ServiceHandlerBehavior.Concurrent)] public IEnumerator<ITask> QueryHandler(Query queryOp) { var response = FindItem(queryOp.Body); queryOp.ResponsePort.Post(response); yield break; }

[ServiceHandler(ServiceHandlerBehavior.Exclusive)]public IEnumerator<ITask> UpdateHandler(Update updateOp) { QueryAge queryAgeOp; yield return _peopleLookupPort.Query(out queryAgeOp); int age = (int) queryAgeOp.ResponsePort; UpdateRecord(age, updateOp.Body.Name); updateOp.ResponsePort.Post(new UpdateResponse()); }

CCR Interleave is iterator-aware, guaranteeing atomicity across asynchronous steps

Dealing With Partial FailureCausalities

Create causality at root of execution graph

Failure occurs on one of the side branchesDeal with error as

message at origin of execution

DSS/CCRLogSync Sample

demo

Log Sync’ing Demo

Create 1000 services and get their state

Visual Programming LanguageOrchestrating DSS services with dataflow Use any DSS service

Connect services Notifications Request/Response

Control and logic Data transformation Control dataflow

Available DSS services

Request

Notification

Response

Service instance

DSS Manifest Editor IBuilding applications from components

Choose services to instantiate

Partner services using design time declarations Resolve generic services Local or remote

Set initial configuration Edit initial service

state document Create deployment

DSS Manifest Editor IIDistributed applications

Multiple execution nodes A “node”

is an OS process Several nodes

per machine or across machines

Partner services across domains

Create deployment Automatic distributed

startup for testing

DSS/CCRVision Processing

demo

New product addressing distributed, concurrent applications

Enterprise customers are using it now Lightweight concurrency runtime that can

be dropped in existing code Lightweight, observable service framework

Summary

http://www.microsoft.com/ccrdss Download toolkit Documentation (tutorials, videos, user guide) Channel9 Videos Case studies Community Forum

Learn more

Evals & Recordings

Please fill

out your

evaluation for

this session at:

This session will be available as a recording at:

www.microsoftpdc.com

Please use the microphones provided

Q&A

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market

conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.