I’ve listed the primary inventors first: Async.NET Working Group – formed by Soma to come up with a better story for asynchrony F# - introduced “async workflows” Axum – also introduced “async methods”. The implementation in C#/VB is copied from that of Axum. Task<T> - came out of MSR and was productized in VS2010. It’s the foundation on which Async is built. The chief executors/implementors of the current CTP are on the second line. We’re a team of five PMs – we managed to release an internal build of CTP with no dev or QA! (but the day after we finished our internal build, we got a QA and he already filed three bugs... And we subsequently got more QA assistance, and they filed another hundred and fifty bugs...)

1

Just so I know we’re speaking the same language, I’ll define the terms as we’re using them. People have often thought that asynchrony was synonymous with “run the work on a background thread and let me know when it’s done”. But that’s just one possible implementation of asynchrony. The actual meaning of asynchrony, as its name suggests, is purely about results that come some time after you initiated the request. Background worker threads are one way for asynchrony to arise – but they’re by no means the only way, and often not the best way either.

2

Incidentally, we hear a lot about multicore, a sort of blind panic. We hear that programmers will suddenly have to start using it everywhere in their programs but that nobody has a clue how to. That’s wrong. We won’t have to start using it everywhere.

3

*hint: the waiter isn’t blocked while awaiting for the diners to finish their meals+

4

*hint: when you’re idle, awaiting a network response, you’re not doing work+

5

100-level talk: demo [but only the first half of it -- skip the threading explanation] 200-level talk: demo 300-level talk: demo + framework + [optionally, design, if the audience is interested] 400-level talk: demo + framework + platform + design + theory

6

GOALS: Watch to this demo to learn how to use the “async” feature in your own code, and how much it will improve your personal productivity over what was available before.

7

8

Async is a feature that was designed at the same time in VB and C#, by the same designers. It was implemented at the same time in the VB and C# compilers, by the same implementors. In these slides I switch freely between C# and VB.

9

10

11

I’ve focused just on the GetDiggStory routine for now.

12

Actually, it’s more common for non-expert developers to “AddHandler, AddressOf MyFunction”. Then, if there are any variables they want shared between the initiator and handler, they have to lift them into class members.

13

14

Again, non-expert users won’t code this way. Instead they’ll take the continuation code out of Button1_Click, and move it inside the body of the handler. They’ve had to modify the structure of their code because they didn’t discover any better way. Also, because they don’t know how to do callbacks, they’re unable to write the kind of modular “compositional” code that they used to in the synchronous world.

15

16

Again, for then non-expert user who moves their caller’s continuation out of the caller and into the body of the handler, they still face this error problem. Question: why can’t we just throw the exception if we discover it inside our handler? Answer: because it will be thrown to the code that invoked our HANDLER, i.e. the network stack, rather than being thrown back up to Button1_Click.

17

Going back up to fix the Button1_Click routine as well.

18

Incidentally, WebClient is helpful because it fires its events back on the UI thread. Other asynchronous methods (e.g. WebRequest.BeginGetResponse), fire back their events on a background worker thread. And background threads are never allowed to modify the UI, neither on Silverlight nor on WPF. Therefore the user often has to do additional code to marshal the callback back onto the UI thread, like this: GetDiggStory( Sub(story) Dispatcher.BeginInvoke( Sub() textBox1.Text = story.Description End Sub) End Sub) Two levels of nested lambdas, instead of just a simple semicolon (or linebreak)... Crazy!

19

20

I hadn’t spelled out the exact change we made to GetDiggComment. Initially it returned a string. Should we modify it now to return a structure containing either an error or a string? Or should it just return the string content of the exception, in case of error? Both possibilities are pretty unappealing.

21

22

23

24

Note that the Return statement has different meaning in an Async method. This GetDiggStory function returns a Task(Of DiggStory), but its return statement just takes a DiggStory operand. The return statement is “Taskified” automatically by all Async methods. (Also, only Async methods are allowed to have Awaits in them).

25

This bubbling-up of asynchronicity happens all the way up the call hierarchy – all the way up to a void-returning method (a “fire-and-forget” method, one which initiates a potentially long-running operation but doesn’t care when the result is done). Or, it can bubble up as far as a “Task.Run(Action)” statement, to run the body of work on a thread in the threadpool. Note that ALL of this user code executes on the UI thread. Even with APIs like WebRequest.BeginGetResponse (which normally execute their callbacks on a background worker thread) – when we call the Task-Async version of them, WebRequest.GetResponseAsync, we will continue on the UI thread. Now think back to the Android Developer Blog at the start of this talk. In what we’ve written here with “Await”, the UI still does the same minimum amount of work, but we haven’t needed to use any background threads to avoid blocking the UI. NOTE: While we’re stuck in the “Await” statement, asynchronously awaiting for the web-service response, control flow has actually returned to the main message-loop. It’s quite like Yield in the old Win16 days. (It’s a bit like DoEvents in Win32 days, except that its cleaner: it returns to the main message-loop instead of starting a subsidiary one). That means that if the user clicks a button while we’re awaiting, then that will initiate a second concurrent request to GetDiggStoryAsync. (concurrency through cooperative

26

yielding). We usually don’t want this kind of re-entrancy. So we normally will protect our top-level handlers against re-entrancy, e.g. Sub Buton1_Click(...) Button1.Enabled = false ... Finally Button1.Enabled = true End Finally End Sub By the same token, if there are multiple concurrent requests happening cooperatively, then at the point where any one awaits, it’s possible for any other one to resume its work. This is “cooperative yielding”. Developers have to make sure their shared data isn’t left in a bad state at the await-points. Background worker thread: shared data has to be protected against pre-emption ANYWHERE, and so you have to use locks all over the place. Async: shared data merely has to be pretected against cooperative yielding at the “Await” points. Much easier.

26

27

[Joke: oh, you don’t want to go home early? Great. In that case I’ll dive into how it actually worked under the hood.] GOALS: To understand the “await” feature at a professional level -- that is, where the flow-of-control goes, which threads are involved, and how they communicate. Also to understand more deeply what asynchrony is. This will be enable you to be a better architect of asynchronous programs, e.g. Silverlight and ASP. This is the demo code, but I simplified it a little. I also split up “GetDiggAsync” and “await” onto separate lines. (Also I ported it over mostly to C#, apart from the XML literals). There are two existing threads allocated by the operating system. One is the UI thread for this process. The other is the “IO Completion Port” thread. Each of these threads has a queue associated with it.

28

When the user clicks a button, this inserts a “button-click-message” into the UI queue. The UI thread is in a while loop, checking for messages. When it gets this message it invokes the button-click handler.

29

The same thread makes function calls. When it calls the API “web.DownloadStringTaskAsync”, this returns immediately. It returns a Task, which I’ve called “downTask”. This task has not yet completed. The network stack will know to mark it as completed once the server’s response comes back. I should stress that “Task” does not mean a “BackgroundThreadTask”. The Task class is unrelated to questions of threads of execution. A “Task” is merely a “future” (C++), a “promise” – it’s an object that exists in one of three states, “InProgress” or “Completed(with result)” or “Faulted(with exception)”. It can be caused to transition from the first state to either of the other two. When this transition takes place, it will invoke any continuations that have been registered with it. And Task is a great unifying abstraction. That’s because it can stand for so many things – for a background worker thread on the current machine, or for a thread of execution on some remote database server, or for things that don’t take any threads at all like a button-click or a DMA transfer from disk to memory. Actually, if you’re familiar with the TPL, Task has a third state “Cancelled”. Through APIs we end up treating this state as equivalent to Faulted(with

30

OperationCancelledException).

30

Now we execute the “await” operator. This does two things. First, it signs up a continuation onto downTask. For now I’ve written the continuation as “ui.Post,K1-” – not in real syntax. We’ll see later what it does. (Note that a task is allowed to have many continuations signed up on it.) Next, the first time we execute the “await” operator in an async method, we return a Task immediately. Once again, this task has not yet completed.

31

We execute the “await” operator. once again, this signs up a continuation onto the task, and returns to the calling thread (the UI thread). The UI thread can now resume it’s “while”-loop, checking for messages. If any other button-clicks happened (or mouse-drags or repaints or window-resizing) now, then they could be dealt with by the UI thread fine. This is where responsiveness comes in. (but it also brings in re-entrancy... imagine if the user clicked the same button again! then we’d start a second concurrent run through this button1_Click handler!)

32

Hey! A few seconds later, and the web services has delivered its answer to the IO Completion Port thread!

33

The IOCP thread knows which task was associated with that response. So it transitions it from the “Running” state to the “Completed with result ‘rss’” state. This causes it to execute the task’s continuation.

34

The continuation merely adds a message into the UI queue. Then it returns, allowing the IO Completion Port thread to resume its work.

35

The UI thread picks up the message from its queue, and responds by dispatching to K1, which assigns into the variable “rss”.

36

The code continues to execute. it comes to the “return” statement. (note that we have already returned the Task<string> “diggTask” from this method. So you know the return statement is going to do something different...)

37

The “return” statement, in an async method, sets its returned task’s state to “Completed”, provides a result, and executes the task’s continuation. Once again, the continuation merely posts to the UI’s message-queue.

38

The method returns. Now the UI-thread can go back to its “while” loop, checking for messages in the queue. (There is one already!)

39

And the UI thread executes the method, puts the story into the text-box, and finishes. Note that ALL user code executed on the UI thread. All of it. That means the user never had to worry about the typical multi-threaded problems (semaphores, mutexes, semaphores, races, locks, ...) Also count how many threads were involved. Just the two that were already provided by the operating system. We didn’t create ANY additional threads.

40

We’ve shown that it’s not necessary to launch a background thread to do long-running operations – since asynchronous waiting just does as well. Indeed it’s positively worse to use a background thread where asynchrony would do. That’s because background threads introduce additional problems – heavy resource overhead, mutexes/semaphores/races, can’t update UI from background thread – without any advantages. The only good reason for pre-emptive multitasking or multicore is for CPU-bound tasks. (however, cooperative concurrency through asynchrony is still very useful!)

41

GOALS: Learn how to extend the “await” feature with your own types, not just Task. This will open up some powerful domain-specific flexibility in your libraries. You now know exactly how the “await” language feature works. Here I’m going to give its implementation details. The interesting thing we’ll gain out of this – is that the feature isn’t even tied solely to tasks! It’s possible to await expressions of your own type. (We actually use this feature in the Async CTP. We let you “await Task.Yield()”, to temporarily return control to the main message-pump. We are also considering “await taskScheduler.SwitchTo()”, to switch the current routine to start executing on that task scheduler – achieving the same effect as Apple’s Grand Central Dispatch.)

42

43

I’ve written “K1” here, just as a fiction to tide us over for four more slides. After that we’ll see what we actually do for K1.

44

45

46

What we’re doing here is making a State Machine. A state machine’s external interface is characterized solely by one single delegate, a “moveNext” delegate, which people can call to transition the machine into its next state. The rest of its fields (_state &c.) are local.

47

48

We said that a “Task” – a “Future” – a “Promise” – is not related to threads of execution. It’s merely an object that exists in one of three states, “InProgress” or “Completed(with result)” or “Faulted(with exception)”. The calls to SetResult and SetException are what transition the returned Task into one of those two states.

49

50

On the “await” *consumption+ side, we are flexible: we can await any type so long as it has the right methods. On the “async method” *production+ side, we are inflexible: we are hard-coded to return only the Task type (or void). Why the inconsistency? 1. Iterators already have this behavior... An iterator method (one which has a “yield” inside) is hard-coded to return either IEnumerable or IEnumerator. However, you can “foreach” over any type which has GetEnumerator/MoveNext/Current members. So Async is just following suite. 2. A task is like a future, so it’s good to hard-code it... A Task is barely more than a future. A future is a basic fundamental part of a language/platform. There’s no reason for a language two have multiple copies of such a fundamental notion. One is enough. It’s so foundational that you might even add keywords to the language to deal with futures.

51

Anyway, if someone has a future-like thing, or a richer notion of task, then they can build it out of Task or Func<Task>. (Our Tasks are already running. If you want to build something that’s “cold”, like F# asyncs or like IObservable, one which doesn’t start until you tell it – then you should build it out of a Func<Task> rather than out of a Task). 3. Further subtleties Define this function: void f<T>(Func<Task<T>> f) And invoke it: f( () => 1 + await t ) We’d like to be able to infer that T=int in this case. Such inference isn’t possible unless the compiler has hard-coded knowledge that the lambda it passes to “f” has type Task<int>.

51

Here we’ve added the ability to “await” for a button click event. It can be dangerous to await for ephemeral events like this – what if the button happened to be click before we called “await Button1”? Then we’d lose the click! It’s a good design practice to await things like Task which, once completed, remain completed forever.

52

Some additional work is needed if the user had “try” blocks in their code. Here’s an example.

53

• The CLR doesn’t let you jump into a nested try block. To support this, we need to have multiple jump tables, one at the start of each try block. They act like staging posts. (Note also that you can’t jump into a catch or finally block. Therefore the “await” keyword isn’t allowed in them.)

• We also keep an “bypassFinally” flag. That’s so that, when we do

“TrySetContinuation” and then “return”, we don’t execute the finally blocks on the way out. The flag makes sure that they only execute at the right time.

54

55

GOALS: To learn how to consume asynchronous framework APIs. To learn how to write asynchronous framework APIs.

56

These are some of the “Task Async Pattern” APIs that are included in the CTP.

57

58

59

60

61

62

We initially had var task2 = task.TimeoutAfter(1000); But we removed it because it didn’t have a clear-enough design. Would you want task2 to end with an OperationCancelledException after 1000ms? Or to end successfully? Both forms are useful. In the end, we figured that cancellation-after-1000ms was easier done like this: var cts = new CancellationTokenSource(); cts.CancelAfter(1000) And we figured that successful-termination-after-1000ms was clearer if you wrote it out manually: var task2 = task.WhenAny(task, Task.Delay(1000))

63

There are three kinds of async methods. The difference between them is subtle. The difference between them is so subtle that we considered eliminating the first kind “void-returning asyncs” entirely. But that would have been wrong. That’s because every single async method you write will be invoked by someone who is also async, all the way up to the very top level of the callstack, to the “fire-and-forget” event handlers at the top like Button1_Click(). So: every program that you’ll ever write will use void-returning asyncs. We have to accept that, and include it as a language feature.

64

65

Something to note here is that Task is composable. For instance, you can write a method which takes any two tasks and awaits until they’re both done. You can’t do that with the APM or the EAP. That’s because APM is just “a pair of methods with arbitrary parameters” and WebClient is just “a set of methods and events you have to call”. They’re not first-class citizens. They’re not things that you can pass as parameters to another method. But you can pass a Task directly to another method. That’s why the TAP is better.

66

GOALS: There are none. This section of the talk is candy for intellectually-curious language enthusiasts who want to understand how the language feature was designed. Here I’m going to talk about how we designed the language feature. There were lots of trade-offs. The ultimate design we chose wasn’t inherently necessary or obviously better than the rivals. Our judgment was that it worked better than the alternatives. I’ll talk about some of them.

67

68

69

70

Stack-spilling is costly to implement. We’ll end up spending half of our development budget solely on stack-spilling. The reason is that normally, CLR is a stack machine... e.g. when you invoke a method or operator, first it pushes the arguments on the stack, then it calls the method or operator. But imagine if the second argument had an “await” in it. The compiler needs to return at this point. But you’re not allowed to return when things are already on the stack! So we have to do the equivalent of register-coloring ourselves. We have to lift local variables AND pushed stack variables into the state-machine. It kind of defeats part of the purpose of the CLR – which was to have one common engine that does this low-level compiler work, leaving the compilers to work just on the higher-level bits.

71

72

73

74

GOALS: This section of the talk is of academic interest to theoretical computer scientists, and aims to inspire interest in the feature at a research level. Here I wanted to talk a bit about the academic computer science and mathematical underpinnings of the feature. I reckon that the best way to design a clean and future-proof language is by tying it to well-understood mathematical foundations and work that’s already been well explored before.. I’d like to understand what are the foundations of “await” and async methods.

75

We’ve said that Task<T> return type is part of re-ifying the stack so that CallCC can work. But it’s more powerful than that, than just an accidental way to reify the stack. In its own right it has interesting theoretical foundations... [next slide]

76

This is all Erik Miejer’s idea. I don’t understand it yet.

77

78