.NET Core Summer event 2019 – Brno, CZ Async demystified .NET Core Summer event 2019 – Brno, CZ Karel Zikmund – @ziki_cz

Agenda History of async patterns in .NET History and evolution of Task async-await Based on internal talk from author of Task, async and all good things around – Stephen Toub, architect of BCL

APM pattern: Asynchronous Programming Model .NET Framework 1.0/1.1 … 2002-2003 interface IAsyncResult { bool IsCompleted; bool IsCompletedSynchronously; object AsyncState; WaitHandle AsyncWaitHandle; } Across BCL: IAsyncResult BeginFoo(..., AsyncCallback callback, object state); void EndFoo(IAsyncResult iar); IAsyncResult AsyncWaitHandle – ManualResetEvent or AutoResetEvent Across BCL Usage either: Wait for callback to be called, or Call EndFoo which will block until completed

APM pattern Synchronous call: Achieving the same: IAsyncResult BeginFoo(..., AsyncCallback callback, object state); void EndFoo(IAsyncResult iar); Synchronous call: Foo(); Achieving the same: EndFoo(BeginFoo(..., null, null)); Leveraging asynchronous calls: BeginFoo(..., iar => { T val = EndFoo(iar); // do stuff ... }); Single operation works fine, but in reality you do more – e.g. in a loop

APM – Example Copy stream to stream: int bytesRead; while ((bytesRead = input.Read(buffer)) != 0) { output.Write(buffer, 0, bytesRead); }

APM – Nesting problem BeginRead(..., iar => { int bytesRead = EndRead(iar); input.BeginWrite(..., iar2 => { int bytesWritten2 = EndWrite(iar2); BeginRead(..., iar3 => { int bytesRead3 = EndRead(iar3); BeginWrite(..., iar4 => { // ... again and again }); Manually it does not work – somehow turn it into loop It’s possible but extremely long and tricky Further complications with IsCompletedSynchronously

APM – IsCompletedSynchronously IAsyncResult r = BeginRead(..., iar => { if (!iar.IsCompletedSynchronously) { // ... asynchronous path as shown earlier } }); if (r.IsCompletedSynchronously) { // ... Synchronous path Even worse in loop Overall very complicated Queueing on ThreadPool much simpler For perf reasons In the loop it is even more complicated However: On MemoryStream, the data is already available … instead of ThreadPool, call delegate immediately -> Leads to recursive calls -> 10K StackOverflow Bottom part: Even BCL lots of wrappers (e.g. in Networking: LazyAsyncResult) with lots of specializations Very complicated

EAP: Event-based Asynchronous Pattern .NET Framework 2.0 obj.Completed += (sender, eventArgs) => { // ... my event handler } obj.SendPacket(); // returns void Did not solve multiple-calls problem, or loops Introduced context Straightforward idea – Completed event Kick off operation, then Completed handler is invoked (generally on ThreadPool) 5-10 classes in BCL … like SmtpMail, TcpClient, BackgroundWorker Downsides: We shipped it in .NET Framework 2.0 and quickly realized that it is interesting experiment, but not exactly addressing real needs

Task .NET Framework 4.0 MSR project – parallel computing Divide & conquer efficiently (e.g. QuickSort) Shaped Task – similar to today Task – represents general work (compute, I/O bound, etc.) = promise / future / other terminology Task / Task<T> – operation (with optional result T) T … in the case of Task<T> State related to synchronization State related to callback 90% right, 10% keeps Toub awake at night even after 10 years and would love to change it NOT tied to ThreadPool – not tied to executing delegate Shove result into it Can be completed Can wake up someone waiting on it

Task / TaskCompletionSource Here's a callback, invoke it when you're done, or right now if you've already completed I want to block here, until your work is done Cannot be completed by user directly TaskCompletionSource … wrapper for Task Holds Task internally and operates on it via internal methods Methods: SetResult SetException SetCancelled Task – something to consume - hook up to, not to change directly (no control) TaskCompletionSource – can alter state of Task … has control over Task Lazy initialization (something over network) … you are in charge who can change complete the work

Task – Consumption Either: Or: Even multiple times: ContinueWith: Task<T> t; Either: t.Wait(); // Blocks until Task is completed Or: t.ContinueWith(callback); // Will be executed after Task is completed Even multiple times: t.ContinueWith(callback2); t.ContinueWith(callback3); ContinueWith: Does not guarantee order of executions Always asynchronous (queued to ThreadPool/scheduler in general) Wait - creates ManualResetEvent which will be signaled when one of the SetResult/SetException/SetCancelled is called Option: TaskExecutionOption to do it synchronously APM - IAsyncResult … no shared implementation Everyone had to have their own implementation Task model - you don't pass delegate at creation, but you can walk up on any of them and say "call me when you're done" Abstractions enabled - like async-await await hooks up the callback

Task.Run We complicated things  Task<T> Task.Run(delegate d) Adds field to Task with ‘d’ Queues work to ThreadPool Thread grabs it, executes it, marks task completed Sets completed = execute callback, waking up things waiting on it, etc.

Task.Run implementation Task<T> Run(Func<T> f) { var tcs = new TaskCompletionSource<T>(); ThreadPool.QueueUserWorkItem(() => { try { T result = f(); tcs.SetResult(result); } catch (ex) { tcs.SetException(ex); } }); return tcs.Task; TaskCompletionSource creates Task Returns Task to be awaited on, etc. Now we implemented Task.Run without storing any delegate on Task

async-await .NET Framework 4.5 / C# 5 Example of asynchronous code: Task<int> GetDataAsync(); Task PutDataAsync(int i); Code: Task<int> t = GetDataAsync(); t.ContinueWith(a => { var t2 = PutDataAsync(a.Result); t2.ContinueWith(b => Console.WriteLine("done")); }); GetData/PutData … maybe across the wire

async-await C# 5 with async-await helps us: Task<int> t = GetDataAsync(); t.ContinueWith(a => { var t2 = PutDataAsync(a.Result); t2.ContinueWith(b => Console.WriteLine("done")); }); C# 5 with async-await helps us: int aResult = await t; Task t2 = PutDataAsync(aResult); await t2; Console.WriteLine("done"); Compiler translates it to the code above (hand-waving involved) Compiler does not treat Task specially, but it just looks for pattern (awaiter pattern)

Awaiter pattern Translated to: int aResult = await t; var $awaiter1 = t.GetAwaiter(); if (! $awaiter1.IsCompleted) { // returns bool // ... } int aResult = $awaiter1.GetResult(); // returns void or T // If exception, it will throw it Bold methods are pattern matching

Awaiter pattern – details void MoveNext() { if (__state == 0) goto label0; if (__state == 1) goto label1; if (__state == 42) goto label42; if (! $awaiter1.IsCompleted) { __state = 42; $awaiter1.OnCompleted(MoveNext); return; } label42: int aResult = $awaiter1.GetResult(); “! IsCompleted” part is complicated – I have to hook up code that comes back here when task completes All of it is part of MoveNext method -- it is a state machine, every await in method is state in state machine (hand waving a bit) OnCompleted has slightly more complicated signature

State Machine State machine: string x = Console.ReadLine(); int aResult = await t; Console.WriteLine("done" + x); State machine: struct MethodFooStateMachine { void MoveNext() { ... } local1; // would be ‘x’ in example above local2; params; _$awaiter1; } How does ‘x’ survive continuation? (it is just on stack) – need to capture it Same in continuations - C# compiler lifts it to keep it on heap allocated object (floats through closures) Same in state machine Compiler optimizes - stores here things only crossing await boundary In debug -- struct is class -- for debuggability, but for perf struct Why? These async methods often complete synchronously – example: BufferedStream … large buffer behind with inner stream If I ask for 1B, but it reads 10K in bulk, then lots of calls end up synchronously If it was class, then we would allocate per call

State Machine – Example public async Task Foo(int timeout) { await Task.Delay(timeout); } public Task Foo(int timeout) { FooStateMachine sm = default; sm._timeout = timeout; sm._state = 0; sm.MoveNext(); return ???; struct FooStateMachine { int _timeout; // param // locals would be here too void MoveNext() { ... } int _state; TaskAwaiter _$awaiter; }

State Machine – Example public Task Foo(int timeout) { FooStateMachine sm = default; sm._tcs = new TaskCompletionSource(); sm._timeout = timeout; sm._state = 0; sm.MoveNext(); return sm._tcs.Task; } AsyncValueTaskMethodBuilder.Create(); _tcs.Task -> _builder.Task; struct FooStateMachine { int _timeout; // param // locals would be here too void MoveNext() { // ... _tcs.SetResult(...); } int _state; TaskAwaiter _$awaiter; TaskCompletionSource _tcs; _tcs on state machine is logically there problem 2 allocations – TaskCompletionSource and Task (inside) For the synchronous case we want 0 allocations ideally (BufferedStream example) Even the Task/TaskCompletionSource is problematic, because it is anything Task-like Each Task-like type (except Task) has attribute defining builder - builder pattern ValueTask has one -> AsyncValueTaskMethodBuilder instead of new TaskCompletionSource() -> AsyncValueTaskMethodBuilder.Create(); We have internally in System.Runtime.CompilerServices structs: AsyncTaskMethodBuilder, AsyncTaskMethodBuilder<T>, AsyncVoidMethodBuilder instead of _tcs.Task -> _builder.Task We eliminated TaskCompletionSource allocation What about the Task?

State Machine – Summary What about Task allocation? Builder can reuse known tasks Task.CompletedTask (without value) boolean – True/False int … <-1,8> LastCompleted (e.g. on MemoryStream) Does not work on SslStream (alternates headers and body) Size: 64B (no value) / 72B (with value) Azure workloads OK (GC will collect) Hot-path: up to 5%-10% via more GCs

ValueTask .NET Core 2.0 Also as nuget package down-level struct ValueTask<T> { T; Task<T>; } Only one of them: T+null or default+Task<T> NET Core 2.1 ValueTask<int> Stream.ReadAsync(Memory<byte>, ...) Methods are 1-liners (if Task<T> == null, do something, else something else) Nicely handles synchronously completing case Note: Non-generic ValueTask does not make sense - only Task inside … we have CompletedTask .NET Core 2.1 Luckily we introduced Memory<T> at the same time, as we cannot overload on return type That's why sometimes in PRs we wrap byte[] in Memory first … to use the ValueTask Design-guidelines: Start with Task … use ValueTask only in hot-path scenarios

ValueTask – Can we improve more? What about the 1% asynchronous case? .NET Core 2.1 struct ValueTask<T> { T; Task<T>; IValueTaskSource<T>; } struct ValueTask { Task; IValueTaskSource; IValueTaskSource – complicated interface almost the awaiter pattern: Are you completed? Hook up a call back Get a result All implementations on ValueTask are now ternary Value: You can implement it however you want, incl. reset (reuse) Complicated to do, so not everywhere Socket.SendAsync/ReceiveAsync Object per send/receive … if one at a time (typical) 0 allocation for loop around Send/Receive on Socket Same: NetworkStream, Pipelines, Channels

Summary @ziki_cz APM pattern = Asynchronous Programming Model .NET Framework 1.0/1.1 (2002-2003) IAsyncResult, BeginFoo/EndFoo Limited nesting / loops EAP = Event-based Asynchronous Pattern (.NET Framework 2.0) Events – similar problems as APM Task (.NET Framework 4.0) Wait / ContinueWith TaskCompletionSource (for Write) async-await (.NET Framework 4.5 / C# 5) Awaiter pattern, state machine ValueTask (.NET Core 2.0) Don’t use unless you are on hot-path Hyper-optimizations possible, stay away!  @ziki_cz