Making the cloud event-driven, and orchestrated Ivan Čuljak

Who am I, and why am I here?  Ivan Čuljak .NET freelancer in love with Azure, and Xamarin.Forms Started playing with serverless somewhere in autmn 2016 Spent the last 18+ months migrating, and developing serverless stuff on a quirky SaaS project Tested, and sometimes deployed, a whole lot of scenarios that probably weren’t intended to be ran on serverless 

Microsoft Tech Summit FY17 5/2/2019 4:01 PM The “evolution” of application platforms © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Tech Summit FY17 5/2/2019 4:01 PM What is the right size of servers for my business needs? How do I deploy new code to my server? What happens in case of server hardware failure? What media should I use to keep backup? Which packages should be on my server? Who monitors my App? What size of servers should I buy? How often should I backup my server? How can I increase server utilization? How can I scale my app? Who has physical access to my servers? Which OS should I use? Do I need secondary network connection? Are my server in a secure location? What happens if the power goes out? Who monitors my Servers? Do I need a UPS? What storage I need to use? How many servers do I need? How can I dynamically configure my app? It takes how long to provision a new server? How often should I patch my servers? On-Premises The “evolution” of application platforms © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Tech Summit FY17 5/2/2019 4:01 PM What is the right size of servers for my business needs? How can I increase server utilization? How many servers do I need? How can I scale my app? How often should I patch my servers? How do I deploy new code to my server? How often should I backup my server? Which OS should I use? Which packages should be on my server? Who monitors my App? On-Premises IaaS The “evolution” of application platforms © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Tech Summit FY17 5/2/2019 4:01 PM What is the right size of “servers” for my business needs? How can I increase “server” utilization? How many “servers” do I need? How can I scale my app? On-Premises IaaS PaaS The “evolution” of application platforms © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Tech Summit FY17 5/2/2019 4:01 PM How do I architect my app? Serverless, the platform for next gen apps On-Premises IaaS PaaS Serverless The “evolution” of application platforms © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft Tech Summit FY17 5/2/2019 4:01 PM “Problem” #1 So… how do we know what happened, and react to that? © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Modern computing is all about events Events moving around in chaos City slides up and question marks fade in and wiggle back and forth

? ? ? ? ? ? ? ? ? ? ? ? Managing events is important but cumbersome Events moving around in chaos ? ?

What if all these events could be managed and directed from one place? Events moving around in chaos

Azure Event Grid Backbone of event-driven computing Fully-managed event routing Near real-time event delivery at scale Broad coverage within Azure and beyond OOTB Publishers + Custom Capabilities Here are some of the key features of Azure Event Grid: Simplicity - Point and click to aim events from your Azure resource to any event handler or endpoint. Advanced filtering - Filter on event type or event publish path to ensure event handlers only receive relevant events. Fan-out - Subscribe multiple endpoints to the same event to send copies of the event to as many places as needed. Reliability - Utilize 24-hour retry with exponential backoff to ensure events are delivered. Pay-per-event - Pay only for the amount you use Event Grid. High throughput - Build high-volume workloads on Event Grid with support for millions of events per second. Built-in Events - Get up and running quickly with resource-defined built-in events. Custom Events - use Event Grid route, filter, and reliably deliver custom events in your app. Backbone of event-driven computing

Handling Webhook Events Even if you use Azure API Management, you still need to have the middleware running behind the API Management proxy since you can't push directly to a topic. Courtesy of Tom Kerkhove: https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/

Handling Webhook Events Wouldn't it be nice if we can get rid of that host and let API Management push the requests directly to Azure Event Grid so that it can fan-out all the external notifications to the required processors? (assumes no validation or business logic) Courtesy of Tom Kerkhove: https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/

Manage all events in one place Routing logic is embedded in the publishers Courtesy of Tom Kerkhove: https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/

Manage all events in one place Routing logic managed centrally inside Event Grid Courtesy of Tom Kerkhove: https://blog.tomkerkhove.be/2017/08/22/exploring-azure-event-grid/

Concepts Event : what happened Event Publisher : where it took place Topic: where publishers send events Event Subscriptions: routes & filters events Filters: EventTypes, SubjectBeginsWith, SubjectEndsWith Event Handlers: app or service reacting to the event

Event Grid, Event Hubs, Service Bus? Service Bus : Transactions, control, pull Event Hub : Telemetry, streaming Event Grid : Event handling, reactive, push Service Bus: High-value enterprise messaging - Order processing and financial transactions Event Hubs: Big data pipeline - Telemetry and distributed data streaming Event Grid: Reactive programming - React to status changes https://buildazure.com/2017/09/07/a-tour-of-azure-messaging-services-queues-event-grid-iot-hub-and-more/

Blob Event Sample [ { "topic": "/subscriptions/{id}/resourceGroups/{rg}/providers/Microsoft.Storage/storageAccounts/{storageacc}", "subject": "/blobServices/default/containers/monitored/blobs/blob.json", "eventType": "Microsoft.Storage.BlobCreated", "eventTime": "2017-09-20T01:12:35.8066557Z", "id": "9ac0de1a-001e-00da-23ad-311d4a06bb2f", "data": { "api": "PutBlob", "clientRequestId": "9ac0de1a-9da0-11e7-9a3b-51cc40a2ca27", "requestId": "9ac0de1a-001e-00da-23ad-311d4a000000", "eTag": "0x9AC0FC4ADA5997D", "contentType": "application/json", "contentLength": 268, "blobType": "BlockBlob", "url": "https://{storageacc}.blob.core.windows.net/monitored/blob.json", "sequencer": "00000000000019200000000000F12B37", "storageDiagnostics": { "batchId": "9ac0de1a-e1a1-44c5-a0a1-a66b2a191fc4" } ]

Custom Event Sample custom data { "topic": "/subscriptions/{id}/resourcegroups/{rg}/providers/microsoft.eventgrid/topics/{topic}", "id": " 9ac0de1a-83f2-4eb7-ab1f-e2e8d2ef1728", "eventType": "CryptoCurrencyPriceObtained", "subject": "cryptoCurrency/price/BTC", "eventTime": "2017-09-19T11:37:01.1513463+00:00", "data": { "AUD": 4991.89 } custom data

Routing Storage Account Events

Microsoft Tech Summit FY17 5/2/2019 4:01 PM “Problem” #2 How do we perform long running or complex tasks on serverless, and make them stateful? © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Enable long running Function Orchestrations Durable Functions Enable long running Function Orchestrations

Pattern #1: Function chaining - Today Problems: No visualization to show relationship between functions and queues. Middle queues are an implementation detail – conceptual overhead. Error handling adds a lot more complexity.

Pattern #1: Function chaining - Better // calls functions in sequence public static async Task<object> Run(DurableOrchestrationContext ctx) { try var x = await ctx.CallFunctionAsync("F1"); var y = await ctx.CallFunctionAsync("F2", x); var z = await ctx.CallFunctionAsync("F3", y); return await ctx.CallFunctionAsync("F4", z); } catch (Exception) // global error handling/compensation goes here CallFunctionAsync uses queues under the covers, thus they can be scaled out to multiple VMs.

Pattern #2: Fan-out/Fan-in - Today Problems: Fanning-out is easy, but fanning-in is significantly more complicated Functions offers no help with this scenario today All the same problems of the previous pattern

Pattern #2: Fan-out/Fan-in - Easy public static async Task Run(DurableOrchestrationContext ctx) { var parallelTasks = new List<Task<int>>();   // get a list of N work items to process in parallel object[] workBatch = await ctx.CallFunctionAsync<object[]>("F1"); for (int i = 0; i < workBatch.Length; i++) Task<int> task = ctx.CallFunctionAsync<int>("F2", workBatch[i]); parallelTasks.Add(task); } await Task.WhenAll(parallelTasks); // aggregate all N outputs and send result to F3 int sum = parallelTasks.Sum(t => t.Result); await ctx.CallFunctionAsync("F3", sum);

Pattern #3: HTTP Async Response Start DoWork GetStatus Problems: Execution state needs to be explicitly stored and managed. Execution state and trigger state must be kept in sync manually. Start and GetStatus is often boilerplate code that is not related to the business problem.

Pattern #3: HTTP Async Response – Built in! > curl -X POST https://myfunc.azurewebsites.net/orchestrators/DoWork -H "Content-Length: 0" -i HTTP/1.1 202 Accepted Location: https://myfunc.azurewebsites.net/orchestrators/b79baf67f717453ca9e86c5da21e03ec Server: Microsoft-IIS/8.0 > curl https://myfunc.azurewebsites.net/orchestrators/b79baf67f717453ca9e86c5da21e03ec -i HTTP/1.1 202 Accepted Content-Length: 173 Content-Type: application/json Location: https://myfunc.azurewebsites.net/orchestrators/b79baf67f717453ca9e86c5da21e03ec Server: Microsoft-IIS/8.0 {"runtimeStatus":"Running","createdTime":"2017-03-16T21:20:36Z","lastUpdatedTime":"2017-03-16T21:20:47Z"} HTTP/1.1 200 OK Content-Length: 175 Content-Type: application/json Server: Microsoft-IIS/8.0 {"runtimeStatus":"Completed","createdTime":"2017-03-16T21:20:36Z","lastUpdatedTime":"2017-03-16T21:20:57Z"} Logic Apps and Microsoft Flow have built-in support for this pattern. Note that specifics URL routes are subject to change (and have already changed since this deck was originally written).

Pattern #4: Actors Problems: Functions are stateless and short-lived. Read/write access to external state needs to be carefully synchronized. Functions do not support the singleton pattern today.

Pattern #4: Actors (Eternal Orchestrations) public static async Task Run(DurableOrchestrationContext ctx) { int counterState = ctx.GetInput<int>(); string operation = await ctx.WaitForExternalEvent<string>("operation"); if (operation == "incr") counterState++; } else if (operation == "decr") counterState--; ctx.ContinueAsNew(counterState); Note that orchestrators are single-threaded, so there is no need to worry about concurrency. Orchestration instances are also implicitly singletons, so they only run on one VM at a time.

Pattern #5: Human Interaction w/Timeout ProcessApproval RequestApproval Escalate Problems: Can’t easily coordinate a timeout with an approval request notification. Need a mechanism to reliably cancel either the approval handling or the timeout depending on the outcome.

Pattern #5: Human Interaction w/Timeout public static async Task Run(DurableOrchestrationContext ctx) { await ctx.CallFunctionAsync<object[]>("RequestApproval"); using (var timeoutCts = new CancellationTokenSource()) DateTime dueTime = ctx.CurrentUtcDateTime.AddHours(72); Task durableTimeout = ctx.CreateTimer(dueTime, 0, cts.Token); Task<bool> approvalEvent = ctx.WaitForExternalEvent<bool>("ApprovalEvent"); if (approvalEvent == await Task.WhenAny(approvalEvent, durableTimeout)) timeoutCts.Cancel(); await ctx.CallFunctionAsync("HandleApproval", approvalEvent.Result); } else await ctx.CallFunctionAsync("Escalate"); We will expose a mechanism for sending named events to an orchestrator from outside the orchestration context – e.g. REST API and/or output binding.

Important Orchestrator Limitations Orchestrator code is replayed on every rehydration to restore all local state (local variables, etc). Function calls are never replayed – the outputs are remembered. This requires the orchestrator code to be deterministic. Rule #1: Never write logic that depends on random numbers, DateTime.Now, Guid.NewGuid(), etc. Rule #2: Never do I/O directly in the orchestrator function. Rule #3: Do not write infinite loops Rule #4: Use the built-in workarounds for rules #1, #2, and #3