Micro-service Orchestration for Serverless Cloud Computing Cathy Zhang, Distinguished Engineer, Huawei Louis Fourie, Senior Staff Engineer, Huawei

What is Serverless Serverless Benefit to Micro-service Developers Serverless Challenge Solution to the Challenge Three Serverless Usage Scenarios

What is Serverless Serverless is a cloud computing code execution model in which the cloud provider fully manages starting and stopping of a Microservice/Application. In Serverless, the cloud provider automatically provisions a container to host the Microservice when it needs to run (runs when that event happens) The Microservice developers do not need to provision Infrastructure resource ( compute, network, security, etc. ) and can concentrate on their own functional logic. The Microservice developers also do not need to manage scalability. The cloud provider (Serverless platform) will auto-scale more container instances to run the Microservice when a burst of events happen. It is based on utility billing, the micro-service developers pay for the compute resource that their Microservice actually uses. Page 3

(One Event Triggers One Function) What is Serverless (One Event Triggers One Function) There are three categories of MicroService scenarios in Serverless domain Page 4

(One Event Triggers Multiple Functions) What is Serverless (One Event Triggers Multiple Functions) Page 5

(Event-Function Interleave Graph) What is Serverless (Event-Function Interleave Graph) Page 6

Benefit to the Serverless Users Internet No Fault Tolerance and HA to Manage Continuous Scale Up/Down Never Pay for Idle No Servers to Manage Reduce Infrastructure management and operation pain : no management of server, software installation, SW life cycle management, networking, security, monitoring, Agility for user to roll out new features No need to plan for the peak. Platform provides dynamic need based Auto-scale. No need for Infrastructure investment, utility/usage based paying. Only charge platform usage No Need to Manage HA，Upgrade, Fault Tolerance Very useful for event driven applications with unpredictable traffic (Snapchat Social Network, IoT), run and scale as needed The users can concentrate on their function logic and roll out new features faster Page 7

Serverless Challenges In Serverless domain, micro-service applications are broken down into a set of cloud functions interleaved with event triggers How to provide generic configuration directives for cloud functions, events, and their relationship How to provide a consistent and simple programming model for the user to specify its micro-service workflow, such as the interaction between events and cloud functions, information passing between cloud functions Consistency in User Facing Serverless Configuration

Service Graph Model List of Events and Event sources: Storage event, HTTP event, Media event, DB access event, email event, code repo update event List of States (model the workflow and weave the event-function together): event-state task-state switch-state wait-state pass-state Actions/Functions associated with a state (the user’s functionality logic ): Directives for parallel/sequential /branching execution of functions Directives for Retry payload filtering/passing A natural way to model a user’s Microservice workflow is a state machine-like service graph which is an Event-Function Relationship Graph as shown in the diagram: what event triggers what function, are the functions executed in sequence or parallel or branches. The graph allows the Microservice developer to group function pipeline and related events together

Serverless Provisioning (Event Configuration) Three Parts in a Service Graph Configuration What operation should trigger an event Event Data Format Event ID (return)

Serverless Provisioning (Function Configuration) Function Code Location Function Name Code Execution Start Point Function Language Runtime (java, python, etc.) Function Version Function Memory Size Function Library Package

Serverless Provisioning (Service Graph Configuration) Event: event-IDs State : event-state, switch-state, task-state, wait-state Function: Function-ID execution-mode: sequence/parallel Function invocation mode: sync/async Function Retry Event Payload Filtering/ Passing Mask

Service Graph Execution Architecture plugin architecture, various backend technologies as plugins

Human triggers stock trade confirmation event (Stock Trade Scenario) Event1: client requests trade FN1: authenticate the user and parse the request. Cloud Function(FN1) result returned in JSON payload ($.result). Switch to different states based on result in JSON payload FN2: process buy operation, check if there is enough money in account, is it valid stock symbol, ask client to confirm FN3: process sell operation, check enough shares, is it valid stock symbol, ask client to confirm Event2: client confirms buy/sell operation FN4: execute the trade, update the account, send client notification This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Human triggers stock trade confirmation event (Stock Trade Scenario) "states": [{ “TradeState": { "type": "EVENT", "event-list": [{ "event-expression": "Event1", "actions": [{ "action": { "function": FN1, "next-state": "SwitchState", } }] }, "SwitchState": { "type": "SWITCH", "payload-switch": [{ "choice1": { "path": "$.choice", "value": “buy", "next-state": “BuyState", "choice2": { "value": “sell", "next-state": “SellState", }],      “BuyState": { "type": "TASK", "actions": [{ "action": { "function": FN2_Buy, } }], "next-state": “ConfirmState", },      “SellState": { "function": FN3_Sell, “ConfirmState": { "type": "EVENT", "event-list": [{ "event-expression": "Event2", "function": FN4_trade, "next-state": “TradeState", }] Text Representation and Graphic Representation This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Multiple events trigger parallel processing (Home IoT Scenario) Event1: Motion detection event Event2: Smoke detection event Event3: Window/Door opening (intrusion) event FN1: Event Correlation Analysis FN2: text and email to home owner FN3: sound alarm and call to police department This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Multiple events trigger parallel processing (Home IoT Scenario) "event-defs": [{    "Ev1": { "source": "Trigger-Source1", },    "Ev2": { "source": "Trigger-Source2", }   "Ev3": { "source": "Trigger-Source3", "EventState": { "type": “EVENT", "event-list": [{ "event-expression":"Ev1|Ev2|Ev3", "action": { "function": FN1, "next-state“:“SwitchState", }] }], "SwitchState": { "type": "SWITCH", "payload-switch": [{ "choice1": { "path": "$.choice", "value": “ok", "next-state":“EventState", } "choice2": { "value": “alarm", "next-state":”ParallelState", }, }], "ParallelState": { "type": "TASK", "action-mode": PARALLEL, "actions": [{ action": { "function": FN2, "action": { "function": FN3, “end": true }] Text Representation and Graphic Representation This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Chatbots Scenario Many chatbots are state-less and have limited functionality Customer service needs stateful chatbots and NLP to respond intelligently Better solution is to use Service Graph to describe chatbot scenario Event1: user login with username and password FN1: Authentication Event2: chat text comes FN2: Natural Language process, interpret the text, generate text response Event3: end chat This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Chatbots Scenario Text Representation and Graphic Representation {"service-graph-name": { "event-defs": [{ “Event1": { "source": “ChatBot-UI", }, “Event2": { }. “Event3": { } }], "states": [{ “AuthState": { "type": "EVENT", "event-list": [{ "event-expression": “Event1", "actions": [{ "action": { "function": FN1, "next-state": “ChatState", }] “ChatState": { "type": "EVENT", "event-list": [{ "event-expression": “Event2", "actions": [{ "action": { "function": FN2, "next-state": “ChatState", } "event-expression": “Event3", “end": true }] }, Text Representation and Graphic Representation This is a classic Hello World example. This code on the left, generates this graph on the right. We specify where we start, we define each state, and we define each state transition. And then we get back to working on the code that makes our apps unque.

Thank you Page 20