Stretching your application from OpenStack into Public Cloud Istio service mesh paves the way John Joyce, Principal Engineer Tim Swanson, Sr. Tech Lead April 26, 2019

Agenda Introduction of Istio and Service Mesh Kubernetes on OpenStack Multicluster Architecture Deployment Details Deployment in Action Mesh Routing Demo Future Improvements References Q&A

Introduction

Extending an Application across Multiple Clouds Public Cloud OpenStack Application Service microservice microservice microservice microservice Decompostion of services Safely High degree of Control Run anywhere On any cloud Public or Private

Stretching Istio’s Service Mesh Across Multiple Clouds Application Service Public Cloud Application Service OpenStack Set policy, traffic routing rules Service Mesh Istio Stretching the mesh Mesh spans the cloud Istio Service Mesh

What is Istio * A uniform way to connect, manage, and secure microservices. Automatic load balancing for HTTP, gRPC, WebSocket, and TCP traffic. Fine-grained control of traffic behavior with rich routing rules, retries, failovers, and fault injection. A pluggable policy layer and configuration API supporting access controls, rate limits and quotas. Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress. Secure service-to-service authentication with strong identity assertions between services in a cluster. Istio is a service mesh framework that provides a uniform way to connect, manage, and secure microservices. Istio provides several features to microservices in the mesh: Automatic multi-protocol load-balancing Multi-protocol traffic controls like service endpoint aware traffic routing, circuit breaking, and fault-injection Pluggable policy layer and configuration API Automatic logging and metrics for microservice traffic Security between all microservices in the mesh * Directly from https://istio.io/about/intro.html

What is a Service Mesh Infrastructure layer for service-to-service communication A mesh of proxies Proxies injected as sidecars Supports numerous protocols (HTTP 1/2, gRPC, TCP, UDP) Can inspect API transactions at Layer 7 or layer 3/4. Intelligent routing rules can be applied between endpoints API Web UI Proxy API Billing DB Proxy Gateway/ Ingress External Requests Proxy API Business Logic Proxy API Accounts DB Proxy Mesh of proxies The APIs between the microservices are front ended by a proxy Control plane for the Proxies Envoy is the proxy Sidecar injected Secure and intelligent routing Istio’s Multicluster enhancement allows cloud boundaries anywhere in above mesh

Istio Architecture Source: https://istio Figure borrowed from Istio.io – would encourage you to go A bit dated Pilot provide mesh topology and routing rules to the proxy Mixer provides Telemetry collection Mixer advanced policy and security checks, especially on server side of the API Istio Auth -> Istio-citadel Provides the Certs to Envoy for both Istio control plane communication & communication between the microservices

Istio Components: Envoy – a proxy injected in every application Pod (as a sidecar) Envoy is a separate open source project in the CNCF Pilot – Serves as a control plane for the proxies to configure them and provide them their routing rules Citadel (previously istio-auth) – Propagates certificates and keys to all the pods to secure both control plane and app exchanges Mixer – Provides telemetry collection as well as sophisticated receive side policy checks.

Kubernetes on Openstack

Openstack API Create K8s cluster k8s Admin How do we use Kubernetes on OpenStack? The idea is similar to other public cloud kubernetes solutions. The kubernetes Admin creates a k8s cluster via openstack APIs (or an installer that uses openstack APIs, ie. kubeanywhere).

Openstack API Use K8s cluster k8s User Masters Minions K8s cluster The k8s cluster is created and then, ideally, the kubernetes user just interacts with the kubernetes cluster directly via k8s APIs. Masters Minions K8s cluster

Kubernetes Openstack Cloud Provider Performs Openstack API operations based on kubernetes events Main Scenario: K8s loadbalancer service is created Openstack LBaaS loadbalancer created and endpoints are setup Certain kubernetes events may have corresponding actions at the cloud infrastructure level to support the desired functionality. Kubernetes defines an architectural component called “cloud-providers” which performs actions on the cloud infrastructure APIs during events in kubernetes. The Openstack cloud-provider is integrated in the kubernetes control-plane. The main scenario pertaining to istio integration with openstack is the openstack cloud-provider’s support for implementing kubernetes loadbalancer objects. The openstack cloud-provider can use the openstack LBaaS API to create loadbalancers and add/remove VIP endpoints corresponding to kubernetes loadbalancer service types. The canonical deployment of istio-ingress relies on a k8s loadbalancer type service. Istio Ingress Depends on Kubernetes LoadBalancer

Masters Minions K8s cluster Neutron Router Load Balancer apiVersion: v1 kind: Service metadata: name: istio-ingress namespace: istio-system labels: istio: ingress spec: type: LoadBalancer ... selector: Neutron Router Load Balancer K8s Service VIP Create Service VIP LB endpoints Openstack Cloud provider The openstack cloud-provider is built into the kubernetes control plane and configured by the kubernetes installer. (e.g. Magnum) The Openstack cloud-provider creates neutron loadbalancers on k8s service type = LoadBalancer setup. Istio Ingress is implemented behind a k8s Loadbalancer type service Masters Minions K8s cluster

LoadBalancers Created for 2 k8s LB Services Created by Kubernetes’ Openstack Cloud Provider Hosting 2 k8s services of type “LoadBalancer” This is the network topology view for a deployment of k8s with the openstack cloud provider setup to create openstack loadbalancers. The images shows 2 Octavia loadbalancer VMs created by the k8s cloud provider when 2 k8s services of type loadbalancer. Octavia implements loadbalancers created by neutron as haproxy VMs (amphora) LoadBalancer VMs K8s cluster VMs

Multicloud Topologies With OpenStack VM Neutron Router Hardware Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo *Also, provider networks Virtual Router Based Multicloud Networking Data Center Infra. OpenStack TOR(s) Internet Edge Infra. VPN/CoLo VM Virtual Router Neutron OpenStack VM Neutron Router + VPNaaS VPNaaS Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo To support the Istio multicluster model the Openstack cloud needs pod reachability inter-cluster. For interconnecting an OpenStack cloud and a Public Cloud there’s a number of options on the openstack side to set it up. The primary requirement is a VPN between the on-prem environment and the public cloud project. With OpenStack these 3 topologies are the primary options.

Istio Multicluster Architecture

Service and Endpoint Discovery Pilot is configured to watch multiple K8s API servers* K8s Clusters KubeAPI Server Controller created in Pilot for each cluster which gathers all service & endpoint data Istio-Pilot KubeAPI Server Istio-Mixer Istio-Citadel Run pilot one cluster Citadel on all clusters Mixer can be either scenario The service and endpoint from all the clusters Clusterregistry style Extended Code base to create multiple controllers Only Kubernetes at the moment Based on master from a few weeks back, but naming on slides is dated GCP vs. GKE. Istio-Citadel OpenStack Cloud * Configuration via file based clusterregistry format

Envoy xDS propagation Pilot Envoy Rules Envoy Micro Service Discovery Micro Service Rules Services() ServiceInstances() Deep dive on details On prem cloud on left Public on right Service form API server Aggregated with config from rules Into a cache Cache provided to Envoy when xDS request Kube API Server Kube API Server Service Registry Istio Rules Services and Endpoints Envoy xDS Requests (grpc/REST-JSON) Envoy xDS Responses (grpc/REST-JSON)

Creating the Stretched Mesh K8s Clusters xDS Data & Telemetry Envoy configured with Pilot & Mixer endpoints for xDS (Discovery Services) APP POD Envoy Istio-Pilot APP POD Envoy Istio-Mixer All envoys talk to common Pilot Common Mixer depicted Pilot holds view of entire mesh Spans all clusters in all clouds Envoy resolves Pilot and mixer addresses local DNS Every cluster needs to have a service and endpoint for Pilot & Mixer Headless services. Envoy Proxy sidecar OpenStack Cloud APP Pods can be Deployed in any cloud uService Application Pods

Certificate Management with Istio-Citadel OpenStack GKE Common CA Root Cert Istio-Citadel provided a common root cert and any intermediate certs required to establish trust chain. Common root Cert allows the apps to authenticate each other across cloud boundaries Run in all clusters Istio-citadel Istio-citadel Certs to Envoy Certs to Envoy Envoy Envoy Micro Service Micro Service Need to ensure Certs are chained to a common root for TLS support Citadel will provide certs to local envoy proxies

Deployment Details

Environment Requirements Must have IP reachability between the clusters For Istio control plane interactions between Pilot, Mixer and Citadel and Envoy Sidecars For application pods to reach each other Solution doesn’t dictate a certain approach to achieve this reachability but generally a VPN would be needed based on current capabilities. Application relies on Kube-DNS to resolve service naming so special attention is needed during app deployment: Use of headless or selectorless services typically required Could be designated to a special purpose DNS server For this demo we provided POD level reachability Could do it differently Relied on on Kube-DNS but could do different High order DNS possible

Istio Control Plane Deployment Istio-Pilot Istio-Citadel Istio-Mixer Istio-ingress OpenStack Cloud Reiterate from above VM on a vsphere We have done this with K8s via Magnum and Openstack Istio-Citadel Istio control plane pods setup only in one Cluster

Bookinfo Application Microservice Architecture V1 = No stars V2 = Black stars V3 = Red stars Pods in GKE Using bookinfo – Istio sample app Blue on prem Tan on GKE Explain stars Pods in OpenStack

Bookinfo Application Flow End User Request For Product page Reviews V1 Productpage Ratings Reviews V2 Response to user Productpage, reviews ratings, details Get Reviews Pods in GKE Get Ratings Reviews V3 External client request is routed through microservices Get Details Pods in OpenStack Details

Deployment in action bookinfo multi-cluster deployment There aren't enough time to conduct a live demo. pre-deployed our multi-cluster environment. walk through some of the major steps with screenshots

two K8s clusters. Top primary in GKE bottom Openstack based cluster We could have easily reversed the roles. Only k8s components only in this step/ POD to POD networking via a VPN

Deploy istio and the sample application bookinfo. many ways to do documented procedure at istio.io site. We chose to use the istio e2e_bookinfo make The make target is executed in the primary cluster in GKE. As such, the istio control plane will be deployed in this GKE cluster.

So we press enter, and the command starts running. deployment process consists of generating istio yaml files Then executing them the clusters

you can see that the deployment is going on quite well at this point.

completed successfully. In GKE all the required istio control components started: The bookinfo app is also running only reviews v1 and v2 in the GKE cluster. On openstack only citadel. There are no bookinfo services running on the openstack cluster at this point

services in both the clusters In GKE all the services have ClusterIPs In GKE istio-ingress has an external IP On Openstack citadel service has cluster IP All other Istio services have match ports opened Head-less services on the remote app can use kube-dns to discover the services that it's calling regardless of cluster

Here, endpoints are shown for both the GKE and the on-prem Clusters Here, endpoints are shown for both the GKE and the on-prem Clusters. On the on-prem cluster, pilot's endpoint is highlighted, and it's the same as it's in the GKE cluster.

The istio-ingress has an external IP configured The istio-ingress has an external IP configured. This IP is used to access the bookinfo service from a browser that's running outside the clusters.

Deploy specific bookinfo services on Openstack cluster. reviews v3 is deployed in Openstack . reviews v3 calls ratings service, Rating service and its endpoints in the GKE cluster A selectorless ratings service op Openstack Discoverable via DNS

Here, it shows the result after deploying the required bookinfo services in the on-prem cluster.

envoy cluster configuration for the reviews service. service_name is a fully qualified dns It also shows envoy will use SDS service to discover all the endpoints that the cluster consists of. On the bottom window, it displays the reviews-v3 pod with an IP 192.170.228.84.

SDS configuration top window . three virtual hosts for the reviews bookinfo service, OpenStack cluster is highlighted. Podium to Tim

Mesh Routing Demo

Future Improvements

Future Improvements: Usability Improvements Increased test coverage Solutions that don’t require a VPN between the Clouds Solutions to address service naming and DNS integration Scale and Performance improvements.

Q & A

References: Initial PR to enable Multicluster: https://github.com/istio/istio/pull/2880 Installation documentation: https://github.com/istio/istio.github.io/pull/1139 Multicloud Design Document: https://tinyurl.com/y7scozvb Zero VPN design document: https://tinyurl.com/zerovpn Bring Your own Service Name Design Document: https://tinyurl.com/svc-name Get involved: istio environments working group: istio-environments@googlegroups.com