1. Service Mesh Patterns on Oracle Cloud Infrastructure Container Engine for Kubernetes 
Sherwood Zern
Consulting Solutions Architect
Oracle A-Team
Peter Jausovec
Consulting Member of Technical Staff
Container Development
October 25, 2018
Confidential – Oracle Internal/Restricted/Highly Restricted

2. Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
Confidential – Oracle Internal/Restricted/Highly Restricted

3. Why a Service Mesh
This is an event branded Section Header with Graphic slide ideal for including a picture with a brief title and optional subtitle. This slide can also be used as a Q and A slide.
Do not customize this slide with your own background.
Confidential – Oracle Internal/Restricted/Highly Restricted

4. Agenda Why a Service Mesh Interaction Patterns
Monitoring and Diagnostics 1 2 3
Confidential – Oracle Internal/Restricted/Highly Restricted

5. Microservices Platform & Service Mesh
While Kubernetes takes care of a lot, there is still a lot left up to the developer
The need for:
Strong and Clear API contract
Service Discovery
Resilient Networking
Tracing and Log Correlation Across Services
Traffic Flow Diagnostics
Secure Communication
Confidential – Oracle Internal/Restricted/Highly Restricted

6. Microservices Platform & Service Mesh
Instead of a direct communication model...
Confidential – Oracle Internal/Restricted/Highly Restricted

7. Microservices Platform & Service Mesh
We insert a sidecar alongside every microservice..
(our new data plane)
Confidential – Oracle Internal/Restricted/Highly Restricted

8. Microservices Platform & Service Mesh
Data Planes:
Linkerd, Envoy, HAProxy, Traefik, more..
Control Planes:
Istio, Synapse, Nelson
Some can work together! +
And add a control plane…
Confidential – Oracle Internal/Restricted/Highly Restricted

9. Istio Service Mesh Architecture
This is an event branded Section Header with Graphic slide ideal for including a picture with a brief title and optional subtitle. This slide can also be used as a Q and A slide.
Do not customize this slide with your own background.
Confidential – Oracle Internal/Restricted/Highly Restricted

10. Istio Service Mesh Architecture
Logically split into a data plane and a control plane.
The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices along with Mixer, a general-purpose policy and telemetry hub.
The control plane manages and configures the proxies to route traffic. Additionally, the control plane configures Mixers to enforce policies and collect telemetry.
Confidential – Oracle Internal/Restricted/Highly Restricted

11. Istio Service Mesh Architecture
Envoy
Deployed as a sidecar
Allows Istio to extract a wealth of signals about traffic behavior
Built in features include:
Dynamic Service Discovery; Load Balancing; TLS termination;
circuit breakers; health checks; fault injection; rich metrics
Default installation of Istio enables automatic sidecar injection
$ kubectl label namespace mynamespace istio-injection=enabled
$ kubectl describe po productpage-v1-f8c8fb8-gshrl
Confidential – Oracle Internal/Restricted/Highly Restricted

12. Istio Service Mesh Architecture
Mixer
Mixer enforces access control and usage policies across the service mesh
Collects telemetry data from the Envoy proxy and other services.
Pilot
Provides service discovery for the Envoy sidecars
Traffic management for intelligent routing (A/B tests)
Resiliency (timeouts, retries, circuit breakers)
Converts the high level routing rules into Envoy-specific configurations and propagates them to the sidecars at runtime
Citadel
Strong service-to-service and end-user authentication
Authorization control
Policy enforcement based upon service identity
Confidential – Oracle Internal/Restricted/Highly Restricted

13. Service Mesh Patterns
This is an event branded Section Header with Graphic slide ideal for including a picture with a brief title and optional subtitle. This slide can also be used as a Q and A slide.
Do not customize this slide with your own background.
Confidential – Oracle Internal/Restricted/Highly Restricted

14. Service Mesh Patterns Ingress, Routing, A/B Testing
Used to specify services that should be used outside the cluster
Istio uses a different configuration model than Kubernetes – Uses a gateway
Gateway allows Istio features such as monitoring, and route rules to be applied to traffic entering the cluster
Confidential – Oracle Internal/Restricted/Highly Restricted

15. Ingress, Routing, A/B Testing Demonstration
cd ~/Projects/nodejs/oraclecodeone/routing/istio
Browser:
kubectl apply -f all-v2.yaml
kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 &
kubectl port-forward -n istio-system $(kubectl get pod -n istio-system -l app=jaeger -o jsonpath='{.items[0].metadata.name}') 16686:16686 &
while true; do curl sleep 1; done;
kubectl apply –f yaml
kubectl apply –f yaml
Confidential – Oracle Internal/Restricted/Highly Restricted

16. Service Mesh Patterns Resiliency Retries and Timeouts
Timeout defaults to 15 seconds for HTTP requests
Override Timeouts in a route rule
Specify the number of retry attempts for an HTTP request in a route rule
Consumers can override the timeout and retry defaults by providing request-level overrides
x-envoy-upstream-rq-timeout-ms
x-envoy-max-retries
Fault Injection
Good means to test your services to ensure services work even when conditions are not perfect (Negative testing)
Can also take corrective action when conditions are met
cd ~/Projects/nodejs/oraclecodeone/resiliency/istio
Confidential – Oracle Internal/Restricted/Highly Restricted

17. Resiliency and Monitoring Demonstration
cd ~/Projects/nodejs/oraclecodeone/resiliency/istio
kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 &
kubectl port-forward -n istio-system $(kubectl get pod -n istio-system -l app=jaeger -o jsonpath='{.items[0].metadata.name}') 16686:16686 &
kubectl apply -f istio/10-90.yaml
while true; do sleep 1; curl
Grafana dashboard: Service Dashboard – Incoming requests by Destination and Response Code
Grafana dashboard: Service Dashboard – Incoming request duration by Source
Go to Jaeger UI:
Set service to greeter-service and minimum duration to 2s
Expand the services then expand tags
Show that the /version is taking the longest amount of time
Now Introduce some delays:
kubectl apply –f httpdelay.yaml
Show that the success of non-500 has decreased
kubectl apply –f httpabort.yaml
70% of the requests return a 400 error
Setup timeouts and retries
In the original helloweb implementation we discovered that a timeout was hardcoded to 5 seconds and in case a call to the greeter service was taking more than that, I would get an error. With Istio you can control the HTTP timeouts and retries outside of your code, in the service mesh.
Grafana and look at the incoming requests and response code graph
Set all traffic to go to version 3:
kubectl apply –f all-v3.yaml
Since we know the call to /version will take 2 seconds, let's try setting the HTTP timeout on the greeter service to 0.5 seconds, so it we can see it timeout before that. Adding a timeout is as simple as adding the timeout key to the virtual service like this:
kubectl apply -f istio/httptimeout.yaml
Forcing a timeout to .5s
Incoming Requests by Source and Response code
Similarly as with timeouts, we could add retries and have Istio retry any timed out calls to the service. Let's modify the virtual service and set the timeout to 2 seconds and add 3 HTTP retries at 0.5 second intervals - this way we can easily see from the Jaeger traces that the service mesh is doing three HTTP retries for us
kubectl apply -f httpretry.yaml
Confidential – Oracle Internal/Restricted/Highly Restricted

18. Service Mesh Patterns Egress
Istio-enabled services are unable to access URLs outside of the cluster
The pod uses iptables to transparently redirect all outbound traffic to the sidecar proxy
Proxy only handles intra-cluster destinations
Accessing External URL
Create a ServiceEntry
Adds entries into Istio’s internal service registry
Create an egress gateway to handle all egress traffic
Create a VirtualService to route from the sidecar to the gateway service
Confidential – Oracle Internal/Restricted/Highly Restricted

19. Service Mesh Patterns Mirror / Shadowing
Introduce changes to production with as little risk as possible
Sends live traffic to a mirrored service. The mirrored traffic happens out of band of the critical request path for the primary service
Use a route rule with the mirror: stanza which specifies the host and the service subset
Requests are sent to the mirrored service with their Host/Authority headers appended with –shadow
Requests are “fire and forget”, all responses are discarded
Confidential – Oracle Internal/Restricted/Highly Restricted

20. Egress and Mirroring Demonstration
/Users/szern/Projects/java/src/github.com/sherwoodzern
curl –H “host: weather.com”
/Users/szern/Projects/java/src/github.com/sherwoodzern/weather-1.0/src/main/istio/egress
/Users/szern/Projects/java/src/github.com/sherwoodzern/weather-1.0/src/main/istio
vi oco-weather-mirror.yaml
kubectl logs weather-service-v f8f64-8wwh2 –tail=5 –c svc
Confidential – Oracle Internal/Restricted/Highly Restricted