It’s the App, Stupid! Orchestration, Automation, Scaling & What’s in Between Yaron Parasol, Uri

Meet Petsy, Selling Pet Art and Supporting Pet Artists Since 2013

So They’re Services Are Deployed on OpenStack Gunicorn PostgreSQL Hadoop ActiveMQ Nginx MongoDB Logstash GraphiteJenkinsNagios

The Business Is Doing Great, Thx (esp. Cat paper crafts), But… Rolling out new code is Painful

The Business Is Doing Great, Thx (esp. Cat paper crafts), But… MTTR is… mehhh Mehhh

They Had a Server Failure Two Weeks Ago One of the web servers crashed due to a bug in the Inventory service It took a few days to get the system back on its feet…

It Was also the Case When They Needed a New Server It’s still troublesome to roll out a new Gunicorn and add it to the existing deployment

Another Thing Is Quality… They have a nice set of unit tests, but integration testing is mostly manual…

They Need to Automate!!

So, Let’s have a closer look at DevOps Processes It’s about Workflows and Triggers

Automated Deployment Triggers: Manual, CI Server Flow: – Provision Cloud Resources: Compute, Storage, Network – Configure servers – Push code – Start components

Infra Upgrade Trigger: Critical updates, patches, etc. Flow: Take down relevant processes, one by one, apply patch, and reconnect to cluster. In some cases restarts server

App upgrade (or more generally, CD) Trigger: CI server, Manual Flow: Canary, Red/Black, A/B… – In general – Partial deploy, verify, rollback / complete deploy

Node Failure Trigger: happens… Flow: Detect, provision VM, attach storage, add to network, reconfigure app (load balancer in case of web serve, DB connection in case of database, etc.)

Scaling Trigger: System SLAs, Schedule Flow: Add new nodes, push code, reconfigure dependent nodes, reconfigure LB

Automation refers to all the layers Proxy Config Python App Schema Infrastructure Middleware NGinx Gunicorn Postgres Application Host Networks Storage WE need Workflows

Take a (Very) Simple Deployment Scenario.. WE need Workflows

Let’s have a look at AWS Compared to OpenStack

The AWS Stack

The OpenStack Equivalents OpenShift/ CloudFoundry **Solum** Heat Nova, Cinder, Neutron etc.. ? ?

He Calls It Devops Automation

Still a bit too rigid

OpsWorks – We can do better! Integrated with Heat Cross Clouds Custom Workflows Not limited to Chef Open Monitoring and Policies

How do you build the WF piece? Introduction to the Cloudify COSMO project

Our TOSCA-Inspired Building Blocks Application Topologies Workflows Policies

Application Topologies Nodes – The layers in the cake - e.g. VM, web server, app module – Actions (can be grouped into Interfaces) Relationships – Actions – Requirements capabilities

Workflows Create and interact with nodes using actions

Policies Trigger workflows based on a certain system state

DSL GUI How It Works in Practice Blueprint + Runtime Model Blueprint + Runtime Model Metrics Plugins Workflow Engine Workflow Engine TaskBroker TaskBroker Policy Engine Policy Engine Agen t Metrics Collecto r Plugin REST API REST API Agen t Plugin

Live Demo – Installing Mezzanine Create the VMs Install Postgres Install Gunicorn Install NGinx Create DB on Postgres Push app to Containers Configure App connection to DB Configure Nginx routing rules Start components in the right order

Some Resources Framework (code name – Cosmo) code and sample app are available on github github.com/cloudifysource/cosmo- manager github.com/cloudifysource/cosmo-mezzanine- example github.com/cloudifysource/cosmo- manager github.com/cloudifysource/cosmo-mezzanine- example

Plans, Plans

How Does It Fit into the OpenStack Eco-System? The need was identified (Call for TOSCA like DSL) Solum Project – Solum Blueprints identified the needs: Cross Cloud Continuous Deployment – As of last week we have joined Solum In order to contribute the relevant parts into Solum

Thank You! Come meet us in booth C27