Continuous Deployment tool Mahesh Veerabathiran AWS Certified Developer Associate AWS Certified Sysops Administrator Associate

Agenda Stages of Transformation High level design overview Significant features Big picture - CICD Pipeline Prerequisites Demo Tool information

Stages of Transformation Developing API and deploy services majorly using CFT and Chef since couple of years ago.

Stages of Transformation Our initial pipeline uses following tools Git – Used for Version controlling. Jenkins – Runs code quality check and prepare artifacts for deployment Uses Sonar for code quality validation. Stores artifact in to Nexus. Provision cloud infrastructure using Cloudformation. Infrastructure configuration is handled by Chef. Using chef, one can build reusable libraries for infrastructure configuration called Cookbooks. Each cookbook can have more than one recipe. Using Chef Role, one can define list of cookbook/recipe to be executed in order as per their requirement. Chef Environment – Let you define any environment specific configuration for your requirement Version of application artifact information is defined in chef environment artifacts

Stages of Transformation Maintenance of multiple CFT’s and Chef artifacts though does not have major differences 1st API Stack 2nd API Stack 3rd API Stack Cloudformation Templates Chef Artifacts (Role, Env.) Cloudformation Templates Chef Artifacts (Role, Env.) Cloudformation Templates Chef Artifacts (Role, Env.)

Stages of Transformation Built Generic CFT’s to maintain one set of CFT templates for all our API’s

Stages of Transformation But it did make our deployment complex, especially for Deployment team as deployment is still manual. ASG ELB

Stages of Transformation Expanded our API footprint further in West region for high availability. Enabled Blue/Green deployment for zero application downtime. Ended up spending more hours in deployment. ASG ELB 2x for Feature deployment 4x for AMI rehydration

Stages of Transformation It complicated further our deployment process. Also, It became difficult to get the release team availability.

Stages of Transformation Realized the need to automate our releases and looked for a solution to leverage automation solution using Chef and CFT’s

Stages of Transformation All the available automation tools are based on Terraform, Docker etc. As our deadlines are narrow, could not spend time in new tools. Expectation EC2 vs Reality Terraform

Stages of Transformation Hence started coding simple script for automation and it became Continuous deployment tool in a week time.

High level flow diagram

Config. file

Significant Features All Cloudformation parameters are config. file driven

Artifact info. hardcoded in Chef Env. Significant Features Artifact info. hardcoded in Chef Env.

Significant Features Now API artifacts (*.war, *.jar) version are not hardcoded in CHEF. Calls Nexus API to get latest artifact version & store artifacts info. in S3. During Instance bootstrapping, pull down artifact info. from S3 and deploy the artifacts using custom cookbooks. Therefore, it eliminates the need for maintaining chef artifacts one for blue and other for green.

Significant Features Once the stack (blue/green) is created, the DNS recordset to create/update R53 is prepared and stored in S3. Configure cloud watch to monitor ELB instance health and triggers SNS when at least one of the instance becomes healthy One could write lambda which subscribes SNS event. When triggered, pull R53 update info. from S3 and update R53 DNS recordset to route traffic to new stack.

Significant Features One could also trigger R53 update manually when application traffic is low using automation job. Sample R53 update JSON file.

Significant features

Big Picture - CICD Pipeline

Prerequisites S3 bucket should be setup for staging temporary information AWS System account & Access keys. System account should have read/write permissions to your S3 bucket. System account should also have following IAM policies enabled iam:PassRole route53:ListResourceRecordSets route53:ChangeResourceRecordSets Need IAM role to access your S3 bucket from EC2

Demo

Questions

Tool information GitHub repo Contact me Deck: https://github.com/dironman/continuousdeployme nt-meetup.git Sample config: https://raw.githubusercontent.com/dironman/conti nuousdeployment- meetup/master/mortagage_loan_config.json Contact me https://www.linkedin.com/in/aboutmahesh