1. Ansible & CloudFormation

2. Ansible & CloudFormation
About me
John Heller. DevOps Engineer.
About this presentation
Ansible – you know what that is. Right?
CloudFormation – AWS proprietary template engine for creating collections of related AWS resources
Many ideas here translate just as easily to Terraform (if you want to get all vendor–agnostic)
This is NOT Ansible vs CloudFormation
Ansible vs CloudFormation
I've heard that it requires a lot of custom modules.

3. Why do we need CloudFormation?
IAM Policies and Roles
S3 Buckets
CloudFront
Route 53 Records
RDS Instances
EC2 Instances
Auto-scale groups
Load Balancers
Security Groups
VPC
Subnets
Route Tables
Gateways
Network ACLs
AWS reference architecture
Why do we need CloudFormation?

4. Anatomy of a CloudFormation template
{ "AWSTemplateFormatVersion" : "version date", "Description" : "JSON string", "Metadata" : { template metadata }, "Parameters" : { set of parameters "Mappings" : { set of mappings "Conditions" : { set of conditions "Resources" : { set of resources "Outputs" : { set of outputs }
Anatomy of a CloudFormation template
Parameters can be passed in, the referred to with a { “Ref”: “Param_name” }
The properties of resources can reference other resources in the same template using either “Ref” or { “Fn::GetAtt”: [ “Resource”, “Attribute” ] }
Resources are declared in any order. CloudFormation works out the best order to create them, and does parallel creation when possible.
Possible outputs: resource ids (e.g. subnet-id), IP addresses, DNS names, arbitrary strings including any of these (e.g a URL to hit an ELB)

5. CLOUDFORMATION is… Good Bad
Provides repeatable parametrised infrastructure. “Infrastructure as code” Handles resource dependency ordering Parallelised resource creation Easy removal
Ugly and difficult to read (JSON sux)
Obsessive about commas (JSON sux)
Templates are limited to 51,200 bytes via API (460,800 via S3)
Updates can be unpredictable
Rollbacks can fail
You cannot manually alter any resource created by CF.
Change sets are now available as an aid to stack updates.

6. DIVIDE and Conquer Super Stack Database Stack Network Stack
{ "Fn::GetAtt" : [ "myStack", "Outputs.BucketName" ] }
Super Stack
Different environments in different network stacks
Different databases for DBA of application devs
Devs can have their own application stack, but use a common database
Database Stack
Network Stack
Application Stack

7. DIVIDE and Conquer Database Stack Network Stack Application Stack
Ansible handles all stack inputs and outputs, and accepts top-level parameters.
Database Stack
Network Stack
Application Stack

8. A simple CloudFormation play
{{ vpc_stack.stack_outputs.SubnetPubA1 }}
- name: Provision Stack
hosts: localhost
connection: local
gather_facts: False
vars_files:
- "vars/cf_vars.yml"
tasks:
- name: Run my CloudFormation stack
cloudformation:
stack_name: "MyStack"
region: "{{ aws_region }}"
state: "{{ stack_state }}"
template: "MyCFTemplate.json"
template_parameters:
VPCId: "{{ vpc_id }}"
VPCCIDR: "{{ VPCCIDR }}"
SubnetToolA1CIDR: "{{ SubnetA1CIDR}}"
SubnetToolB1CIDR: "{{ SubnetB1CIDR }}"
tags:
Owner: "{{ owner }}"
CostCentre: "{{ cost_centre }}"
register: vpc_stack
A simple CloudFormation play
Template parameters are passed as YAML dict
An Ansible variable is registered is outputs from the module.
If stack with same name and template already exists, nothing is done, but outputs are still returned.
Stack outputs can be referenced in subsequent tasks.
Tags aren’t applied to some things – e.g. EBS volumes attached to EC2 instances.
{{ vpc_stack.stack_outputs.SubnetPubA1 }}

9. A role for each stack
- name: Provision Stack hosts: localhost connection: local gather_facts: False vars_files: - ./vars/cf_vars.yml roles: - "{{ cf_stack }}"
Keeps together the template and any vars and associated tasks.
Can have role dependencies in the ‘meta’ section.
Run a stack’s role and it will automatically run roles it depends on first. This will either create the required stack, or acquire its outputs if it already exists.
Must be careful not to trigger stack updates.
What about stack removal?
Network Stack
Database Stack
Application Stack

10. Reversing Order for Removal
- name: Run the first stack cloudformation: &my_anchor stack_name: "MyStack" state: "{{ stack_state }}" ... when: stack_state == "present" - name: Run the second stack cloudformation: stack_name: "MySecondStack“ - name: Run the first stack last when deleting cloudformation: *my_anchor when: stack_state == "absent"
Uses YAML anchors and aliases
Only really works with 2 stacks

11. - name: Create the parametised CF stack template template: src: "application_stack.template.j2“ dest: "application_stack.template“ - name: Compress the CF stack shell: “python compress_json.py application_stack.template application_stack.json” - name: Run the CF stack cloudformation: stack_name: "{{ app_stack_name }}” region: "{{ aws_region }}“ state: "present“ disable_rollback: true template: "application_stack.json“ register: app_stack
Template the Template
No need to pass parameters – write them straight into the template
Jinja can do some handy things in templates
Include common sections
Loop over dicts
Strip whitespace from templates to fit more in the char limit.

12. Be a Jinja Ninja https://goo.gl/HD9cL1 GitHub Gist Code example
{% import 'cloudformation.macros.j2' as cf with context %} {
"Description": “Security groups and NACLs",
"Resources": {
{{ cf.securitygroups(security_groups) }},
{{ cf.nacls(nacl_rules) }} }
GitHub Gist
Code example

13. Ansible Maths
Define a set of subnet CIDR ranges based on an environment number.
Env 0 : /22
Env 1: /22
Env 2: /22
vpc_base: 64 VPCCIDR: "10.1.{{ vpc_base + env|int * 4 }}.0/22“ SubnetToolsA1CIDR: "10.1.{{ vpc_base + env|int * 4 }}.0/25“ SubnetToolsB1CIDR: "10.1.{{ vpc_base + env|int * 4 }}.128/25“ SubnetDbA1CIDR: "10.1.{{ vpc_base + env|int * }}.0/25“ SubnetDbB1CIDR: "10.1.{{ vpc_base + env|int * }}.128/25“ SubnetAppA1CIDR: "10.1.{{ vpc_base + env|int * }}.0/25“ SubnetAppB1CIDR: "10.1.{{ vpc_base + env|int * }}.128/25“ SubnetPubA1CIDR: "10.1.{{ vpc_base + env|int * }}.0/25“ SubnetPubB1CIDR: "10.1.{{ vpc_base + env|int * }}.128/25"

14. Working with Assumed ROles
- set_fact:
role_arn: "arn:aws:iam::{{ account_id }}:role/stackRole“
- name: Assume a role
sts_assume_role:
role_arn: "{{ role_arn }}“
role_session_name: "{{ session_type }}-{{ jenkins_build_no }}“
region: "{{ aws_region }}“
register: assumed_role
- name: Run the CF stack
cloudformation:
aws_access_key: "{{ assumed_role.sts_creds.access_key }}“
aws_secret_key: "{{ assumed_role.sts_creds.secret_key }}“
security_token: "{{ assumed_role.sts_creds.session_token }}“
stack_name: "{{ app_stack_name }}”
state: "present“
template: "application_stack.json“
register: app_stack
Set a role on the EC2 instance running Jenkins that allows it to assume other roles.
Your Jenkins can then run stacks in other accounts.

15. Thank You