1. MANAGE AWS INFRASTRUCTURE AS CODE USING TERRAFORM

2. 0.
AGENDA

3. 0. AGENDA State of things Terraform 101 Terraform 201
Getting started with Terraform
Terraform 201
Advanced concepts in Terraform
Demos
CI/CD with Terraform
Demo

4. 1.
STATE OF THINGS
AWS + Infrastructure as code

5. AVAILABLE TOOLS AWS CloudFormation Puppet, Chef, Ansible, Salt…
AWS API, libraries (Boto, Fog)
Terraform by HashiCorp
Who is using AWS API directly or using libraries (like Troposphere written in Python) ?

6. AVAILABLE TOOLS AWS CloudFormation Puppet, Chef, Ansible, Salt…
Puppet, Chef, Ansible, Salt…
AWS API, libraries (Boto, Fog)
Terraform by HashiCorp
Who is using AWS API directly or using libraries (like Troposphere written in Python) ?

8. “HashiCorp is Atlassian for DevOps.” Someone at DevOps conference

9. TERRAFORM
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently.

10. TERRAFORM FACTS Latest version: 0.6.8 (released 2.12.2015)
Open-source, written in Golang.
Very active development:
CHANGELOG.md (ca. 1 release per month)
GitHub Issues (ca issues resolving daily)
Growing community (IRC, Mailing list, Stack Overflow)

11. TERRAFORM VS CLOUDFORMATION
Principles

12. JSON HCL/JSON No Yes Yes! Limited Yes (hard) Only AWS
CloudFormation
Terraform
Configuration format
JSON
HCL/JSON
State management No
Yes
Execution control
Yes!
Logical comparisons
Limited
Supports iterations
Manage already created resources
Yes (hard)
Providers supported
Only AWS
20+ (incl. AWS, GCE, Azure)
State management - TF has local tfstate file describing metadata of created resources
Execution control = well controlled. Plan => output file or limit by targets => apply with confidence. CF can only validate syntax.
Logical comparisons = more, less, equal value. In TF you can use “count=0” or “count=1” resource parameter instead of boolean true/false to control resource creation.
Manage already created resources like EIP, S3 buckets, VPC is not possible without deleting them first.

13. AWS SPECIFICS 121 103 90% Work in progress Optional rollback
CloudFormation
Terraform
AWS resource types
121
103
Resource properties and operations completeness
90%
Work in progress
Handle failures *
Optional rollback
Fix it & retry
Contribute? No
Yes!
GH issue #28
Some resource properties (for example, ec2 keypair) can be created using AWS API, but not available in CloudFormation.
Terraform uses AWS API, so you can get/update missing properties in many cases.
update_rollback_failed = contact customer service
---
Handle failures => Partial State and Error Handling
If an error happens at any stage in the lifecycle of a resource, Terraform stores a partial state of the resource. This behavior is critical for Terraform to ensure that you don't end up with any zombie resources: resources that were created by Terraform but no longer managed by Terraform due to a loss of state.

14. AWS CLOUDFORMATION DESIGNER

15. TERRAFORM GRAPH

16. 2.
TERRAFORM
Commands

17. TERRAFORM COMMANDS
$ terraform usage: terraform [--version] [--help] <command> [<args>] Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation version Prints the Terraform version

18. TERRAFORM COMMANDS
$ terraform usage: terraform [--version] [--help] <command> [<args>] Available commands are: apply Builds or changes infrastructure destroy Destroy Terraform-managed infrastructure get Download and install modules for the configuration graph Create a visual graph of Terraform resources init Initializes Terraform configuration from a module output Read an output from a state file plan Generate and show an execution plan refresh Update local state file against real resources remote Configure remote state storage show Inspect Terraform state or plan taint Manually mark a resource for recreation version Prints the Terraform version

19. TERRAFORM REMOTE Configures remote state storage with Terraform
AWS
infrastructure
Atlas, Consul,
S3,
etcd, HTTP
Atlas, Consul, etcd, S3 or HTTP
Terraform will automatically update remote state file once where are any changes in it.
There are also ways to pull and push to remote state file.
*.tf
terraform.tfstate

20. TERRAFORM PLAN Generates an execution plan for Terraform AWS
infrastructure
Refresh state locally and generate execution plan based on tf configs
*.tf
terraform.tfstate

21. TERRAFORM APPLY
Builds or changes infrastructure according to Terraform configuration files
AWS
infrastructure
Apply the changes required to reach the desired state of the configuration.
Or the pre-determined set of actions generated by a terraform plan execution plan.
*.tf
terraform.tfstate

22. TERRAFORM REFRESH
Update the state file of your infrastructure with metadata that matches the physical resources they are tracking
AWS
infrastructure
Refresh state locally and generate execution plan based on tf configs
*.tf
terraform.tfstate

23. TERRAFORM DESTROY Destroy Terraform-managed infrastructure
AWS
infrastructure
Destroy resources managed by Terraform.
Can be previewed by running “$ terraform plan -destroy”.
Also can be limited by using “-target” and it will delete resources which were dependent on targets.
*.tf
terraform.tfstate

24. TERRAFORM TAINT
Manually mark a resource as tainted, forcing a destroy and recreate on the next plan/apply
AWS
infrastructure
This will not modify your infrastructure.
This command changes your state to mark a resource as tainted so that during the next plan or apply, that resource will be destroyed and recreated.
*.tf
terraform.tfstate

25. TERRAFORM GRAPH
Draw nice visual dependency graph of Terraform resources according to configuration files
$ terraform graph -draw-cycles | dot -Tpng -o graph.png
This will not modify your infrastructure.
This command changes your state to mark a resource as tainted so that during the next plan or apply, that resource will be destroyed and recreated.

26. TERRAFORM etc
$ terraform --help

27. 3.
TERRAFORM
Warm up...

28. TERRAFORM - WARM-UP
Keep Terraform shared state files on Amazon S3 and enable bucket versioning:
aws s3api create-bucket \
--bucket my-terraform-states \
--acl authenticated-read \
--create-bucket-configuration LocationConstraint=eu-west-1
aws s3api put-bucket-versioning \
--versioning-configuration Status=Enabled

29. TERRAFORM WARM-UP QUESTIONS?
How many environments?
How many AWS regions?
How many DevOps will be involved?
It is not so important.

30. TERRAFORM
In action

31. TERRAFORM - DEMO 1
There was nothing in AWS account, so let’s create new VPC, subnets and deploy heavy web-app
Complete code and slides:

32. PROJECTS
/SHARED-AWS

33. PROJECTS
/HEAVY

34. TERRAFORM WAYS TO STRUCTURE CONFIGS
One-in-all:
Good for partial and disposable setups
Separate by environments:
One project = one environment
Each environment may contain different modules
Read more
Layered projects (shared infrastructure):
Separate responsibilities (eg, “read-only” shared infrastructure for app developers)
Easy to extend layers independently (using modules)
Small = fast

35. TERRAFORM HOW TO STRUCTURE CONFIGS
Keep 1 Terraform state for each combination of project per environment (in 1 AWS region)
eg, one-in-all = 1 Terraform state per environment
More environments = more combinations
Global AWS resources (eg, S3 buckets, EIP, IAM, Route53 zones, CodeDeploy applications):
Keep them in Terraform states without separation by environments
Use environment in resource tags
Use modules

36. TERRAFORM - MODULES
“Modules in Terraform are self-contained packages of Terraform configurations that are managed as a group.”
Support versioning:
Community modules -
module "vpc" {
source = "github.com/terraform-community-modules/tf_aws_vpc_only?ref=v1.0.0"
cidr = "${var.vpc_cidr}" }

37. TERRAFORM - DEMO 2 New resource!
Let’s import very important S3 bucket into Terraform configs, so that we can manage that resource using Terraform.
Explanation: Import of already created resources to Terraform state is not supported by Terraform natively, but it is possible.
New resource!

38. TERRAFORM - DEMO 3 Updated!
Our application ELB should contain custom security policy with specific set of SSL ciphers.
Explanation: “SSL ciphers” is not implemented as aws_elb resource type property.
Updated!

39. TERRAFORM - DEMO 4
Our Heavy application team needs Redshift cluster available, so that developers can query it.
Explanation: Redshift is not among supported resource types by Terraform, but it is supported by AWS CloudFormation.

40. TERRAFORM - DEMO SUMMARY
Terraform can create and manage AWS infrastructure which is:
New (has no resources)
Contains already existing resources
Terraform can:
Supplement resource types properties currently not supported natively
Supplement resource types currently not supported natively.

41. 4. TERRAFORM Demo: Continuous Integration &
Continuous Deployment (beta)

42. TERRAFORM - CI/CD Using feature branches Lock master branch
New push into feature branch:
terraform production init + plan
Feature merged into master branch:
terraform production init + plan + apply
Too risky? Combine:
terraform plan -out=plan_${GIT_SHA}.out
terraform apply plan_${GIT_SHA}.out
Terraform plugin for Jenkins, if you ask

43. Responsibly deploy applications and make changes to infrastructure with Atlas by HashiCorp
atlas.hashicorp.com

44. SUMMARY
Terraform is cool, isn’t it ?

45. I REALLY LIKE QUESTIONS

46. THANK YOU! All code from this talk: