1. Compliance and Control of AWS Resources at Scale with Cloud Custodian
AWS User Group Meetup – January 25, 2017
Compliance and Control of AWS Resources at Scale with Cloud Custodian
Automate Compliance/Governance
Author: Mark Cwetna
Title: Consultant

2. Geronimo – aka: "the one who yawns"

3. Problem Statement(s):
How do I secure resources at cloud-scale?
How do I apply/manage governance in the cloud?
How do I control deployment of resources?
Where is the accountability when running at cloud-scale?

4. Governance In the Cloud
Cost management/allocation
Cost optimization
Custom EC2 scheduling
Cleanup
Think garbage-collection (unused/underutilized EC2/DBs can be incinerated)
Tagging (enforce tagging on all AWS resources)
Billing
Resource management/control
Compliance
Spinning up EC2 instances with public IPs
Close security group port vulnerabilities
Operations
Discover service limits before you hit them

5. Welcome Cloud Custodian
Capital One created Cloud Custodian to help solve the problems associated with cloud-scale compliance of AWS resources and to enforce governance

6. How Do They Do It?? Created a rules-based engine in Lambda (MU)
Template control with YAML semantic control structures
Targetable to AWS accounts/regions
STS cross-account Role support
Consumption of Cloudtrail logs by Lambda MU engine for robust accountability
Stateless design structure using serverless infrastructure
Multi-step workflows

8. AWS Resource Support - account - directory - healthcheck
- launch-config
- route-table
- distribution
- hostedzone
- log-group
- security-group
- acm-certificate
- dynamodb-table
- iam-certificate
- network-acl
- simpledb
- alarm
- ebs
- iam-group
- network-addr
- sns
- ami
- ebs-snapshot
- iam-policy
- peering-connection
- sqs
- app-elb
- ecr
- iam-profile
- rds
- streaming-distribution
- app-elb-target-group
- ecs
- iam-role
- rds-cluster
- subnet
- asg
- efs
- iam-user
- rds-cluster-snapshot
- vpc
- cache-cluster
- elasticsearch
- internet-gateway
- rds-snapshot
- vpn-connection
- cache-snapshot
- elb
- key-pair
- rds-subnet-group
- vpn-gateway
- cache-subnet-group
- emr
- kinesis
- rds-subscription
- waf
- cfn
- eni
- kinesis-analytics
- redshift
- cloudsearch
- event-rule
- kms
- redshift-snapshot
- customer-gateway
- firehose
- kms-key
- redshift-subnet-group
- glacier
- lambda
- rest-api

9. Core Components Policies Resources (ec2, asg, s3, elb, s3, etc)
Modes (poll - cached, CWE, periodic – non-cached, Config Rules)
type
events
Filters (JMESpath and nesting support)
type (Cloudwatch events support)
key
Value
tag
Actions
value
Force

10. CWE (Cloud Watch Events)
Subscribe to any API supported by Cloudtrail
Continuously scans Cloudtrail logs
Replays Cloudtrails logs at a much lower latency (polling every 90 seconds at 99th percentile)
Cloudtrail latency into S3 can vary by up to 15 min
Caching supported to minimize API calls
Queries current state of an event’s resource
Leverages SSM (Simple Systems Manager)

11. YAML DSL Driven Templates

12. Let’s Get Going pip install c7n > custodian -h
usage: custodian [-h] {report,logs,metrics,version,validate,schema,run} ...
Cloud fleet management
positional arguments:
  {report,logs,metrics,version,validate,schema,run}
    report              CSV report of resources that a policy matched/ran on
    logs                Get policy execution logs from s3 or cloud watch logs
    metrics             Retrieve metrics for policies from CloudWatch Metrics
    version             Display installed version of custodian
    validate            Validate config files against the custodian jsonschema
    schema              Interactive cli docs for policy authors
    run                 Execute the policies in a config file

13. CLI Options run [-h] [-r REGION] [--profile PROFILE]
                     [--assume ASSUME_ROLE] -c CONFIG [-p POLICY_FILTER]
                     [-t RESOURCE_TYPE] [-v] [-l LOG_GROUP] -s OUTPUT_DIR
                     [-f CACHE] [--cache-period CACHE_PERIOD] [-d] [-m]
schema
validate
version
metrics
logs
report
-m (cloud watch metrics)
-l <log_group>
--assume-role (cross account)
--dry-run (run filters only) -h

14. CLI Driven Results
> custodian run --dryrun -c ec2_public_instance.yml -s out --metrics -- log-group=/cloud-custodian/testing/us-east :36:47,690: custodian.policy:INFO Running policy ec2- interogate-instances resource: ec2 region:us-east-1 c7n: :36:49,078: custodian.resources.ec2:INFO Filtered from 18 to 0 ec :36:49,078: custodian.policy:INFO policy: ec2-interogate- instances resource:ec2 has count:0 time:1.34

15. Terminate Unused DBs

16. Always Do a Dry Run!
> custodian run --dryrun -c public_elb.yml -s out --metrics --log- group=/cloud-custodian/testing/us-east :59:23,636: custodian.policy:INFO Running policy public- elb-exposed-instance-roles resource: elb region:us-east-1 c7n: :59:23,691: custodian.resources.ec2:INFO Filtered from 22 to 22 ec :59:23,692: custodian.resources.elb:INFO Filtered from 4 to 0 elb :59:23,692: custodian.policy:INFO policy: public-elb- exposed-instance-roles resource:elb has count:0 time:0.01

17. Scary Live Demo

18. Auto tag resources
> custodian run -r us-west-2 --profile slalom --dryrun -c tag_auto_owner.yml -s out --metrics --log-group=/cloud- custodian/slalom/us-east :20:11,136: custodian.policy:INFO Running policy ec2- auto-tag-owner resource: ec2 region:us-west-2 c7n: :20:11,902: custodian.resources.ec2:INFO Filtered from 6 to 6 ec :20:11,902: custodian.policy:INFO policy: ec2-auto-tag- owner resource:ec2 has count:6 time:0.72

19. Stop EC2 Instances Tagged with Custodian Dry Run
> custodian run -r us-west-2 --profile slalom --dryrun --output-dir=out --config=custodian.yml :37:22,904: custodian.policy:INFO Running policy my- first-policy resource: ec2 region:us-west-2 c7n: :37:23,566: custodian.resources.ec2:INFO Filtered from 6 to 3 ec :37:23,566: custodian.policy:INFO policy: my-first-policy resource:ec2 has count:3 time:0.66

20. Stop EC2 Instances Tagged with Custodian
> custodian run -r us-west-2 --profile slalom --output-dir=out -- config=custodian.yml
:39:57,779: custodian.policy:INFO Running policy my-first- policy resource: ec2 region:us-west-2 c7n:
:39:57,788: custodian.resources.ec2:INFO Filtered from 6 to 3 ec2
:39:57,788: custodian.policy:INFO policy: my-first-policy resource:ec2 has count:3 time:0.01
:39:57,789: custodian.actions:INFO Stop 3 of 3 instances
:39:58,508: custodian.policy:INFO policy: my-first-policy action: stop resources: 3 execution_time: 0.72

21. Supplemental
Cloud Custodian, a serverless rules engine for the cloud - Kapil Thangavelu
Compliance Architecture: How Capital Automates the Guard Rails for Developers
Capital One Cloud Custodian Documentation
Cloud Custodian Github Repo
CloudWatch Metrics

22. QUESTIONS??