Compliance and Control of AWS Resources at Scale with Cloud Custodian AWS User Group Meetup – January 25, 2017 Compliance and Control of AWS Resources at Scale with Cloud Custodian Automate Compliance/Governance Author: Mark Cwetna Title: Consultant mark.cwetna@slalom.com

Geronimo – aka: "the one who yawns"

Problem Statement(s): How do I secure resources at cloud-scale? How do I apply/manage governance in the cloud? How do I control deployment of resources? Where is the accountability when running at cloud-scale?

Governance In the Cloud Cost management/allocation Cost optimization Custom EC2 scheduling Cleanup Think garbage-collection (unused/underutilized EC2/DBs can be incinerated) Tagging (enforce tagging on all AWS resources) Billing Resource management/control Compliance Spinning up EC2 instances with public IPs Close security group port vulnerabilities Operations Discover service limits before you hit them

Welcome Cloud Custodian Capital One created Cloud Custodian to help solve the problems associated with cloud-scale compliance of AWS resources and to enforce governance

How Do They Do It?? Created a rules-based engine in Lambda (MU) Template control with YAML semantic control structures Targetable to AWS accounts/regions STS cross-account Role support Consumption of Cloudtrail logs by Lambda MU engine for robust accountability Stateless design structure using serverless infrastructure Multi-step workflows

AWS Resource Support - account - directory - healthcheck - launch-config - route-table - distribution - hostedzone - log-group - security-group - acm-certificate - dynamodb-table - iam-certificate - network-acl - simpledb - alarm - ebs - iam-group - network-addr - sns - ami - ebs-snapshot - iam-policy - peering-connection - sqs - app-elb - ecr - iam-profile - rds - streaming-distribution - app-elb-target-group - ecs - iam-role - rds-cluster - subnet - asg - efs - iam-user - rds-cluster-snapshot - vpc - cache-cluster - elasticsearch - internet-gateway - rds-snapshot - vpn-connection - cache-snapshot - elb - key-pair - rds-subnet-group - vpn-gateway - cache-subnet-group - emr - kinesis - rds-subscription - waf - cfn - eni - kinesis-analytics - redshift - cloudsearch - event-rule - kms - redshift-snapshot - customer-gateway - firehose - kms-key - redshift-subnet-group - glacier - lambda - rest-api

Core Components Policies Resources (ec2, asg, s3, elb, s3, etc) Modes (poll - cached, CWE, periodic – non-cached, Config Rules) type events Filters (JMESpath and nesting support) type (Cloudwatch events support) key Value tag Actions value Force

CWE (Cloud Watch Events) Subscribe to any API supported by Cloudtrail Continuously scans Cloudtrail logs Replays Cloudtrails logs at a much lower latency (polling every 90 seconds at 99th percentile) Cloudtrail latency into S3 can vary by up to 15 min Caching supported to minimize API calls Queries current state of an event’s resource Leverages SSM (Simple Systems Manager)

YAML DSL Driven Templates

Let’s Get Going pip install c7n > custodian -h usage: custodian [-h] {report,logs,metrics,version,validate,schema,run} ... Cloud fleet management positional arguments:   {report,logs,metrics,version,validate,schema,run}     report              CSV report of resources that a policy matched/ran on     logs                Get policy execution logs from s3 or cloud watch logs     metrics             Retrieve metrics for policies from CloudWatch Metrics     version             Display installed version of custodian     validate            Validate config files against the custodian jsonschema     schema              Interactive cli docs for policy authors     run                 Execute the policies in a config file

CLI Options run [-h] [-r REGION] [--profile PROFILE]                      [--assume ASSUME_ROLE] -c CONFIG [-p POLICY_FILTER]                      [-t RESOURCE_TYPE] [-v] [-l LOG_GROUP] -s OUTPUT_DIR                      [-f CACHE] [--cache-period CACHE_PERIOD] [-d] [-m] schema validate version metrics logs report -m (cloud watch metrics) -l <log_group> --assume-role (cross account) --dry-run (run filters only) -h

CLI Driven Results > custodian run --dryrun -c ec2_public_instance.yml -s out --metrics -- log-group=/cloud-custodian/testing/us-east-2 2017-01-24 18:36:47,690: custodian.policy:INFO Running policy ec2- interogate-instances resource: ec2 region:us-east-1 c7n:0.8.21.2 2017-01-24 18:36:49,078: custodian.resources.ec2:INFO Filtered from 18 to 0 ec2 2017-01-24 18:36:49,078: custodian.policy:INFO policy: ec2-interogate- instances resource:ec2 has count:0 time:1.34

Terminate Unused DBs

Always Do a Dry Run! > custodian run --dryrun -c public_elb.yml -s out --metrics --log- group=/cloud-custodian/testing/us-east-1 2017-01-24 22:59:23,636: custodian.policy:INFO Running policy public- elb-exposed-instance-roles resource: elb region:us-east-1 c7n:0.8.21.2 2017-01-24 22:59:23,691: custodian.resources.ec2:INFO Filtered from 22 to 22 ec2 2017-01-24 22:59:23,692: custodian.resources.elb:INFO Filtered from 4 to 0 elb 2017-01-24 22:59:23,692: custodian.policy:INFO policy: public-elb- exposed-instance-roles resource:elb has count:0 time:0.01

Scary Live Demo

Auto tag resources > custodian run -r us-west-2 --profile slalom --dryrun -c tag_auto_owner.yml -s out --metrics --log-group=/cloud- custodian/slalom/us-east-2 2017-01-25 10:20:11,136: custodian.policy:INFO Running policy ec2- auto-tag-owner resource: ec2 region:us-west-2 c7n:0.8.21.2 2017-01-25 10:20:11,902: custodian.resources.ec2:INFO Filtered from 6 to 6 ec2 2017-01-25 10:20:11,902: custodian.policy:INFO policy: ec2-auto-tag- owner resource:ec2 has count:6 time:0.72

Stop EC2 Instances Tagged with Custodian Dry Run > custodian run -r us-west-2 --profile slalom --dryrun --output-dir=out --config=custodian.yml 2017-01-25 10:37:22,904: custodian.policy:INFO Running policy my- first-policy resource: ec2 region:us-west-2 c7n:0.8.21.2 2017-01-25 10:37:23,566: custodian.resources.ec2:INFO Filtered from 6 to 3 ec2 2017-01-25 10:37:23,566: custodian.policy:INFO policy: my-first-policy resource:ec2 has count:3 time:0.66

Stop EC2 Instances Tagged with Custodian > custodian run -r us-west-2 --profile slalom --output-dir=out -- config=custodian.yml 2017-01-25 10:39:57,779: custodian.policy:INFO Running policy my-first- policy resource: ec2 region:us-west-2 c7n:0.8.21.2 2017-01-25 10:39:57,788: custodian.resources.ec2:INFO Filtered from 6 to 3 ec2 2017-01-25 10:39:57,788: custodian.policy:INFO policy: my-first-policy resource:ec2 has count:3 time:0.01 2017-01-25 10:39:57,789: custodian.actions:INFO Stop 3 of 3 instances 2017-01-25 10:39:58,508: custodian.policy:INFO policy: my-first-policy action: stop resources: 3 execution_time: 0.72

Supplemental Cloud Custodian, a serverless rules engine for the cloud - Kapil Thangavelu Compliance Architecture: How Capital Automates the Guard Rails for 6000 Developers Capital One Cloud Custodian Documentation Cloud Custodian Github Repo CloudWatch Metrics

QUESTIONS??