©2013 CloudBees, Inc. All Rights Reserved 1 Next Step in Automation: Elastic Build Environment Kohsuke Kawaguchi / CloudBees, Inc. Jesse Glick / CloudBees, Inc.
©2013 CloudBees, Inc. All Rights Reserved 2 Have You Met Jenkins?
©2013 CloudBees, Inc. All Rights Reserved 3
4
5
6 My Jenkins around 2006
©2013 CloudBees, Inc. All Rights Reserved 7
8
9
©2013 CloudBees, Inc. All Rights Reserved 10 If only we had more computers… Just building & testing them all… Running tests more frequently Testing individual commits
©2013 CloudBees, Inc. All Rights Reserved 11
©2013 CloudBees, Inc. All Rights Reserved 12 Just enough computers just in time Elasticity
©2013 CloudBees, Inc. All Rights Reserved 13 My Jenkins around 2007
©2013 CloudBees, Inc. All Rights Reserved 14
©2013 CloudBees, Inc. All Rights Reserved 15
©2013 CloudBees, Inc. All Rights Reserved 16
©2013 CloudBees, Inc. All Rights Reserved 17
©2013 CloudBees, Inc. All Rights Reserved 18
©2013 CloudBees, Inc. All Rights Reserved 19 Just enough computers of the right kind just in time Elasticity!
©2013 CloudBees, Inc. All Rights Reserved 20
©2013 CloudBees, Inc. All Rights Reserved 21 Correct answer Test assumes a fixture running on port 8080 –Doesn’t check if it’s already being used If another test runs at the same time…?
©2013 CloudBees, Inc. All Rights Reserved 22
©2013 CloudBees, Inc. All Rights Reserved 23 Correct answer Because of “pkill -f -9 tomcat” cleanup
©2013 CloudBees, Inc. All Rights Reserved 24
©2013 CloudBees, Inc. All Rights Reserved 25 Isolation At odds with large multi-core systems x86 virtual machines User isolation Kernel containers
©2013 CloudBees, Inc. All Rights Reserved 26
©2013 CloudBees, Inc. All Rights Reserved 27
©2013 CloudBees, Inc. All Rights Reserved 28 Correct answer Same Maven ID, two different jars Different projects designate different ones Local cache gets cleaned up periodically Whichever first runs after cache cleanup “wins”
©2013 CloudBees, Inc. All Rights Reserved 29
©2013 CloudBees, Inc. All Rights Reserved 30 Correct answer Test script leaves background daemon process behind Over time it’ll slowly choke slaves
©2013 CloudBees, Inc. All Rights Reserved 31
©2013 CloudBees, Inc. All Rights Reserved 32 Throw away & create new Elasticity!
©2013 CloudBees, Inc. All Rights Reserved 33
©2013 CloudBees, Inc. All Rights Reserved 34 Ladder to Cloud
©2013 CloudBees, Inc. All Rights Reserved 35 Solid OSS Elasticity Plugins EC2 plugin Jclouds plugin –OpenStack, CloudStack Launch and tear down slaves on demand
©2013 CloudBees, Inc. All Rights Reserved 36 VMWare auto-scaling plugin Snapshot Power on-off management Hypervisor-aware scheduling Folder based pooling VMWare tools integration One-time use support
©2013 CloudBees, Inc. All Rights Reserved 37 Host that runs Docker Docker plugin
©2013 CloudBees, Inc. All Rights Reserved 38 Mansion Slave CloudBees
©2013 CloudBees, Inc. All Rights Reserved 39 Maven Git Ant Mercurial Gradle Subversion Linux Kernel Hardware Linux Container = zero cost virtualization
©2013 CloudBees, Inc. All Rights Reserved 40 For OS X Maven Git XCode Git XCode Subversion OS X QEMU Linux Kernel Apple Hardware
©2013 CloudBees, Inc. All Rights Reserved 41 Kernel Same-page Merging OS X
©2013 CloudBees, Inc. All Rights Reserved 42
©2013 CloudBees, Inc. All Rights Reserved 43 Mansion Slave Workspace 1 Workspace 2 Workspace 3 Workspace 4
©2013 CloudBees, Inc. All Rights Reserved 44 Slave Workspace Workspace’’ Workspace’
©2013 CloudBees, Inc. All Rights Reserved 45 Slave ~/.m2/repository
©2013 CloudBees, Inc. All Rights Reserved 46 Parallel Testing Test Group #1Test Group #2Test Group #3
©2013 CloudBees, Inc. All Rights Reserved 47 Parallel Testing foo #10 Test Group #1 foo #11 Test Group #2 foo #12 Test Goup #3
©2013 CloudBees, Inc. All Rights Reserved 48 Validated Merge upstream repo upstream repo gate repo gate repo
©2013 CloudBees, Inc. All Rights Reserved 49
©2013 CloudBees, Inc. All Rights Reserved 50 Workflow System Alternative to “freestyle” projects Scripted control flow Resumable execution across restarts All-in-one build/test/deploy pipelines Under active development
©2013 CloudBees, Inc. All Rights Reserved 51 Workflow with Elastic Slaves One-line provisioning from cloud Language-level parallelism Run commands, archive files, test results Now integrates with parallel test plugin
©2013 CloudBees, Inc. All Rights Reserved 52
©2013 CloudBees, Inc. All Rights Reserved 53 Conclusion: Elasticity Benefits Just-in-time capacity Diversity without overhead Isolation Productivity gain –parallel testing –validated merge –workflow
©2013 CloudBees, Inc. All Rights Reserved 54
©2013 CloudBees, Inc. All Rights Reserved 55 © Kurt Löwenstein Educational Center International Team Creative Commons Attribution 2.0 Generic
©2013 CloudBees, Inc. All Rights Reserved 56
©2013 CloudBees, Inc. All Rights Reserved 57 Device Cloud
©2013 CloudBees, Inc. All Rights Reserved 58