Download presentation
Presentation is loading. Please wait.
Published byJordan Beasley Modified over 9 years ago
1
©2013 CloudBees, Inc. All Rights Reserved 1 Jenkins Scalability Summit Oct 2013
2
©2013 CloudBees, Inc. All Rights Reserved 2 Logistics Where is …? Network
3
©2013 CloudBees, Inc. All Rights Reserved 3 Agenda Part 1: Story time Learn from our collective experience Identify high-priority problems What is hurting serious users? Concrete details are good
4
©2013 CloudBees, Inc. All Rights Reserved 4 Agenda Part 2: Discuss & Design Collectively pick a few topics Split into 2 tracks See if we can start shaping up solutions Please keep shared notes –Details!
5
©2013 CloudBees, Inc. All Rights Reserved 5 Possible topics includes … CPU/memory/disk/network/… consumption Stability/diagnosability of slaves Access control of builds/UI Organizing jobs and build records Master to master communication Workflow / choreography User interface Stability of Jenkins releases Plugin compatibilities
6
©2013 CloudBees, Inc. All Rights Reserved 6 Goals No one goes listen-only mode Produce notes to show to the broader community Figure out how to do this better next year
7
©2013 CloudBees, Inc. All Rights Reserved 7 What We’ve Done & What We Can
8
©2013 CloudBees, Inc. All Rights Reserved 8 Thread Reduction Per-slave consumption –SSH (down to 1) –Channel (down to 1) Executor thread on demand
9
©2013 CloudBees, Inc. All Rights Reserved 9 Thread Reduction: TODO NIO Asynchronous job execution –More about this in workflow
10
©2013 CloudBees, Inc. All Rights Reserved 10 Memory reduction Lazy loading of build records Database plugin
11
©2013 CloudBees, Inc. All Rights Reserved 11 Design Choices That Constrain Us Compatibility –Adding is doable, changing is often hard Heterogeneousness in data model Thread-driven execution model
12
©2013 CloudBees, Inc. All Rights Reserved 12 ©2013 CloudBees, Inc. All Rights Reserved CloudBees Scalability Efforts
13
©2013 CloudBees, Inc. All Rights Reserved 13 Multi-master / Meta Jenkins Master-to-master channel Extensions on top of it –“Cloud” impl to lease slaves –Push security realm –Push update center –…
14
©2013 CloudBees, Inc. All Rights Reserved 14 Multi-master Where we think we can go with this –Move jobs around –Sharding with UI mashup Where this wouldn’t take us –Loss of inflight builds –True horizontal scaling
15
©2013 CloudBees, Inc. All Rights Reserved 15 Workflow Brand new job type Kill multiple birds in one stone Inspired by buildflow & jenkow
16
©2013 CloudBees, Inc. All Rights Reserved 16 Ingredients No single thread to execute the whole thing –Analogous to NIO –Check pointing BPMN-like workflow execution model –Surface syntax independent Groovy DSL –With continuation-passing style execution model Post-execution visualization –But no pre-execution visualization
17
©2013 CloudBees, Inc. All Rights Reserved 17 Ingredients Unified properties, build variables, and environments Open up direct access to file store Hands-free process forking Interop with existing job types
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.