1. E Camden Fisher Yale University ITS
Yale Build and Deploy
Sneak Preview
E Camden Fisher
Yale University ITS

2. Who am I?
E Camden Fisher Technical Lead Unix Infrastructure and

3. What is Jenkins?
Jenkins is an application and a framework that manages and monitors the execution of repeated tasks.

4. Who Uses

5. Overview Before Jenkins After Jenkins Building is slow, error prone
Testing is onerous
Code coverage is onerous
Bugs caught later
Devs worry about servers
No change control for deployments
Slow progress
Different artifact per environment
Inconsistent configuration per environment
Deployments are "hard"
Integration Hell
"fire and forget", consistent!
Testing is automated!
Code coverage is easy!
Bugs caught early and often!
Devs worry about code!
Change control in the right places for deployments!
Rapid progress! Agility!
Identical artifact per environment!
Identical configuration per environments!
Deployments are “click” easy
Integration Nirvana

6. Let’s Not Forget Continuous Integration!
Maintain a Single Source Repository
Automate the Build
Make Your Build Self-Testing
Every Commit Should Build the Mainline on an Integration Machine
Keep the Build Fast
Test in a Clone of the Production Environment
Make it Easy for Anyone to Get the Latest Executable
Everyone can see what's happening
Automate Deployment
Avoid "Integration Hell!!”

7. Why did we chose Jenkins?
Easy!
Extensible
Scalable
Flexible
Open Source
Supported!
Easy
Install: easy up and running with java -jar jenkins.war, or run it in a java container (like jboss, tomcat, etc)
Configure: web based UI, or web services endpoints to manage with other tools
Extensible:
hundreds of community plugins
ability to write our own plugins
web services API
Scalable:
built-in deployment and management of jenkins slaves
virtual machine management
Flexible:
runs on many different operating systems
can mix masters and slaves master on redhat linux 5, slaves running redhat linux 6, and windows

8. Yale History Lesson Application lifecycle is a progression
Source Code Management
Maven and Artifactory
Building and Testing with Jenkins
Container Configurable artifacts
Runtime Configurable Containers
Managed deployments with Jenkins

9. SCM Islands of SCM begat central SCM (subversion)
SCCS
RCS
VSS
CVS
cp –pr
Islands of SCM begat central SCM (subversion)
Single source repository!
Easy to get access.
Easy to onboard.
Easy to support.
Everyone wins!

10. MVN Maven Project Object Model (POM)
Simplifies dependency resolution ("oops I forgot that .jar!")
Makes the build process easy and uniform
Artifactory (Maven Repository)
Where do I put my built artifacts?
Makes it easy for everyone to get the latest build!

11. Builds: Before Jenkins
We have a single source repo + a place to store built artifacts, but…..
Builds still take a long time
Testing takes longer!
Code coverage take even longer!
Build environments are not standardized
Mistakes are caught later, hard to debug
Result: developers don't build, test or report on their code regularly enough because it's just too time consuming.

12. Builds: Enter Jenkins SCM commits automatically kick off a build
Testing and code coverage is automated and is run on every commit.
Broken builds immediately notify the team and the committer
Tests run in a Clone of the Production Environment!
Everyone can easily see what's happening!
Developers can get back to coding instead of building and testing.
Releases are quick and easy
Result: developers don't build, test or report on their code regularly enough because it's just too time consuming.

14. Deployment: The problem.
Configuration must be different per environment
Choice between loss of control or loss of agility
Often devs configure the container
Either devs can edit deployables or they can't
If they can... things are bad
Code is deployed, edited and removed w/out Change Control
Moving quickly… "I'll clean it up later!"
Dev environments quickly diverge from Production
Security is compromised
If they can't... things are worse
Change requests are "slow”
Systems groups must do everything

15. Deployments: How do we fix them?
Standardize the process!
Promote SANE change control
Normalize deployables/artifacts
Single location for app configuration data
Standardize the container
Externalize what makes an environment unique and special

16. Yale Application Installer Plugin
Standardize the process!
Why?
SSH creds + Delegation
Manage keys outside
Node name substitution
How?
Installer is co’d, scp’d
install.properties on dest
Pre-Install
SSH exec installer
Post-Install

17. Container Configurable Artifacts
Artifacts were built with embedded configuration.
ie. datasources, CAS service endpoints, etc
Artifacts are different per environment! What!?
"Oops, I forgot to update that parameter!"
Externalization of configuration parameters
deployable XML
Apps self configure with JNDI
Now we have Container Configurable artifacts
The SAME artifact migrates between environments
XML configuration data (stored as build parameters in Jenkins) is all that differentiates environments

18. Yale Standard Java Container
JBoss EAP (5.1 now)
Minimal customization to externalize configuration into runtime
Packaged in RPM
Configuration Management to install + manage what’s “special”
Meets 100% of use cases (so far)
Runtime configurable container is key!

19. Yale Standard Java Container
$JBOSS_HOME/deploy/jbossweb.sar/server.xml
<!-- Connector for SSL ECF -->
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="${jboss.bind.httpsport}" address="${jboss.bind.address}”
secure="true" clientAuth="false" scheme="https”
proxyName="${jboss.proxyname}" proxyPort="${jboss.proxyport}”
SSLCertificateFile="${jboss.server.home.dir}/conf/server.crt"
SSLCertificateKeyFile="${jboss.server.home.dir}/conf/server.pem”
URIEncoding="UTF-8” SSLProtocol="TLS”>
/etc/init.d/jboss_nodexx
# JBoss variables
export JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT} \
-Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}”
${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g ${JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR}

27. Deployments: The result.
Consistency!
Jenkins writes configuration XML: $human_error--
Eliminate shells, and elevated privileges on servers
Container is managed by infrastructure with the O/S
Developers can deploy to DEV at will
Empowers developers to GTD
Puts gates at appropriate places
Changes to the Jenkins jobs, containers + deploys to Test/Prod require change control
Frees Systems folks to work on more interesting things
Standard Container + Cloudy IaaS + Container Configurable artifacts = Vendor-lock-in-less PaaS!
Deployments to DEV look like deployments to TEST which look like deployments to PROD

28. Drupal Continuous Deployment
Yale has a large shared Drupal infrastructure
Shared = needs change control
Migration process looks similar to Java
Some end users want to edit themes!
Jenkins to the rescue!

33. What’s New? Workflow integration through web services
Build + Release of Apache Servicemix bundles and “features”
Spawning and Destroying Servicemix child instances
Deployment of Features and OSGI bundles to Apache Servicemix
Deploying Drupal 7
git + pull

34. Other Languages Php Ruby .Net Code coverage and unit testing available
Automated unit testing
Automated code coverage
Automated deployment coming soon
.Net
Build, unit test, archive creation

35. The Future RBAC + folders More ruby Enterprise Service Bus
delegate responsibility to other systems groups
More ruby
unit testing and code coverage
deployments
Enterprise Service Bus
Centralized SSH mangagement
the tools are better now
Managing/Provisioning with cfg mgmt
Cloud deployments

36. E Camden Fisher camden.fisher@yale.edu @fishnix
Questions?
E Camden