Transcript Yale Build and Deploy
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Yale Build and Deploy
E Camden Fisher Yale University ITS
Jenkins User Conference
Who am I?
New York, May 17 2012 #jenkinsconf
E Camden Fisher Technical Lead Unix Infrastructure and Virtualization [email protected]
@fishnix
Jenkins User Conference
What is Jenkins?
New York, May 17 2012 #jenkinsconf Jenkins is an application and a framework that manages and monitors the execution of repeated tasks.
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Who Uses Jenkins @Yale?
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Overview
• • • • • • • • • • •
Before Jenkins
Building is slow, error prone Testing is onerous Code coverage is onerous Bugs caught later Devs worry about servers No change control for deployments Slow progress Different artifact per environment Inconsistent configuration per environment Deployments are "hard" Integration Hell • • • • • • • • • • •
After Jenkins
"fire and forget", consistent!
Testing is automated!
Code coverage is easy!
Bugs caught early and often!
Devs worry about code!
Change control in the right places for deployments!
Rapid progress! Agility!
Identical artifact per environment!
Identical configuration per environments!
Deployments are “click” easy Integration Nirvana
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Let’s Not Forget Continuous Integration!
Maintain a Single Source Repository Automate the Build Make Your Build Self-Testing Every Commit Should Build the Mainline on an Integration Machine Keep the Build Fast Test in a Clone of the Production Environment Make it Easy for Anyone to Get the Latest Executable Everyone can see what's happening Automate Deployment Avoid "Integration Hell!!”
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Why did we chose Jenkins?
Easy!
Extensible Scalable Flexible Open Source Supported!
Jenkins User Conference
Yale History Lesson
New York, May 17 2012 #jenkinsconf
Application lifecycle is a progression Source Code Management Maven and Artifactory Building and Testing with Jenkins Container Configurable artifacts Runtime Configurable Containers Managed deployments with Jenkins
Jenkins User Conference
SCM
New York, May 17 2012 #jenkinsconf
cp –pr SCCS RCS
Subversion
CVS VSS
Jenkins User Conference
MVN
New York, May 17 2012 #jenkinsconf
Maven
Project Object Model (POM) Simplifies dependency resolution ("oops I forgot that .jar!") Makes the build process easy and uniform
Artifactory (Maven Repository)
Where do I put my built artifacts?
Makes it easy for everyone to get the latest build!
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Builds: Before Jenkins
We have a single source repo + a place to store built artifacts, but…..
– Builds still take a long time – Testing takes longer!
– Code coverage take even longer!
– Build environments are not standardized – Mistakes are caught later, hard to debug
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Builds: Enter Jenkins
SCM commits automatically kick off a build Testing and code coverage is automated and is run on every commit.
Broken builds immediately notify the team and the committer Tests run in a Clone of the Production Environment!
Everyone can easily see what's happening!
Developers can get back to coding instead of building and testing.
Releases are quick and easy
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Deployment: The problem.
Configuration must be different per environment Choice between loss of control or loss of agility – – Often devs configure the container Either devs can edit deployables or they can't If they can... things are bad – – – – Code is deployed, edited and removed w/out Change Control Moving quickly… "I'll clean it up later!" Dev environments quickly diverge from Production Security is compromised If they can't... things are worse – – Change requests are "slow” Systems groups must do everything
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Deployments: How do we fix them?
Standardize the process!
Promote SANE change control Normalize deployables/artifacts Single location for app configuration data Standardize the container Externalize what makes an environment unique and special
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Yale Application Installer Plugin
Standardize the process!
Why?
– – – SSH creds + Delegation Manage keys outside Node name substitution How?
– – – – – Installer is co’d, scp’d install.properties on dest Pre-Install SSH exec installer Post-Install
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Container Configurable Artifacts
Artifacts were built with embedded configuration.
– – – ie. datasources, CAS service endpoints, etc Artifacts are different per environment! What!?
"Oops, I forgot to update that parameter!" Externalization of configuration parameters – – deployable XML Apps self configure with JNDI Now we have Container Configurable artifacts – – The SAME artifact migrates between environments XML configuration data (stored as build parameters in Jenkins) is all that differentiates environments
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Yale Standard Java Container
JBoss EAP (5.1 now) Minimal customization to externalize configuration into runtime Packaged in RPM Configuration Management to install + manage what’s “special” Meets 100% of use cases (so far) Runtime configurable container is key!
Jenkins User Conference
New York, May 17 2012 #jenkinsconf
Yale Standard Java Container
$JBOSS_HOME/deploy/jbossweb.sar/server.xml
${jboss.proxyname} " proxyPort=" ${jboss.proxyport} ” SSLCertificateFile=" ${jboss.server.home.dir} /conf/server.crt" SSLCertificateKeyFile=" ${jboss.server.home.dir} /conf/server.pem” URIEncoding="UTF 8” SSLProtocol="TLS”> /etc/init.d/jboss_nodexx # JBoss variables export JBOSS_OPTS=“-Djboss.bind.httpport=${HTTPPORT} \ Djboss.bind.httpsport=${HTTPSPORT} ${ADDLJBOPTS}” ${JBOSS_HOME}/bin/run.sh ${JBOSS_OPTS} -c ${JBOSS_NODE} -g ${JBOSS_PARTITION} -b ${JBOSS_BIND} -u ${CLSTR_ADDR} Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Jenkins User Conference New York, May 17 2012 #jenkinsconf Consistency! Jenkins writes configuration XML: $human_error- Eliminate shells, and elevated privileges on servers Container is managed by infrastructure with the O/S Developers can deploy to DEV at will Empowers developers to GTD Puts gates at appropriate places Changes to the Jenkins jobs, containers + deploys to Test/Prod require change control Frees Systems folks to work on more interesting things Standard Container + Cloudy IaaS + Container Configurable artifacts = Vendor-lock-in-less PaaS! Jenkins User Conference New York, May 17 2012 #jenkinsconf Yale has a large shared Drupal infrastructure Shared = needs change control Migration process looks similar to Java Some end users want to edit themes! Jenkins to the rescue! Jenkins User Conference New York, May 17 2012 #jenkinsconf Workflow integration through web services Build + Release of Apache Servicemix bundles and “features” Spawning and Destroying Servicemix child instances Deployment of Features and OSGI bundles to Apache Servicemix Deploying Drupal 7 – git + pull Jenkins User Conference New York, May 17 2012 #jenkinsconf Php – Code coverage and unit testing available Ruby – – – Automated unit testing Automated code coverage Automated deployment coming soon .Net – Build, unit test, archive creation Jenkins User Conference New York, May 17 2012 #jenkinsconf RBAC + folders – delegate responsibility to other systems groups More ruby – – unit testing and code coverage deployments Enterprise Service Bus Centralized SSH mangagement – the tools are better now Managing/Provisioning with cfg mgmt Cloud deployments Jenkins User Conference New York, May 17 2012 #jenkinsconf E Camden Fisher [email protected] @fishnixDeployments: The result.
Drupal Continuous Deployment
What’s New?
Other Languages
The Future
Questions?