CONTINUOUS INTEGRATION WITH JENKINSAUTOMATING YOURSELF OUT OF A JOB

(ONE THAT YOU DON'T WANT)Created by Lloyd Benson / lloyd.benson@gmail.com

http://github.com/lloydbenson/presentations

http://github.com/lloydbenson/demo-jenkins

WHOAMIlloyd benson (lloyd.benson@gmail.com)@walmartlabs node teammany hats (devops/architect/sysadmin/web developer/gluerof all things)the guy that did the black friday node release

GOALSget you thinking about continuous integrationtalk about a free open source solution (jenkins)cover featuresdemonstrate usefulnessbest practicesadditional adviceadvanced topics

WHAT IS CONTINUOUS INTEGRATION TO YOU?automated feedback system on test failures / blamecommitters!small autonomous changes to easily to tell what brokecontinuous deployment to qa or prod?reporting / trending of environments and project tests

A software methodology in which you can do small sets ofchanges that are automatically tracked, tested, and potentially

deployed if successful from a commit. If unsuccessful, it providesfeedback in order to immediately revert or fix the problem

quickly. It should keep historical trending of failures so you canstart making better decisions about general software

management.

WHAT CONTINUOUS INTEGRATION MEANS TO ME

ADVANTAGES OF CI VS TRADITIONAL INTEGRATIONearly detection of issues due to immediate testingrevert codebase to a bug-free state without wasting timedebuggingavoid last-minute chaos at release datesavoid merge issue mess (due to slightly incompatible versionsin everyones branches)easy isolation of bugs with smaller change sets

ADVANTAGES OF CI (CONT)constant availability of a "current" build for testing, demo, orrelease purposesimmediate feedback to developers on the quality,functionality, or system-wide impact of code they are writingforces more modular, less complex code due to frequent codepushesproduces higher quality code due to metrics generated (suchas code coverage, code complexity, unit/integration tests)

DISADVANTAGES OF CIcould be too much overhead if you have a very small appci is only as good as your tests!it takes alot of commitment to keep up with testsci software is always changing (like any software) and canbreak during upgrades. The irony!

WHAT IS JENKINS?jenkins is open source continuous integration tool written injavaa tool with a GUI that monitors executions of repeated jobs,such as building a software project or jobs run by cron.See for more details.https://jenkins-ci.org

JENKINS HISTORYjenkins was originally developed as the hudson projecthudson's creation started in summer of 2004 at sunmicrosystemsthe project was forked from hudson after a dispute with oracle

WHY USE JENKINS?take advantage of ci methodologiesabstract scripts to a GUI interfacebuilt in restful apiextensibility (hooks and plugins to have it work how you needit to)do not have to know java to effectively use it

WHY NOT USE JENKINS?if you hate java and all it touches and ruby is the best, then thisis likely not the tool for youyou already have a tool that takes care of your needs and donot want to add complexityyou have a tiny little project that will never growyou have deep respect for butlers and find the logo offensive

JENKINS FEATURESeasy installationeasy configurationchange set supportpermanent linksfeedback integration

after-the-fact taggingtest reportingdistributed buildsfile fingerprintingplugin support

EASY INSTALLATIONjava -jar jenkins.war, or deploy it in a servlet container. It is a flat

file structure, no database to setup.

EASY CONFIGURATIONjenkins can be configured entirely from the web GUI. You don'tneed tweak XML files, although for automation or configuration

management, you will still be manipulating them.

CHANGE SET SUPPORTjenkins can generate a list of changes from the previous build.

PERMANENT LINKSjenkins gives you clean readable URLs for most of its pages,

including some permalinks like "latest build" / "latest successfulbuild", so that they can be easily linked from elsewhere.

FEEDBACK INTEGRATIONMonitor build results by RSS, e-mail, IM to get real-time

notifications on failures.

AFTER-THE-FACT TAGGINGBuilds can be tagged (to source control) after builds are

completed.

TEST REPORTINGJUnit/TestNG test reports can be tabulated, summarized, and

displayed with historical information. historical trending isplotted into a graph.

DISTRIBUTED BUILDSjenkins can distribute build/test loads to multiple computers. This

allows you to do OS specific builds, increase the number of jobsyou can run, or idle workstations sitting beneath your desk.

FILE FINGERPRINTINGjenkins can keep track of which build produced which artifacts.This works even for artifacts that are produced outside jenkins,

and is ideal for projects to track dependency.

PLUGIN SUPPORTjenkins can be extended via 3rd party plugins. You can write your

own plugins though you DO need to know java for this.

DEVELOPMENT FLOW (PRE-DEMO)fork project (lloyddemo user)do a change and commit it to your own repo (lloyddemo user)submit a PR to the original repo (lloyddemo user)merge in PR if green (lloydbenson user)automatically test, build, and deploy your app (jenkins usinglloyddemo user for PR and lloydbenson user for credentials foreverything else)

DEMO TIMEenough lecture already!hate watching demos and want to see what it does?

prepare, start, configure oh myyytest that when successful, jenkins automatically builds anddeploysshow pull request integration with github that uses the githubapi to show statuswith automation, there is more time for boring lecture!

http://github.com/lloydbenson/demo-jenkins

HOW DID THE DEMO WORK?lloyddemo user has pull/push privs on repo where PR is done.With this access, the ghprb plugin utilizes the github API via akey in order to use change status to pending and then finallypass or fail. It will also use the jenkins URL you defined in yourconfiguration for the link.demo.test and demo.pr do a poll every 1 min (do notrecommend doing that but useful for demo and onlytemporary).

NAVIGATING JENKINSconfig / securityupdates / pluginssimple / multi-config jobscm setupbuild section

triggers / chainingconsole outputtrendingdashboardchange sets

There are numerous best practices.You can decide what is best for you or your organization.

BEST PRACTICES

SECURITYalways secure jenkins (it is not by default).AD / LDAP / local users (I had issues with the latest activedirectory plugin)by default anyone (anonymous) can do anything which meansthey will do anything!you may have sensitive data like apikeysaudit trail plugin is useful for tracking jenkins changes

BACKUPbackup jenkins regularlyeasy to backup jenkins because of flat filesmake a backup jenkins job!especially important for upgrades

FINGERPRINTINGuse file fingerprinting to manage dependencies

SOURCE CONTROL MANAGEMENT (SCM)fully use source control (should be obvious these days)reliable builds will be clean builds (avoids typical "well it workson my machine" developer mantra)track your changes and easily revertgood communication mechanism to collaborate with team orcommunity

ISSUE TRACKINGtightly integrate with issue trackerreduce the need for maintaining a change logtie builds to defects

REPOSITORY VIEWtightly integrate with repository view tool (e.g. github orfisheye)repository browsing provides a graphical diff when an updatehappens

TREND REPORTINGproject managers and developers can quickly visualize projectprogress statusunit testing is often not enough to provide confidence that thedelivered software complies to the desired quality

COMPLETE TESTSdo as much testing as possibleprioritize integration over unit testsrun quick limited set of "smoke tests" before full test suite

CLEANUPallocate lots of disk space due to archives and historyas with anything the more you keep, the worse theperformance may be to keep track of it allarchive/backup jobs before you fully delete themclean up after yourself (e.g. src directories)you can wipe workspacearchive limits for number of days / builds

JOB SEPARATIONseparate jobs for each branch or releasesupport parallel development efforts

CONFLICTSif jobs require network, make sure to allocate different orrandom ports effectively as there may be parallel jobs runningin node, we avoid this by injection to not even use the networkstackthink about potential race conditions

MANAGE TIMEdo not run everything at oncechange job times to efficiently use the systemadd slaves to distribute if required

FEEDBACKsetup feedback to ALL team members to improvecommunicationensure failures are reported as soon as possibleperiodically check communication mechanism to ensure itsworking

IDENTIFY SUCCESStag / label / baseline after successful buildif later you find unsuccessful, fix testing to avoid it next time

UTILIZE SLAVESutilize slaves only for jobs if you need a large distributablesystemmake sure high priority things have a dedicated hardware ifqueuing times are unreasonable

ADDITIONAL ADVICEdecide one jenkins instance or multipledo not get crazy with plugins (maintanence nightmare)minimize permissions (maintanence nightmare) and try tostart with trust!chaining is awesomeupdate software regularly, but try to minimize disruption

ADDITIONAL ADVICE (CONT)limit execution threadschange default java optionsput all scripts in the repo so code blocks are not hard codedutilitize views

ADVANCED TOPICScliapiautomationcore plugins

CLIdownload the cli from start SSH daemon in configuration that allows you to add a sshkey to a user and authenticateyou can use the same cli commands

http://localhost:8081/cli

APIgo to just any url and put in /apiprovides restful api in xml, json, pythoncan do an optional parameter depth=1 (default is 0) to getmore information but takes more bandwidthbuild queue and load statistics have separate api but just justgo to for more detailshttp://localhost:8081/api

AUTOMATIONlook at the demo-jenkins project to see automated installationbin/prepare.sh for installation and bin/configure.sh for theplugin and job installationutilized the clicould use the api to check status of kicked off jobsautomation with authentication is a bit more trickysuggest install plugins, then copy in a global config, restart,then from there use authentication to install jobspreserve the identity based files that are located in jenkinshome directory

CORE PLUGINS I USEgit github github-api (should be default)ghprb (github pull request builder)greenballs (seriously blue balls?)postbuild (useful for doing custom blocks based on results)dashboard view (show dashboard stats but can use walls if youprefer)email-ext (highly customizable email reporter default is plain)html publisher (attach any old html generated page)these are all installed in the demo along with dependencies

PLUGINS YOU LIKE?

QUESTIONS/COMMENTSCreated by Lloyd Benson / lloyd.benson@gmail.com

http://github.com/lloydbenson/presentations

http://github.com/lloydbenson/demo-jenkins