Jenkins User Conference 2012 San Francisco

Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf

Jenkins: State of Union

Kohsuke KawaguchiJenkins Creator@kohsukekawa

#jenkinsconf


Thank You To Our Sponsors


In Average Week of Jenkins


1 new release comes out

Gets downloaded 48,000 times


That fills up 513 DVDs


3.5 Brand-new Plugins Released


25 New Versions of Plugins Released

Gets downloaded 500,000 times


80 Tickets Filed


50 Tickets Solved


145 Commits Are Made


400 Jenkins Installations Pops Up


1200 New Slaves Are Added


19,000 New Jobs Are Added


# of plugins

Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-120

100

200

300

400

500

600

606

# of unique plugins

releases/weeknew plugins/week


Ticket Activities


2000

4000

6000

8000

10000

12000

14000

16000

15200

Total tickets

Resolved tickets


Tracked Installations (conservative)


5000

10000

15000

20000

25000

30000

35000

40000

45000 47000

# of masters



Recently Introduced Featureshttp://www.flickr.com/photos/randar/3879926596/


Demo


Not Every Frontend Change Is Visible

• Modernization– Dependency bump-ups: Prototype 1.7, YUI 3– Native JSON/CSS selectors where available

• Page loading performance– Liberal use of gzip compression– Client-side caching for assets in plugins

• Reuse common dependencies as plugins: jQuery

• Modularization of core JavaScript code• No ajax updates when pages aren’t visible


Multi-configuration Project Improvement

• Concurrent parent builds (between #N, #N+1, …)• More flexible workspace sharing• Sparse and see-through child builds• Axis whose values change dynamically• SCM checkout behavior is now extensible• Execution order / filtering is

now extensible


More Backend Stuff

• REST API improvements– API token– Update center / plugin manager / executors / slaves– Efficient data filtering by the “tree” parameter– Nestor is a good client example

• CLI improvements– Transport encryption– User authentication with SSH keys– ssh(1) as your client

ssh -p 2222 jenkins.acme.com build foo -s -v

https://wiki.jenkins-ci.org/display/JENKINS/Nestor


Fresh Off The Oven

• Does your Jenkins take 5 minutes to start?– Lazy-loading to the rescue– Unloading, too– More changes to come

• More self-descriptive XML

<scm class=“hudson.plugins.git.GitSCM” plugin=“[email protected]”> <branches> …


And Long List of Improvements…

• Subversion 1.7• Localization to 40 languages• Enforce naming convention to jobs• Core spins off more plugins• Arbitrary build step in Maven project type• Hash support in Cron• Slaves automatically comes back when disk is

available• Symlink support on Windows Vista/2003+


Improvements for Plugin Developers

• Cut off JDK5 compatibility to support JDK7• Dependency injection via Guice• JUnit4 test harness• Groovy for view templates– Debuggable– Your IDE auto-completes expressions

• More extension points (111 total)• Gradle, JRebel support


Survey

• http://jenkins-ci.org/survey• Tell us what you care– If you can, please give us testimonials– We’ll be discussing this later today

• Prizes– 2 Dr Dre headphones– Kindle– 2 $100 Visa Gift Cards– 1 Lightsaber, Sphero– T-shirts, giftbags, conference passes, JRebel license,…


Java Requirement

• Proposal: start requiring Java6– For running masters and slaves– For running Maven project type

• Data points– About 1.5% people run their Jenkins on Java5– Java5 reached end-of-life in Oct 2009– Widespread consensus among developers

• If you have a problem, tell us now!


With My Day Job Hat On…

http://www.flickr.com/photos/sushicam/7211787664/


Demo


BuildHive.cloudbees.com

• Free for anyone• Love to get your feedback


Jenkins Enterprise by CloudBees

• Enterprise plugins + Support– Active/standby high-availability– Validated merge (aka “Unbreakable builds”)– Templates, RBAC, custom update center, …

• CloudBees Free Enterprise Plugins– Folder– Backup

http://www.cloudbees.com/jenkins-enterprise-by-cloudbees-overview.cb




Beyond Coding

http://www.flickr.com/photos/acmace/4410650624/


JUCs Around the World

• Last year, we went to 5 places


… and more meet-ups

• FOSDEM, SCALE 10x• Copenhagen user event


… and more meet-ups / hackathon

• Austin, Munich, 3 more places in Japan


Upcoming Events

• FOSDEM and JUC 2013 in SF for next 12 months• Shift to cheaper & more open events– Half-day, to avoid catering– Cheaper venue - university, large companies– Free/minimal admission

• We want to work with local folks– “Start your own Jenkins meetup in 10 easy steps”

http://bit.ly/QtZMdK– Asia, where are you?

http://bit.ly/QtZMdK




Jenkins CIA Program

• Speaking about Jenkins in a meet-up?– We’ll send you a T-shirt

and stickers• We are building a world

map with push pins

• Join the CIA to spread words!

https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CIA+Program

https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CIA+Program


Security Advisories

• When vulnerabilities are discovered, we prepare a fix and issue a security advisory

• Please subscribe via e-mail or RSS– http://jenkins-ci.org/advisories

• Especially if you run Jenkins on the internet


On Timely Resolution of Vulnerabilities

• File issues to the SECURITY project– http://issues.jenkins-ci.org/browse/SECURITY/

• [email protected]– Closed-door discussion group for devs


Wiki Static Cache

ConfluenceNginx

HTML

in cache?

fallback

generate upon changes &

periodically


Usage Statistics

• Installation count growth in plugin page• Also available in JSON


Stats.jenkins-ci.org

• Understand how Jenkins & plugins are used– Platform– # of slaves– Types of jobs


Blackbox e2e Testing

• Motivations– Test the final deliverables– Test in the same setup you use

• Results thus far– Ruby + Capybara + Cucumber– Test Jenkins via Selenium– Abstraction to start/stop Jenkins– Plugin to assist assertions & selections

https://github.com/jenkinsci/selenium-tests

https://github.com/jenkinsci/selenium-tests


Better Source Code Access

• Mother of all repositories– Search usage of code across all the plugins– http://git.jenkins-ci.org/all.git

• List of all the repositories– https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Repositories– Along with groupId:artifactId

• List of all the extension points– https://wiki.jenkins-ci.org/display/JENKINS/Extension+points– Along with their usage

http://git.jenkins-ci.org/all.git

http://git.jenkins-ci.org/all.git

https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Repositories

https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Repositories

https://wiki.jenkins-ci.org/display/JENKINS/Extension+points


Other Infra Improvements

• Server monitoring + Pager Duty• 9 mirrors around the world• Puppet-managed DNS servers• More offloading– repo.jenkins-ci.org, thanks to JFrog– jenkins.ci.cloudbees.com,

thanks to CloudBees

http://nagios.jenkins-ci.org/

http://nagios.jenkins-ci.org/


Office Hours

• Bi-weekly WebEx conference call– Wednesday, complements project meetings

• Originally intended as virtual meet-up– Mini presentation, Q&A, show&tell

• Currently more like “Ask Kohsuke” session for plugin/core developers

https://wiki.jenkins-ci.org/display/JENKINS/Office+Hours

https://wiki.jenkins-ci.org/display/JENKINS/Office+Hours


Upcoming Features


Future Direction: Which Way?

http://www.flickr.com/photos/quinnanya/3500158343/


Make Complex Things Easier

• Running big, complex deployment should be easier– Master-to-master communication channel– Follow-up work for lazy loading– Per-job database storage, open for plugins

• More automatic slave managements


Make Complex Things Easier

• Better Workflow/pipeline support– Already a lot going on in plugins!– Slice up build histories– Shared context between related builds

• Modeling external systems– Databases, test environments, …– Remembering who’s using what– Remember what was done to them


Make Simple Things Simpler

• More spying, less asking– E.g., “where are unit test reports?”

• Simpler project creation ala BuildHive– Only needs to be 70% accurate– Sharable archetypes, repository sniffing

• Encourage sharing of recipes

http://buildhive.cloudbees.com/


Help You Find Plugins You Need

• Revisit bundled plugins– Why Git plugin isn’t in there?

• Plugin packs (aka iTunes playlist)– Follow the lead of the Jenkins PHP website

• Recommendation based on what you have• Reviews, feedbacks


Improving Plugin Development Experience

• More extension points• Better Plugin Development Environment– More Wiki articles/samples?– Training course?

• Better development mode– Skeleton generation– Checking common pitfalls in the dev mode

• Tell us what you need!


Conclusions

• A lot has been done, not just in code– And a lot more to come!

• Please help the project• Please enjoy the day!



Fighting Spams (AKA we are popular)

• Wiki spams have been a problem– Manual attack?– A scanner was developed to flag & delete them

• JIRA spam has been spotted• E-mail spams have been a problem

• If you find them, please report them• If you are willing to help policing them, let us know

https://github.com/jenkinsci/backend-confluence-spam-remover

https://github.com/jenkinsci/backend-confluence-spam-remover


More Number Crunching Needed

• Correlation• Recommendation

• We need volunteers– Should be fun!


And Longer Term…

• If we have 100 machines per developer, what can Jenkins do?

• Distributed parallel test execution• Job that spans multiple processes– Load test, system test, …

• What visualizations do we need?

Technology

Jenkins User Conference 2012 San Francisco