Workflow, container, and beyond

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

1© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

Workflow, Containers, & Beyond

Kohsuke KawaguchiCreator of Jenkins @kohsukekawa

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

2© 2015 CloudBees, Inc. All Rights Reserved 2

Workflow

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

3

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

4

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

5

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

6

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

7

node(“linux”) {git ‘https://github.com/foo/bar.git’sh “mvn install”parallel({ sh “./regression-test.sh” },{ sh “./performance-test.sh” })…

}

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

8

Key Features• Pipeline as Code• Parallel, other interesting control flow• Allocate slave nodes and workspaces

–As many as you like, whenever and wherever• Standard project concepts

–SCM, artifacts, test reports, …

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

9

Build #153Build #153

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

10

1.10

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

11

Reusing Workflow

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

12

node(“linux”) {git ‘https://github.com/foo/bar.git’sh “mvn install”parallel({ sh “./regression-test.sh” },{ sh “./performance-test.sh” })…

}

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

13

acme_service “bar”

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

14

Custom DSLs & Push

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

15

vars/acme_service.groovydef call(name) { node(“linux”) {

git “https://github.com/foo/${name}.git”

sh “mvn install”parallel(

{ sh “./regression-test.sh” },{ sh “./performance-test.sh” }

)…

} }

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

16

Use Cases• I have lots of repos that are built, tested, & deployed the same way (e.g., micro-services)

• I want to maintain centralized control of the build process

• My product developers aren’t Jenkins experts

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

17

Using Workflow is as easy as 1, 2, 31. Create “Jenkinsfile” in repo2. Create job in Jenkins3. Specify SCM URL

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

18

To keep it simple …

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

19

Demo

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d


Containers

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

21

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

22

Containerized apps

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

23

Docker Commons Plugin

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

24

Transforming container

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

25

Docker Hub Notification Trigger plugin

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

26

Validation of containers

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

27

Promotion of containers

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

28

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

29

Too many image problems• Image names change all the time

• ‘latest’ changes what it points to

• Image gets transformed all the time

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

30

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

31

Docker Traceability Plugin

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

33

Demo Scenario

base image

app image

app src

base src

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

34

Demo

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d


Beyond

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

36

Title goes here

36

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

37

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

38

Title goes here

38

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

39

Installation growth y/y

30%

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

40

Next 10 years

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

41

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

42

Jenkins needs to adjust to this change• Our pitch on website hasn’t changed• We still bundle CVS & Subversion plugins• Lots of new stuff being done, but requires hand-assembly

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

43

Jenkins 2.0

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

44

Goals• Claim Continuous Delivery space

–We have lots of pieces, but we are not communicating any of them very well

• Revisit out-of-the-box experience of Jenkins

–Things people use everyday has been stale• Keep the ecosystem that makes Jenkins great

2.0 sends a clear signal that we are moving forward

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

45

Aspirational Timeline• Announce intention publicly (now)• Discuss and nail the details of 2.0 (Sep-Oct)

• Execution (Oct-Dec)–Periodic alpha/beta releases & feedbacks–PR activities–RC

• Work on plugin “2.0 readiness” (Jan)–Ensure key plugins are compatible

• Release (end-Jan)• Drop 1.x development

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

46

Pitfalls to avoid

Developers not getting

it

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

47

Pitfalls to avoid

Scope creep

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

48

Pitfalls to avoid

Disruptive 2.0

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

49

Pitfalls to avoid

Breaking plugins

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

50

Pitfalls to avoid

Data incompatibi

lity

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

51

Pitfalls to avoid

Massive Internal Changes

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

52

Pitfalls to avoid

Focusing too much on internals

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

53

That means …• Jenkins 2.0 needs to be limited-scope “evolutionary improvements” exercise

• Not “let’s start from scratch” kind of 2.0

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

54

Three Pillars1. Messaging changes

2. Out of the box experience that caters to CD needs

3. Targeted internal plumbing changes

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

55

Messaging•jenkins-ci.org → jenkins.cd•More up-to-date feature list page

•Authoritative & curated getting started guide

•Easier-to-contribute blog•Room for future growth

–Better plugin directory

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

56

Messaging Infra•Infra that enables

–Drupal → static site generation

–Confluence retirement (?)

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

57

OOTB Experience•Initial setup wizard

–Recommend initial set of plugins•UX refresh

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

58

OOTB Experience for CD•Deliver modern functionalities, OOTB

–Workflow–Pipeline as Code–Folders

•CloudBees is open-sourcing workflow stage view

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

59

Internals• Policy for removing deprecated APIs• Brand-new frontend development method• Selected library upgrades

–Limited to “should be compatible” kinds• Java 8 & Servlet 3.0

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

60

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d


Discussion

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

62© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

63

Integration/consumption of containers

ircbot: 51jira: 7wiki: 13

© 2

015

Clou

dBee

s, In

c. A

ll Ri

ghts

Res

erve

d

64

Technology

Workflow, container, and beyond