Embed Size (px)
Citation preview
© 2016 Mesosphere, Inc. All Rights Reserved.
Roger Ignazio | January 20, 2016
ELASTIC JENKINS WITH MESOS AND DCOS
© 2016 Mesosphere, Inc. All Rights Reserved.
Roger Ignazio
QE Tech Lead @ Mesosphere
@rogerignazio
ABOUT ME
© 2016 Mesosphere, Inc. All Rights Reserved.
mesosinaction.com
MESOS IN ACTION
© 2016 Mesosphere, Inc. All Rights Reserved.
● Conventional methods for scaling Jenkins● Elastic Jenkins on Mesos and Mesosphere DCOS● Demo● Future work
AGENDA
© 2016 Mesosphere, Inc. All Rights Reserved.
ALLOW ME TO INTRODUCE…
© 2016 Mesosphere, Inc. All Rights Reserved.
● General-purpose cluster resource manager● Represents many machines as a single entity● Advertises resources directly to applications● Works at scale: Apple, Twitter, Netflix, Airbnb, …
A BRIEF INTRO TO MESOS
© 2016 Mesosphere, Inc. All Rights Reserved.
A BRIEF INTRO TO MESOS
© 2015 Mesosphere, Inc. All Rights Reserved. 8
A BRIEF INTRO TO DCOS● Enterprise-grade
distributed operating system
● Mesos as a distributed systems kernel
● Built using open source and commercial tools
Existing Infrastructure
Mesosphere DCOS
Services & Containers
© 2016 Mesosphere, Inc. All Rights Reserved.
● Open source, distributed CI tool● Repeatable build/test of software projects● Large community, rich plugin ecosystem
A BRIEF INTRO TO JENKINS
© 2016 Mesosphere, Inc. All Rights Reserved.
SCALING JENKINS
© 2016 Mesosphere, Inc. All Rights Reserved.
SINGLE JENKINS MASTER
Team A . . .
Jenkins Agent 1
Jenkins Agent 4
Jenkins Agent N
Team B
Jenkins Agent 2
Jenkins Agent 5
Team X
Jenkins Agent 3
Jenkins Agent 6
. . .
Jenkins Master
© 2016 Mesosphere, Inc. All Rights Reserved.
SINGLE JENKINS MASTER → FAILED!
Team A . . .
Jenkins Agent 1
Jenkins Agent 4
Jenkins Agent N
Team B
Jenkins Agent 2
Jenkins Agent 5
Team X
Jenkins Agent 3
Jenkins Agent 6
. . .
Jenkins Master
© 2016 Mesosphere, Inc. All Rights Reserved.
MULTIPLE JENKINS MASTERS
Team A Jenkins
. . .Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team B Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team X Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
© 2016 Mesosphere, Inc. All Rights Reserved.
MULTIPLE JENKINS MASTERS
Team A Jenkins
. . .Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team B Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team X Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
© 2016 Mesosphere, Inc. All Rights Reserved.
MULTIPLE JENKINS MASTERS → STATIC PARTITIONING
Team A Jenkins
. . .Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team B Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team X Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
90% Utilized(Normal)
140% Utilized(40 builds in queue)
0% Utilized(Idle)
© 2016 Mesosphere, Inc. All Rights Reserved.
JENKINS ON MESOS
Team A Jenkins . . .
Mesos Agent 1
Mesos Agent 4
Mesos Agent N
Team B Jenkins
Mesos Agent 2
Mesos Agent 5
Team X Jenkins
Mesos Agent 3
Mesos Agent 6
. . .
Mesos Master
© 2016 Mesosphere, Inc. All Rights Reserved.
DEMO
© 2015 Mesosphere, Inc. All Rights Reserved. 18
FUTURE WORK
Tighter CI/CD integration with DCOS
● App deployment● Service discovery
© 2016 Mesosphere, Inc. All Rights Reserved.
Q & A@[email protected]
mesosphere.com/productmesos.apache.orggithub.com/jenkinsci/mesos-plugin
