Bintray ach new
Building a Private CI/CD Pipelinewith Java and Docker
whoamiBaruch Sadogursky, JFrog Developer Advocate, @jbaruch
Going to talk about JFrog in 2 slidesGoing to talk about why
DevOps in 3 slidesGoing to talk about Artifactory next
2
Poll Time!
DockerDocker everywhere
Poll Time!Heard about DockerCan do the tutorialPoCing, playing
etc.Production, baby!
Y u noDocker production?
Docker Workflow
Union File System
Image Layers - OpenJDK
Getting Started with Docker
Y u noDocker production?
I like youBut I dont trust you
Continuous integrity.
JFrog Artifactory + Docker
Whos using Docker and nothing else?
Solomon Hykes17
Do we have an existing pattern?Do we need to adapt it?
CI/CD pipelines?Doing them for years.
The Promotion PyramidDevelopment buildsDev Integration
testsIntegr. testsStagingPre-ProdProdAmount of buildsBuild/Deploy
timeAmount of binaries
Pipeline: Quality Gates and Visibility
Source: Agile ALM, Michael Httermann, Manning Publications
Co.
$docker build
Docker buildAll the things!
Too easy!
Fast and cheap buildsNot always the way to go
Thats why.
Lets fix it!
Lets fix it (again)!
Lets fix it (again)!
Thats why you dont trust Docker
Traditional Server Pattern
http://martinfowler.com/bliki/ImmutableServer.html
Immutable Server Pattern
http://martinfowler.com/bliki/ImmutableServer.html
I dont always build promotion pipelinesBut when I do, its with
immutable and stable binaries
Whats up with the gates?!
Whats up with the gates?!
Whats up with the gates?! - QA shouldnt test dev images
Whats up with the gates?! - QA shouldnt test dev images -
non-tested images shouldn't be staged
Whats up with the gates?! - QA shouldnt test dev images -
non-tested images shouldn't be staged - non-staged, non-tested or
dev images shouldnt end up in production!!!
Not so fast
Trumped-up limitations
The Anatomy of Docker Tag
Wait a second, how can I have more than one repository per host
now?!
How can we support
this?https://host:8081/artifactory/docker-dev/busyboxhttps://host:8081/artifactory/docker-staging/busyboxhttps://host:8081/artifactory/docker-qa/busyboxhttps://host:8081/artifactory/docker-prod/busybox
One registry per host ought to be enough for anybody.
Panic!
Virtual hosts/ports to the
rescuehttps://host:8081/artifactory/docker-dev/busyboxContext
nameVirtual repository nameTag name
https://host:port/v2/busybox
server { listen 5001;
server_name 192.168.99.100; if ($http_x_forwarded_proto = '') {
set $http_x_forwarded_proto $scheme; } rewrite ^/(v1|v2)/(.*)
/artifactory/api/docker/docker-dev/$1/$2; }}
But then you realizeWait a second, now I need to pull, retag and
push for every step?!
Lets Design This one Right!
Anatomy of a container
Our LayersApplication: .war fileFramework: JDK8 + WildflyBase:
CentOS
Framework buildVerified base imageAdd system dependencies from
artifactoryJDKTomcatOwn it!
Minimal Framework build DockerfileFROM centos:7MAINTAINER
[email protected]
Application buildFramework is your baseRun a java buildAdd a
file to baseDone!
Application build DockerfileFROM
yourorg-docker.jfrog.io/yourorg/framework:latestMAINTAINER
[email protected]
ADD
https://yourorg.jfrog.io/yourorg/java-release-local//app-[RELEASE].war
/opt/jboss/wildfly/standalone/deployments/app.war
?
Application build DockerfileFROM
yourorg-docker.jfrog.io/yourorg/framework:latestMAINTAINER
[email protected]
ADD
https://yourorg.jfrog.io/yourorg/java-release-local//app-[RELEASE].war
/opt/jboss/wildfly/standalone/deployments/app.war
?
?
Application build DockerfileFROM
yourorg-docker.jfrog.io/yourorg/framework:latestMAINTAINER
[email protected]
ADD
https://yourorg.jfrog.io/yourorg/java-release-local//app-[RELEASE].war
/opt/jboss/wildfly/standalone/deployments/app.war
?
?
???
Framework PipelineApplication Pipeline
Framework PipelineApplication Pipeline
Framework PipelineApplication Pipeline
gradle-build is triggered by source changesDocker-framework
triggered by changes to dependencies (notifications on Bintray) and
XrayDocker-app is triggered by either gradle-build or
Docker-framework (if gradle-build is sufficiently fast, possibly
just gradle-build)Framework-test is downstream of
Docker-frameworkDocker-app-test is downstream of Docker-appFull
automation!67
HIGH QUALITY(software and information)SPEEDLOW
COST(automation)Fast releases > Modular > Automation
Conclusions: Release Fast or Die!
