JavaOne 2015: Scalable Continous Deployment with Maven

Scalable Continuous Deployment with Maven

fromfragiletoagile.com@AbrahamMarin

@AbrahamMarin

About Me

@AbrahamMarin

About Me

@AbrahamMarin

About Me

@AbrahamMarin

Table of Contents

• Continuous Deployment with Maven• Scaling Continuous Deployment• Analyse Data from the CI System• Automate Analysis and Visualisation

@AbrahamMarin

Table of Contents

• Continuous Deployment with Maven• Scaling Continuous Deployment• Analyse Data from the CI System• Automate Analysis and Visualisation

@AbrahamMarin

What is Continuous Deployment?

Continuous Integration: check everything is still working after every commit•• Continuous Deployment: every successful

commit turns into a release•

@AbrahamMarin

Why maven?

• Just because…

@AbrahamMarin

Other technologies

@AbrahamMarin

To Caesar what is Caesar’s

Based on John Ferguson Smart’s“Real-World Strategies for Continuous Delivery with maven and Jenkins”

http://youtu.be/McTZtyb9M38

@AbrahamMarin

John’s approach

Maven wasn’t built for Continuous Deployment

commitcommitcommit...

0.0.1-SNAPSHOT

Release!

0.0.1

@AbrahamMarin

John’s approach

Don’t use RELEASE pluginUse VERSIONS plugin

Set version to <version scheme>.<build number>Run mvn deployCommit pom file to repository

@AbrahamMarin

John’s approachSet version to <version scheme>.<build number>

mvn versions:set –DnewVersion=**your version**

@AbrahamMarin

John’s approachRun mvn deploy

mvn clean deploy

@AbrahamMarin

John’s approachCommit pom file to repository

<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-scm-plugin</artifactId><version>1.9.1</version><executions><execution><id>commit</id><phase>deploy</phase><goals>

<goal>checkin</goal></goals>

</execution></executions></plugin>

@AbrahamMarin

John’s approach

@AbrahamMarin

John’s approach

0.0.1.1commit

BUILD!

0.0.1.2commit

BUILD!

0.0.1.3commit

BUILD!

@AbrahamMarin

Table of Contents

• Continuous Deployment with Maven• Scaling Continuous Deployment• Analyse Data from the CI System• Automate Analysis and Visualisation

@AbrahamMarin

How do you scale this?

@AbrahamMarin

SUPER APP

# Files: 75# Tests: 800Build Time: 4 minOutput: superapp.war

@AbrahamMarin

SUPER APP

# Files: 113# Tests: 1200Build Time: 6 minOutput: superapp.war

@AbrahamMarin

SUPER APP

# Files: 169# Tests: 1800Build Time: 9 minOutput: superapp.war

@AbrahamMarin

When Builds Get Too Big

@AbrahamMarin

"MAN Atlante fronte 1040572" by Lalupa - Own work. Licensed under GFDL via Commons - https://commons.wikimedia.org/wiki/File:MAN_Atlante_fronte_1040572.JPG#/media/File:MAN_Atlante_fronte_1040572.JPG

SUPER APP

# Files: 169# Tests: 1800Build Time: 9 minOutput: superapp.war

APP BACKEND

SUPER APP

# Files: 115# Tests: 1200Build Time: 6 minOutput: superapp.war

# Files: 72# Tests: 800Build Time: 4 minOutput: appbackend.jar

@AbrahamMarin

<dependency><groupId>com.superappfactory</groupId><artifactId>appbackend</artifactId><version>??????</version></dependency>

<dependency><groupId>com.superappfactory</groupId><artifactId>appbackend</artifactId><version>LATEST</version></dependency>

Setting up dependencies

APP BACKEND

SUPER APP

appbackend.jar superapp.war

@AbrahamMarin

@AbrahamMarin

@AbrahamMarin

@AbrahamMarin

Setting up dependencies

APP BACKEND

SUPER APP

appbackend.jar superapp.war

@AbrahamMarin

Rebuilding old versions

@AbrahamMarin

Rebuilding old versions

Using “LATEST” makes it impossible to build old versions correctly

@AbrahamMarin

<dependency><groupId>com.superappfactory</groupId><artifactId>appbackend</artifactId><version>???????</version></dependency>

<dependency><groupId>com.superappfactory</groupId><artifactId>appbackend</artifactId><version>1.5.3.1</version></dependency>

Rebuilding old versions

APP BACKEND

SUPER APP

appbackend.jar superapp.war

@AbrahamMarin

Update versions of dependenciesmvn versions:use-latest-releases

@AbrahamMarin

APP BACKEND

SUPER APP

APP BACKEND

SUPER APP

DATA MODEL

SUPER APP

DATA MODEL

GUI

APP BACKEND

Like it?

@AbrahamMarin

Like it?

@AbrahamMarin

Like it?

@AbrahamMarin

Like it?

@AbrahamMarin

Like it?

@AbrahamMarin

@AbrahamMarin

http://thechive.com/2014/02/26/youre-doing-it-wrong-31-photos-2/

I Can Help

@AbrahamMarin

Problem: Infinite Trigger

commitbuild

deploy

commit

@AbrahamMarin

@AbrahamMarin

@AbrahamMarin

Problem: Unnecessary rebuilds

APP BACKEND

SUPER APP

commit

@AbrahamMarin

@AbrahamMarin

Problem: Unnecessary rebuilds

@AbrahamMarin

Problem: Unnecessary rebuilds

Get last committer

buildAgent?

Proceed normally

Don’t run build

NO YES

touch skip_build

@AbrahamMarin

Problem: Unnecessary rebuilds<profiles><!-- Plugins that need to be disabled when doing a no-run --><profile><id>do.nothing</id><activation><file>

<exists>skip_build</exists></file></activation>

<build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.1</version><configuration>

<skipMain>true</skipMain><skip>true</skip>

</configuration></plugin>

@AbrahamMarin

<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-install-plugin</artifactId><version>2.5.2</version><configuration>

<skip>true</skip></configuration></plugin>

<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-deploy-plugin</artifactId><version>2.8.2</version><configuration>

<skip>true</skip></configuration></plugin>

Problem: Unnecessary rebuilds

@AbrahamMarin

Problem: Unnecessary rebuilds

APP BACKEND

SUPER APP

commit

@AbrahamMarin

Problem: Unnecessary rebuilds

@AbrahamMarin

Problem: Necessary rebuilds

APP BACKEND

SUPER APP

commit

@AbrahamMarin

Get last committer

buildAgent?

Proceed normally

Don’t run build

NO YES

Problem: Necessary rebuilds

@AbrahamMarin

@AbrahamMarin

Get last committer

buildAgent?

Proceed normally

NO YES

Check dependencies

Up to date?

NO Don’t run build

YES

touch skip_build

@AbrahamMarin

Problem: Necessary rebuilds

@AbrahamMarin

Problem: Doomed Build

build

deploy

commit

commit

commitpom.xml

@AbrahamMarin

Get last committer

buildAgent?

Proceed normally

NO YES

Check dependencies

Up to date?

NO Don’t run build

YES

touch skip_build

@AbrahamMarin

Get last committer

buildAgent?

Check pom.xml

NO YES

Check dependencies

Up to date?

NO

Don’t run build

YES

touch skip_build

Up to date?

NO

Proceed normally

YES

@AbrahamMarin

Problem: Doomed Build

@AbrahamMarin

@AbrahamMarin

Table of Contents

• Continuous Deployment with Maven• Scaling Continuous Deployment• Analyse Data from the CI System• Automate Analysis and Visualisation

@AbrahamMarin

A real case scenario

@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging

28%

@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging

28%

28%

28%

@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging

28%

28%

28%

20%

@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging

48%

28%

28%

20%

@AbrahamMarin

WAR file

WAR file

WAR file

Parent POM

Logging@AbrahamMarin

WAR file

WAR file

WAR file

@AbrahamMarin

Build-Driven Architecture

@AbrahamMarin

Table of Contents

• Continuous Deployment with Maven• Scaling Continuous Deployment• Analyse Data from the CI System• Automate Analysis and Visualisation

@AbrahamMarin

WAR file

WAR file

WAR file

@AbrahamMarin

Manual processingtakes time...

@AbrahamMarin

Automating Build Analysis

• Most CI systems provide an API• Calculations aren’t complex• Multiple graphical tools available

@AbrahamMarin

Build Hotspots

github.com/quiram/build-hotspots

@AbrahamMarin

Automating Build Analysis

• Add colour and size• Add support for other CI systems• Show subset of builds• Update data automatically (for build displays)• Anything else you may find useful!

@AbrahamMarin

Summary

• Setting up Continuous Deployment is possible• Scaling is challenging, but also possible• Build data can help you shape the architecture

of your application• Still plenty to improve, please join me

@AbrahamMarin

Questions?

@AbrahamMarin

fromfragiletoagile.com@AbrahamMarin

Thank You!