From ci to cd - LavaJug 2012

FROM CI TOCD&D

LAVAJUG

NOVEMBER 8, 2012

*

TOCSOFTWARE CHALLENGES

CONTINUOUS INTEGRATIONCONTINUOUS DEPLOYMENT

NATIVE PACKAGING

ABOUT ME - @HGOMEZ

TODAY - CI ARCHITECT @ AXWAY

OPS HAT - FORMER SENIOR OPS DIRECTORDEV HAT - FORMER DEV, TEAM LEADER AND ARCHITECT

QA HAT - FORMER QA TECH LEADER

JPACKAGE , OBUILDFACTORY AND DEVOPS INCUBATOR PROJECT FOUNDER

PROUD MEMBER OF ASF (TOMCAT & XML-RPC ) & OSSGTP

*

* * *

* * *

TODAY SOFTWARECHALLENGES

SOFTWARE PROJECTS ARE LARGER AND COMPLEXMORE AND MORE COMPONENT INTEROPERABILITY

MANY TEAMS AND OFTEN REMOTE TEAMSTIGHT SCHEDULES

FREQUENT NEEDS AND REQUIREMENTS CHANGES

EARLY AGES (PREHISTORIC)

BUILD / RELEASE ON A “PER REQUEST” BASIS -> GOPROJECT IN PROPER STATE -> YES IT BUILD

WELL DOCUMENTED PER STANDARDS -> DOC IS NOT AN OPTIONPROPERLY REVIEWED -> SOMEONE TESTED PRODUCT

BUT IT FAILED TOO OFTEN ;(

THEN CAME AGILE

TOOLING NEEDED FOR AGILESTANDARDIZE BUILD, TEST AND DEPLOYMENT

PROVIDE CONTINUOUS BUILDS, TEST AND DEPLOYMENTENSURES SOLID, REPRODUCIBLE AND ANALYZABLE STEPS

VISIBLE TO ALL ACTORS (DEV, QA, PRODUCT OWNER)

AGILE IDEAL CYCLE :-)

CONTINUOUS INTEGRATION"CONTINUOUS INTEGRATION IS THE PRACTICE OF INTEGRATING

EARLY AND OFTEN, SO AS TO AVOID THE PITFALLS OFINTEGRATION HELL".

GOALTHE ULTIMATE GOAL IS TO REDUCE TIMELY REWORK AND THUS

REDUCE COST AND TIME.

WHEN DONE WELL, CONTINUOUS INTEGRATION HAS BEENSHOWN TO ACHIEVE THESE GOALS.

WORKLOAD - HUMAN

TEAM MEMBERS WORK ON CODECODE AND UNIT TESTS

PUSH ON SCM AFTER LOCAL TESTS PASSED

WORKLOAD - MACHINEPOLLS SCM AND CHECK OUT IF CHANGES

BUILD ALL CODE RUN ALL UNIT TESTS

DEPLOY SNAPSHOTS TO ARTIFACTS REPOSITORY

WORKFLOW

CONTINUOUS INTEGRATIONSTEPS

FETCH FROM SCMBUILD CODE

RUN UNIT TESTSRUN INTEGRATION TESTS

DEPLOY ARTIFACTSRUN ACCEPTANCE TESTS

PRODUCE REPORTS

NOTES ABOUT TESTS

TESTS SHOULD COVERS A WIDE SCOPE FROM DEV UP TORUNTIME

UNIT TESTINGINTEGRATION TESTINGACCEPTANCE TESTING

PERFORMANCES TESTING

UNIT TESTINGHERE WE TEST CODE

ENSURES THE CODE IS DOING WHAT WE INTEND IT TO DOMANY EXTREMELY QUICK AND SMALL TESTS

UT ENSURE WE MAKE THE SOFTWARERIGHT

INTEGRATION TESTINGUSED TO VALIDATE CONTRACTS BETWEEN PARTICIPANTSSHOULD BE PORTABLE (NOT PLATFORM DEPENDANTS)

"EXTERNAL COMPONENTS" SHOULD BE EMBEDDEDABLE ORMOCKED

IT ENSURE WE MAKE THE RIGHTSOFTWARE

ACCEPTANCE TESTINGKEEP THEM SIMPLE

REAL TARGET COMPONENTS SHOULD BE USEDPREFER PRE-REQUISITE (DONT EMBED SQL DB ENGINES)

DATASETS IN USE SHOULD BE MODERATE

PREPARE FOR PRODUCTION - ENROLLYOUR OPS

PERFORMANCE TESTINGFIX QUICKLY PERFORMANCE REGRESSIONS

PREPARE FOR CAPACITY PLANNING

PROTECT YOURSELF FROM IN-PRODDISCOVERY

CONTINUOUS DEPLOYMENT

CONTINUOUS DEPLOYMENT IS CONTINUOUS INTEGRATION ONSTEROIDS

BIG PICTURE

WELCOME TO OPS LAND

CONTINUOUS DEPLOYMENTFOR ?YOUR DEVS YOUR QAS

YOUR SUPPORTYOUR SALES

AND OF COURSE YOUR CUSTOMERS

ANY MAGIC RECIPES ?

KEEP IT SIMPLETHINK COMPONENT

ASSEMBLE THEM ALL !

LEGO MODEMANDATE COMPONENT APPROACH

ASSEMBLE ARTIFACTSCOMPONENTS ASSEMBLY GUIDELINES

TAKE SPECIAL CARE ON COMPONENTS CONFIGURATIONS

ASSEMBLE THEM ALLDEPLOY USABLE ARTIFACTS

READY TO USE BY UPSTREAM TEAMS LIKE QA-OPSSHOULD BE CONFIGURABLE FROM THE OUTSIDE

HEY, DON'T FORGETCONTINUOUS DELIVERY !

THERE IS CONTINUOUS DEPLOYMENTBUT DON'T FORGET CONTINUOUS DELIVERY

DELIVERYPROVIDE PRODUCT TO END USER

USER COULD THEN INSTALL OR UPDATE IT ON HIS OWNSYSTEMS

DEPLOYMENTPROVIDE OPERATIONAL SOLUTION TO END USER

USER COULD DIRECTLY USE IT ON HIS OWN OR LEASED SYSTEMS

EXTERNAL APPROACH

MANY GOOD PROVIDERS, CLOUDBASED

CLOUD BEES

HEROKU

OPENSHIFT

BUT YOU MAY DON'T HAVE CHOICES AND HAD DO IT IN-HOUSE

*

*

*

IN-HOUSE APPROACH

MATERIALIZE 3 CHAINS

SOFTWARE FACTORYQUALITY FACTORY

DELIVERY&DEPLOYMENT FACTORY

AND DONT FORGET SERIOUS MONITORING FOR ALL OF THEM

SOFTWARE FACTORY

FROM SCM TO ARTIFACTS

BUILDUNIT TESTS

INTEGRATION TESTSCODE QA REPORTS (SONAR)

DEPLOY ARTIFACTS

QUALITY FACTORY

FROM ARTIFACTS TO RUNTIMEREPORTS

CONSUME ARTIFACTS FROM SOFTWARE FACTORYRUN ACCEPTANCE TESTS

PERFORM PERFORMANCE TESTSPROVISION CAPACITY PLANNING

REPORTS GENERATION

DELIVERY&DEPLOYMENTFACTORY

FROM ARTIFACTS TO RUNNABLE

BUILD NATIVE PACKAGESBUILD IMAGES (VMDK, AMI, DVD ISO)

PROVIDE RUN INFRASTRUCTURE (INSITE)INTERFACE TO RUN INFRASTRUCTURE (OFF-SITE/PUBLIC)

CATALOG & PROVISION RUNTIMES

INTERACTIONS BETWEENFACTORIES

INTERACTIONSSWF PRODUCES SOFTWARE MATERIALS

QAF CONSUME SWF MATERIALSQAF GIVE 'GO FOR PRODUCTION' IN D&DF

D&DF PROVIDES QAF RUN INFRASTRUCTURED&DF COULD PROVIDES DEV RUN INFRASTRUCTURE

TOOLING SCM : SUBVERSION, GIT, MERCURIAL (CVS SIC)ORCHESTRATION : JENKINS (WHAT ELSE ?)

CONTENT REPOSITORIES : ARCHIVA , ARTIFACTORY , NEXUS ,HTTPD

QUALITY : SONAR

RUN FORCE : JENKINS AGENTSRUN MIDDLEWARE : ESXI , VIRTUALBOX , VAGRANT ,

OPENSTACK /CLOUDSTACK

MONITORING : ZABBIX , GRAPHITE , COLLECTD , JMXTRANS

*

* * **

*

* * ** *

* * * *

NATIVE PACKAGINGWIDELY USED IN UNIX WORLD

OS COMPONENTISATION APPROACHRUNTIME DEPENDENCIES AWAREPOWERFUL LIFE-CYCLE SUPPORT

WEB BASED PACKAGES REPOSITORIES FOR MODERN DELIVERY

WIDELY USED IN UNIXWORLD

YOUR OPS KNOWS THEMYOUR CUSTOMER OPS KNOWS THEM

AVOID "NOT INVENTED HERE SYNDROM", BRING MORECONFIDENCE

OS COMPONENTISATIONAPPROACH

YOUR APP SHOULD BE A COMPONENT LIKE THOSE BUNDLED BYOS

EASY TO MANAGE AS SYSTEM SERVICETAKE CARE OF SYSTEM HABBIT (USER BY SERVICE)

RUNTIME DEPENDENCIESAWARE

PROVIDES OR DEPENDS ON OTHERS OS PACKAGESHELP SPLIT COMPLEX SOLUTIONS IN SET OF SIMPLE PACKAGESINJECT SQL ENGINES, FRONT-ENDS CONFIGURATION VIA HOOKS

PUPPET/CHEF MANAGEABLE

POWERFULL LIFE-CYCLESUPPORTEASY INSTALLATION

EASY UPDATEEASY REMOVAL

INTERACT WITH OTHER COMPONENTS LIFE-CYCLE

MODERN DELIVERYYUM, ZYPPER, APT ARE USING WEB BASED PACKAGES

REPOSITORIESBUILD / PROVIDE YOUR OWN COMPANY WEB REPOSITORIES

SAME SOURCE FOR YOUR DEV, QA, SUPPORT, SALES ANDCUSTOMERS

100% CLOUD-READY WITHOUT EFFORT

SMART CD WITH JENKINS &RPM

1. BUILD (MAVEN)2. PACKAGE (RPMBUILD) 3. UPDATE REPO4. DEPLOY !

DEPLOYMENT STEPS INPIPELINE VIEW

DEMONSTRATION AT VIMEO

IN A NUTSHELLCONTINUOUS INTEGRATION MOTORIZE AGILITY

THEN CAME CONTINUOUS DELIVERY AND DEPLOYMENTCONTINUOUS D&D COULD BE DONE IN HOUSE

CONSIDER NATIVE PACKAGING FOR CONTINUOUS DEPLOYMENT