Continuous Integration (CI) By Jim Rush Version Control Build Test Report

Continuous Integration(CI)

By Jim Rush

VersionControl

Build

Test

Report

Agenda

• About Me

• What is Continuous Integration

• Components of Continuous Integration– Commonly used Tools– Tour through an actual Jenkins deployment– Exploration of major components– Advice

About Me• Development Manager building enterprise products for Call

Centers• I work for Enghouse, which is a Canadian holding (acquisition)

company. The local office was once known as Syntellect.• In the last seven years we’ve built various types of products that

are built and validated by CI systems:– Telephony (call control)– API (middleware) and User Interfaces (Winforms & SilverLight)– Multiple development languages (Java, C#, C++)– Different test tools and frameworks– Team consists of those fresh out of school to the young at heart

me@jimrush.netwww.jimrush.net

What is Continuous Integration

continuous integration (CI) implements continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.

---Wikipedia

More Simply

Periodically, Validate your Project

More Simply

Periodically, Validate your Project• Periodically – as often as you can afford to.

Ideally, anytime something changes• Validate – insure project is operating within a

margin of expected results• Project – Everything that constitutes the

creation, validation and maintenance of your software.

Why

• Know when you break existing functionality. – Issues are typically easier to address the sooner

they are found.– Issues are typically easier to address when the

number of changes between working and failing is small.

• Visibility into health of project– Fail fast – If there is a problem, discover and plan

for it early. Don’t wait until the end of your project to realize it doesn’t work.

Common CI Components

• Software Repository (version control)• Build Software

– Build scripts (Batch files, Ant Scripts, Make files, Visual Studio)• Monitor and trigger process

– Software package that monitors software repositories for changes and triggers a build

– Identifies and reports failures/successes• Automated Testing

– Unit, Integration, System, X-ability testing• Metrics Collection and Reporting

– ESLOC, Test Counts, Code Coverage, performance data, …

Common CI Workflow

Version Control Build Unit

TestIntegration

TestSystem

Test

Reporting / Monitoring / Metrics

Continuous Integration Manager

Continuous Integration Manager

• Manages the CI Workflow– Monitors Version Control– Handles CI workflow and dependencies– Reporting

• Common Tools– Cruise control– Jenkins/Hudson– Custom scripts and code

http://en.wikipedia.org/wiki/List_of_build_automation_software

Why do I like Jenkins

• Reasonably friendly web based administration, but uses easy to change XML for persistence

• Flexible• Enables distributed build and testing• Provides a plugin for nearly every other available

tool in the CI workflow– Various artifact repositories– Various triggers– Authentication system, including not having one– Very rich set of reporting components

Very Quick Tour of Jenkins

Our Main Page

Job Configuration

A Build Job(Compiles Project)

A Test Job

Custom Test Detail Report

Version Control System (VCS)

• If you aren’t using Version Control, why not?– It can be free– It’s fairly easy to setup and manage a basic repository

• Why to have it– Backups– Tracking changes to time, person, feature, …– Support multiple versions of code including exploratory versions

• Common Solutions– Subversion (SVN)– Mercurial– GIT– …

Building

• A build is any set of steps that creates a distribution• Compiles(if needed) are usually technology driven• Common tools:

– Apache Ant– Maven– Batch files / Shell scripts

• Common Challenges– Repositories or other intermediate build components– Error reporting– Dependency management– Build efficiency

Build Advice

• Build a complete product including installation• Invest in quality build scripts– Well documented– Good logging– Good error handling and reporting– Build all distribution flavors

• Multiple project members can maintain the build• Apply static code analysis or other tools that can

validate your code• Profile your builds• Design builds to support distribution

Testing• Testing Types

– Unit – modular, isolated– Integration – not modular nor isolated– System – Complete distribution and dependencies

• Common automation tools/frameworks– Code testing: xUnit (Junit, NetUnit,…)– UI Testing tools: Accessibility layers and/or commercial products– Custom scripts, frameworks and simulators

• Testing Results– Junit XML files are the most flexible– Test count– Code coverage

Testing Advice

• Write quality tests cases as they can become harder to maintain than the product

• Provide good feedback for working and failing steps

• Test failure recovery can be hard• Integration and Testing frameworks can be

more complicated and more difficult than the project you are creating

Metrics

• Collect what is important or useful• Most data is only valuable as a trend• There are no magic goals for some data

Metrics Advice

• Unless you have a tool that captures what you want, build your own. Simple code to collect and a database is all you need.

• Writing small applications that grab data and generate HTML isn’t hard.

• Linking graphs to data tables in Excel is easy.• Periodically review the data

Metrics presented with Excel

Detail Test ReportShowing last 10 test results

Code Coveragewith Cobertura

Thinking about your CI Infrastructure

• It can be general purpose trigger and scheduling tool

• It can be a reporting / metrics hub• It doesn’t have to be singular– Multiple sites / teams– Different versions or types of projects

Parting Advice

• Treat your CI environment as a core part of your project• Multiple people should understand it your CI system• Knowledge is power (metrics/project insight), but use it

carefully• Data doesn’t have to be pretty, focus on content and value• Continuously Improve – Allocate time to improve• Don’t let your process or metrics make you stupid; focus on

your goals and how you are going to get there• There isn’t enough time to do everything, prioritize• It is both easier, and harder than it appears• Your infrastructure needs to be reliable enough to be trusted

Questions ?