Integration testing at large scaled agile projects

1

The relation between time-to-market and the level of integration

Derk-Jan de Grood[SC]2 – 25 May 2016

Aim of this session

2

Get a collective

understanding of

how we deal with

integration issues

Integration is vital

when we want to

deliver business

value

How to ensure the product as a whole functions as desired?Relation between time-

to-market and the level

of integration

Our End Goal

3

Deliver our product (or new features) with a short time to market

CI/CD Assumptions

4

Teams Collaborate

Integration is Continue

Tests are Automated

Deployment is hands-off process

No Automation Backlog

Clear Acceptance Criteria

Feedback loop to improve Testing

Frequent Product Launch

….Integration problem

sFeature driven: When you work with more teams on the same system

Component driven: When teams work on adjacent systems

Legacy system: When code adaptions have unknown impact

5

No matter how we are organized we have….

What kind of integrations do we have?

6

What is your system boundary?

7

Component System Service Organization

8

What we achieve each sprintWhat we should do, but do not achieve each sprint

Definition of (un)doneThe definition of done gives a quick insight in the level at which integration takes place. The definition of Undone (see Less framework) identifies integration tests that are not done within each sprint. Both are indicators of the system boundaries that are taken into account

Ensuring Integration

9

Organization

Component

System

Service

Continuously(in the sprint)

Occasionally(e.g. prior to a release)

CI/C

D

MBT

Service

Virtualization

Stubbing

e2e

ST

Test

Automation

UT

Integration

sprint

Manual RT

Interface

testing

General trend: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market and this insight might lead to targeted improvements

Ensuring Integration (rough sketch)

10

Organization

Component

System

Service

Continuously(in the sprint)

Occasionally(e.g. prior to a release)

CI/C

D

MBT

Service

Virtualization

Stubbing

e2e

ST

Test

Automation

UT

Integration

sprint

Manual RT

Interface

testing

Reduce amount of releases

Ensuring Integration (rough sketch)

11

Organization

Component

System

Service

Continuously(in the sprint)

Occasionally(e.g. prior to a release)

CI/C

D

MBT

Service

Virtualization

Stubbing

e2e

ST

Test

Automation

UT

Integration

sprintManual RT

Interface

testing

issues after the

sprint

Ensuring Integration (rough sketch)

12

Organization

Component

System

Service

Continuously(in the sprint)

Occasionally(e.g. prior to a release)

CI/C

D

MBT

Service

Virtualization

Stubbing

e2e

ST

Test

Automation

UT

Integration

sprint

Manual RT

Interface

testing

Organizational Readiness

Another Case Study

13

Architecture• What are the

business processes?

• What are the components?

• What are the interfaces?

Acceptance criteria• What is the

Minimal Viable Product?

• What integrations are needed to make it work?

Requirements traceability• When are we

complete?• How do test

results add up to acceptance?

14

Missing

What should a car minimally do?

15

Planned Integration Tests

16

Integrations needed to

make it work

Release Date

In one of my projects I used tested integrations as a measure of progress. Only if the integration is done (and tested) you know that the solution will work as a whole…The Burn down Chart was used to inform management on the progress of the project and to plan the next integrations in the scrum-of-scrums

Exchange Experiences

On what level do you integrate (SUT) and with what speed?Is there a need to speed up?

17

For example CI/CD MBT UT Automated System test Automated e2e test Interface testing Manual Regression testing Integration sprints other

WRAP-UP

18

Summary

There is a common goal: Deliver our product (or new features) with a short time-to-market.CI/CD helps to do so, but requires a lot from the organization, a.o. frequent integrationThere are levels on which you can integrate, e.g. Component, System, Service and OrganizationLarge scale integration, gives more reliable and commercially feasible products, but integrates is harder.Making this clear to management enables to manage expectations or helps to target your next improvements.

Derk-Jan

ValoriColtbaan 4a3439 NG NIEUWEGEINThe Netherlands

• derkjandegrood@valori.nl• +31(0)651807878• www.valori.nl• @DerkJanDeGrood• http://djdegrood.wordpress.com

Derk-Jan

20

Success !