Agile IT - A value driven approach to IT delivery final

An HML white paper: Agile IT

A value-driven approach to IT delivery

© HML 2012. All rights reserved.2

CONTENTS

2: About this paper

3. About HML

Introduction

4. The need for

change

Lean agile roots

5. The journey to

Agile IT

7. Overcoming

challenges to change

8. The Eight

Principles of Agile IT

13. The benefits of

Agile IT

14. Agile IT at work:

Forbearance Project

15. About the author

ABOUT THIS PAPER

The purpose of this paper is to demonstrate how applying lean and agile

principles can create value in software development.

It will be of interest to people who are responsible for IT projects, budget

control, organisational change or who have an interest in lean and agile

thinking.

The paper describes how HML has put these principles into practice in a

form of software development called ‘Agile IT’, which addresses key

business concerns like time to market, responsiveness to changing

needs, customer satisfaction and cost reduction.

The paper outlines the history of lean and agile approaches and explains

how these have been applied to create Agile IT. It sets out Eight

Principles that underpin HML’s approach, and how these enable the

delivery of the right IT capabilities at the right time to maximise value to

customers.

“Agile IT has allowed HML to meet the strategic

challenges of wasted resource and lengthy change

management processes. What might have taken

months now takes weeks and our IT department is

engaged and able to offer clients tangible results

throughout the process. ”

Andrew Jones, Chief Executive Officer, HML


HML‟s success in

applying lean and

agile principles has

resulted in a

nomination for the

Best Agile

Newcomer at this

year‟s UK Agile

Awards.

For more

information about

the Awards see

http://www.agileaw

ards.co.uk/Awards2

012.html

ABOUT HML

HML is the UK’s largest specialist mortgage servicer, providing

outsourced mortgage administration services to over 50 leading financial

institutions. HML operates from three UK locations: Skipton, Londonderry

and Glasgow. The company was established in 1988 and manages

approximately £43bn of mortgage assets and 400,000 customer

accounts.

INTRODUCTION

HML’s iConnect platform provides a sophisticated and comprehensive

tool to support a diverse range of client requirements. iConnect is

constantly updated, driven by HML’s desire to continually improve its

offering, but also by the regulatory environment in which HML and its

clients operate.

In 2008, HML began to apply lean 6-sigma improvement techniques

principally within its operational areas. In 2010 it embarked on an

ambitious programme to improve iConnect in support of this drive for

operational excellence.

To meet this challenge, the IT teams needed to develop their ways of

working, so Agile IT was born.

Agile IT is achieved by the application of eight principles:

1. Make it Visible

2. Make Collaboration Easy

3. Assume Change: Experiment, Discover, Learn, Iterate

4. Focus on Delivering Useful Value Today

5. Build a Sustainable Flow of Value

6. Build Quality In

7. Continuously Improve

8. Agility is Everyone’s Responsibility

HEADLINE RESULTS

In just 12 months, Agile IT has enabled HML to increase its rate of

delivery of new IT capabilities by a factor 4, while reducing the time

to deliver a new feature from months to a matter of weeks; all

without impacting cost or quality.

http://www.agileawards.co.uk/Awards2012.html




At the start of 2011,

HML recognised

that its IT teams

needed to evolve

their ways of

working in order to

become more

responsive to

changing needs

and reduce the

„time to value‟ of

changes, whilst

retaining the

improvements in

quality.

THE NEED FOR CHANGE

In common with many financial institutions, HML has in the past relied on

a sequential software development lifecycle, often known as ‘waterfall’,

where projects progress through a series of discrete stages:

• Request

• Design

• Build

• Test

• Implement

Each stage within this sequence was largely undertaken by a separate

specialist functional group and work was handed off from one group to

the next with formal documentation providing the communication between

groups.

This approach worked well for HML’s needs at the time and over five

years had resulted in a steady improvement in the stability of production

systems, with progressively fewer defects escaping into production, and

improved predictability of delivery dates.

However, these improvements tended to come at the expense of

responsiveness to change and long delivery timescales.

At the start of 2011, HML recognised that its IT teams needed to evolve

their ways of working in order to become more responsive to changing

needs and reduce the ‘time to value’ of changes, whilst retaining the

improvements in quality.

These changes were to be guided by the adoption of lean and agile

software development principles and practices – an approach that

complemented HML’s existing use of lean and 6-sigma techniques in its

Operational Service Areas.

LEAN-AGILE ROOTS

The term ‘Agile’ as applied to software development was first coined in

2001 to describe a range of methods that emphasised adaptability to

change. The Manifesto for Agile Software Development and its

associated 12 Principles sets out what being agile means; while a range

of methods, such as Scrum, eXtreme programming and DSDM, provide

ways of developing software in an agile way.

The idea of lean software development began at a similar time. It takes

the core principles of lean thinking, e.g. flow, pull, and continuous

improvement, and applies them to software development.


The timely and

sustained delivery

of valuable IT

capabilities to

HML‟s internal

customers and

external clients

remains

paramount, and is

at the core of Agile

IT.

Lean and agile can be seen as complementary approaches: lean tends to

address the wider organisational concerns, while agile focuses at the

development team level.

The combination of lean and agile principles and methods, together with

other approaches such as 6-Sigma, Kanban and the Theory of

Constraints provide the foundations for Agile IT at HML.

See http://agilemanifesto.org/

See http://www.scrum.org/ and http://www.scrumalliance.org/

See http://www.extremeprogramming.org/

See http://www.dsdm.org/

See http://www.lean.org/WhatsLean/Principles.cfm

THE JOURNEY TO AGILE IT

Lean and agile software development is not just something that you do;

it’s equally about how you think. Changing the ways of working in IT had

to address both transformation of its culture and the adoption of new

practices and techniques.

And while lean and agile principles and techniques have been at the

heart of the changes, these have always been treated as a means to an

end rather than an end in their own right. The timely and sustained

delivery of valuable IT capabilities to HML’s internal customers and

external clients remains paramount, and is at the core of Agile IT.

At the start of the change transition, HML appointed an experienced agile

development practitioner to provide expertise, leadership and support

through hands-on coaching. Reporting directly to the Head of IT provided

the appropriate executive authority and clearly signalled the importance

of the changes.

Building awareness of lean and agile approaches, principles and

techniques has been a cornerstone of the transition. Alongside formal

training and hands-on coaching, peer-based informal ‘learning lunches’

and ‘community of practice’ forums have been used to regularly share

experiences and learn new techniques.

Task boards - big visible displays that show a team’s flow of work - were

one of the first items to be introduced and they remain one of the most

visible aspect of the changes to visitors. Impediments – things either

slowing down or blocking flow – are quickly identified enabling action to

be swiftly taken to resolve.

http://agilemanifesto.org/

http://www.scrum.org/

http://www.scrumalliance.org/

http://www.extremeprogramming.org/

http://www.dsdm.org/

http://www.lean.org/WhatsLean/Principles.cfm


FIGURE 1: IMPEDIMENTS CAN BE ESCALATED TO A DEPARTMENT

LEVEL BOARD

HML has borrowed

from many of the

well known agile

methods, such as

Scrum, eXtreme

Programming (XP)

and DSDM, but has

deliberately chosen

to avoid mandating

a particular method

in order to

encourage

adaptation,

innovation and

ownership by

teams in how they

work.

Symbols used to indicate type of impediment:

Stopping progress

Slowing progress

Significant risk of stop or slow

Two other key changes were also introduced early in the transition: co-

location of teams and incremental development; where business goals

are progressively broken down into a series of features that are

developed in short iterations and delivered over a series of releases.

Fundamental to the long term success of changing the way of working

has been to make continuous improvement and learning a way of life.

Fortnightly ‘retrospectives’ enable a team to reflect on what is working

well, and what is not; and gives them responsibility for changing their

process. Over time, this process has itself been adapted, refined and

improved, with both a team and department wide improvement regime.

Instead, HML has developed eight principles of Agile IT that can be

used by everyone involved, from individual IT practitioners, to teams and

stakeholders to guide how they should work.

The principles help reinforce the view that Agile IT is primarily cultural,

and represents a journey not a destination.


Iterative

development and

incremental

delivery creates

significant

challenges for

upstream and

downstream

processes.

Creating an IT

capability to build

and deliver new

features on a just-

in-time basis also

requires a business

organisation that

can define and

prioritise a flow of

desired features,

and one that can

absorb and apply

them usefully.

OVERCOMING CHALLENGES TO CHANGE

Change is never easy, and HML faced a number of challenges on its

Agile IT journey.

One of the first challenges HML encountered was the creation of co-

located, cross-functional teams. Unsurprisingly, some individuals felt

unhappy to have to move desks, and breaking up long standing functional

groups risked losing their sense of community. HML worked hard to

involve those who needed to move in the planning, and introduced

changes progressively. The existing functional line-reporting structure

and regular functional team meetings were also retained. Only at the start

of 2012 did formal line management change to reflect the new cross-

functional structure of teams.

Throughout the transition, HML needed to ensure governance was not

compromised. Many of the existing governance controls were organised

around or were dependent on waterfall stages and functional teams, so

the move to cross-functional teams and iterative development potentially

reduce these controls’ effectiveness. Close collaboration with the Risk

and Compliance teams ensured appropriate oversight and HML is

currently introducing Practice Leaders who will be responsible for

governing key practice standards.

Finally, iterative development and incremental delivery creates significant

challenges for upstream and downstream processes. Creating an IT

capability to build and deliver new features on a just-in-time basis also

requires a business organisation that can define and prioritise a flow of

desired features, and one that can absorb and apply them usefully. HML

has made some progress in this area, using agile techniques to define

business cases for increments rather than whole projects, but this

remains an area of development and improvement.


Teams own their

visual management

boards, and this

fosters innovation

in how boards and

the underlying

process they

represent evolve to

meet the team‟s

needs.

THE EIGHT PRINCIPLES OF AGILE IT

1. Make it Visible

The process of software development is difficult to visualise, and while

Gantt charts have a place, they rarely map well to the needs of IT.

Agile IT makes use of large, physical display boards that show the flow of

work being undertaken by the team, an approach known as Visual

Management. Task boards and similar visual management tools help a

team visualise the ‘state of play’ of the work they are doing and make it

easy to spot and respond to impediments to a team’s development flow,

contributing to shorter delivery times.

Teams own their boards, and this fosters innovation in how boards and

the underlying process they represent evolve to meet the team’s needs.

For example, blockages are highlighted with stop signs; avatars of team

members are used to indicate who’s doing what; and colour is used to

indicate different types of work.

FIGURE 2: SHOWING FLOW OF WORK ITEMS, TYPICALLY USER

STORIES AND ASSOCIATED TASKS

As well as helping the team, visible task and status boards have proved

invaluable for updating stakeholders. They’ve also fostered greater team

spirit and visibility of the status of the project with the wider business.

Avatars

show

assignment

Impediments

highlighted with

Red Flags


Face to face

communication is

encouraged as the

primary method of

communication

IT projects face two

key challenges:

customers do not

know precisely

what they really

need; and in

today‟s competitive

business

environment those

needs are often a

moving target.

Agile IT assumes

that change is

inevitable and

seeks to exploit this

rather than resist it.

2. Make Collaboration Easy

IT development is a creative process. Customers and IT professionals

have to work together to ensure they deliver what is needed in a

background of imperfect knowledge and change. Making collaboration

easy is therefore essential for successful projects.

With Agile IT, work is undertaken by small co-located, cross-functional

teams, known as ‘pods’ who take responsibility for the end-to-end

delivery of new software features.

As well as bringing together expertise in different IT disciplines, such as

test, engineering and analysis, customer representatives and other key

stakeholders work closely with teams throughout the development and

delivery lifecycle.

Face-to-face communication is encouraged as the primary method of

communication. Documentation remains important, but the emphasis is

on doing just what is necessary and using alternative forms of

documentation, such as wikis and other electronic knowledge bases,

rather than paper documents.

3. Assume Change: Experiment, Discover, Learn, Iterate

IT projects face two key challenges: customers do not know precisely

what they really need; and in today’s competitive business environment

those needs are often a moving target.

The traditional approach to addressing these challenges is to undertake

extensive up-front analysis before locking down scope and then limiting

subsequent amendments through change control regimes.

Unfortunately it’s also common to see these IT projects take a very long

time to deliver what turns out to be the wrong thing.

Agile IT assumes that change is inevitable and seeks to exploit this rather

than resist it. Using an iterative approach to development coupled with

incremental delivery ensures that the most valuable features needed

today are delivered first whilst others are deferred for later increments.

IT projects at HML typically use fortnightly iterations, and are able to

deliver new features into production monthly. Stakeholders are able to

review working versions of iConnect at least every iteration, and this

feedback is used to adjust what features will be developed next.


Agile IT seeks to

incrementally

deliver value by

delivering in small,

valuable

increments. Each

increment is

focused on

delivering the

minimum needed to

provide benefit to

its customers

today.

4. Focus on Delivering Useful Value Today

Delivering something of value early is generally much more beneficial

than delivering a collection of things much later. As well as increasing

return on investment, focusing on delivering what’s useful today helps

prevent speculative development or ‘gold-plating’.

Research by the Standish Group suggests this is far from uncommon,

reporting that 45 per cent of software applications’ features are never

used. As well as incurring unnecessary cost, building these features

means that other more useful features are either delayed or not built at

all.

Agile IT seeks to incrementally deliver value by delivering in small,

valuable increments. Each increment is focused on delivering the

minimum needed to provide benefit to its customers today.

Stakeholders are actively involved in determining the order of

development of features and the team is encouraged to seek out

opportunities for delivering increments early. Monthly releases enable

new features to be rolled out quickly so it’s possible for a feature to go

from idea to delivery in just a few weeks.

Delivering incrementally places new pressures on repetitive tasks such

as build, deployment and regression testing and HML has steadily

focused on automating and refining this process. Similarly, architecture

and detailed designs need to support change, and technical practices

such as Test Driven Development and Refactoring have been adopted to

support this.

5. Build a Sustainable Flow of Value

Many organisations seek to maximise the utilisation of staff and the

number of projects that are in progress. Unfortunately, this approach

tends to result in long delivery times for any one activity and relatively low

completion rates. Whilst it can give the appearance of cost efficiency it

comes at the expense of time to value and responsiveness to changing

needs – qualities that usually far outweigh the savings made by

maximising utilisation.

Agile IT seeks to take a holistic view of the whole delivery value stream -

not just that part in IT - optimising for a fast, steady stream of deliveries of

valuable features. Instead of maximising utilisation (and minimising unit

cost), Agile IT seeks to minimise cycle time, the elapsed time it takes for

valuable increments to be delivered, and so increase throughput, the rate

of delivery of benefits.


Simply put, Agile IT

values finishing

over starting; doing

less in order to

achieve more, and

systematically

finding and

removing anything

that disrupts or

impedes flow.

Agile IT recognises

that speed and

agility requires

attention to quality

and technical

excellence; it is

simply not possible

to go fast if the

process or product

is unreliable.

Reducing cycle time enables returns on investments to be achieved

sooner and improves responsiveness to changing needs. Increasing

throughput maximises the opportunity to deliver what is needed.

Simply put, Agile IT values finishing over starting; doing less in order to

achieve more, and systematically finding and removing anything that

disrupts or impedes flow.

Teams at HML use either iterations - short timeboxes of typically two

weeks - or explicit work in progress limits to manage flow. Projects are

decomposed into small increments which are in turn broken down into

small features that can be completed in days rather than weeks. And

teams actively identify and resolve impediments to flow, escalating those

they cannot quickly resolve to a department-wide daily impediments

meeting attended by management to ensure visibility.

6. Build Quality In

Deming famously declared that “You can not inspect quality into the

product; it is already there.”* Most traditional approaches to software

development focus on the (often late) detection of problems, through

inspections and testing, and make extensive use of manual processes

that inevitably introduce variation and errors.

Agile IT recognises that speed and agility requires attention to quality and

technical excellence; it is simply not possible to go fast if the process or

product is unreliable.

Teams at HML employ automation for repetitive tasks, such as

integration, build, check and deploy; enabling them to execute these

tasks frequently – typically multiple times a day. They also use

techniques such as pair programming and test driven development that

help prevent errors or catch them as soon as possible after they are

introduced.

*From Out of the Crisis, by W Edwards Deming (MIT Center for

Advanced Engineering Study, 1986)


The idea that a

single process will

work equally well

for all time and

across all projects

and teams is not

reasonable, and so

Agile IT promotes a

culture of

continuous

adaptation and

improvement and

advocates that

those who do the

work are best

placed to improve

how the work is

done.

„Doing agile’ is

important, but

‘being agile’ is

essential.

Fundamentally,

agility is a mindset

that everyone in the

organisation needs

to express.

7. Continuously Improve

The need to respond to change applies just as much to the process of

development as to the capabilities of the software. The idea that a single

process will work equally well for all time and across all projects and

teams is not reasonable, so Agile IT promotes a culture of continuous

adaptation and improvement and advocates that those who do the work

are best placed to improve how the work is done.

HML has made knowledge sharing explicit; sponsoring ‘learning lunches’

and the formation of communities of practice, and has recently introduced

Practice Leaders to guide and support practice improvement.

Frequent and regular retrospectives - workshops focusing on

improvements - are undertaken by teams, typically every two weeks,

where teams are encouraged and empowered to make small incremental

changes to their processes to drive improvement.

8. Agility is Everyone’s Responsibility

‘Doing agile’ is important, but ‘being agile’ is essential. Fundamentally,

agility is a mindset that everyone in the organisation needs to express.

Agile IT provides an organisation with the potential for becoming an Agile

Business.

At HML, agility in general and Agile IT specifically, has become central to

its ethos.


An HML consultant

feeds back after a

recent project to

automatically and

securely retain

customer card

details goes live:

“I have to give you

this feedback

because it is proof

of the success of

card re-use

functionality. We

had over 200 calls

today and not

dropped a single

one, on last

working day!

Normally by now

we would have

expected to drop

calls as it is so

busy. It's obvious

that there are a lot

of card payment

only calls and we

are flying them.

Customer feedback

is positive and

consultant

feedback is very

positive.”

BENEFITS OF AGILE IT

Over the first 18 months of its Agile IT journey, HML has significantly

improved its IT capability:

Increased throughput

HML has been able to move from quarterly releases to monthly releases.

Delivering more frequently has had two benefits: firstly, it is much easier

to release opportunistically – stakeholders, having seen an iteration

demo, may realise that the software would be beneficial if deployed as is,

and monthly releases allow these opportunities to be exploited. Secondly,

more frequent and smaller releases reduce the risks associated with

each deployment.

Reduced time to deliver value

Before the transition to Agile IT, IT projects at HML, even those that used

a phased delivery approach, would typically take between three and nine

months from approval to their first delivery into production. Agile IT has

reduced this ‘time to value’ considerably, with projects typically now

delivering benefits within three months, and often considerably sooner.

Greater flexibility and responsiveness to changing needs

Focusing on achieving a business goal frees teams to explore and adapt

their solution so that it best meets that goal, enabling them to respond to

changing circumstances and new knowledge about what is important.

This flexibility has enabled projects to start and begin to deliver useful

benefits despite imperfect information; while allowing new needs to be

much more easily accommodated.

Improved stakeholder and client satisfaction

By working more closely with its stakeholders, building iteratively and

acting on their feedback, HML has been able build trusted relationships

and a real feeling that they were a part of our product development.


INDUSTRY

REACTION TO

AGILE IT:

“I got the impression

that [lean-agile

principles] were truly

embedded in the day-

to-day work of the

teams.”

“I was greatly

impressed by the IT

department...to see an

IT team working so

cohesively is something

we can only dream of.”

AGILE IT AT WORK: PROJECT FORBEARANCE

At the start or 2012, responding to a request from one of its clients, HML

began a project to implement improvements to iConnect to support

anticipated regulatory requirements proposed by the FSA.

Speed of response was critical to the client’s needs, but uncertainty

around what was really needed was high – HML was seeking to be first to

market – and operational usability was essential to ensure minimal

impact to customers and call centre consultants.

During the start-up phase of the project, the team, through a series of

workshops with stakeholders, created a high level vision, a prioritised list

of high level requirements and an outline roadmap. Work then proceeded

using fortnightly iterations that planned, built and demonstrated new

functionality. Regular retrospectives generated frequent improvement

‘experiments’ which drove constant adaptation and improvement in how

the team worked.

As the project progressed many items originally in the backlog were

considered unnecessary, and many new items were added in response

to improved understanding of needs identified through the

demonstrations. Throughout the process, both the client and internal

stakeholders were involved, and this has resulted in high levels of

engagement and satisfaction.

What the client said:

“The projects team's approach throughout has been one of collaboration

best demonstrated by their invitation to me, to not only attend, but part-

facilitate the two day forbearance training event that was delivered to our

Credit Management colleagues.” (Compliance Manager)

“It has been a pleasure to work with the Forbearance project team.

Communication has been excellent throughout the project, they have

been responsive to requirements (including late ones) and all signs point

to delivery meeting agreed scope and originally agreed timescale.”

(Operations Manager)

“I was surprised at just

how good the visual

management was in IT

in terms of its variety

and complexity.”

Delegate

responses from a

meeting of the

Lean Service

Forum, hosted at

HML in April 2012.

For more

information on the

Lean Service

Forum see:

http://www.oeeuk.c

om/community-

leanforum.asp

Key metrics

Mobilisation time: 2 weeks

First demo of useable functionality:

2 weeks after mobilisation, and fortnightly thereafter

First delivery of useful features: 6 weeks after mobilisation, and monthly (on demand) thereafter

Number of software defects escaped into production:

0

Internal stakeholder satisfaction:

Consistently rated high

Client satisfaction: Consistently rated high

http://www.oeeuk.com/community-leanforum.asp






ABOUT THE AUTHOR: Andy Lawrence, Lean-Agile Practice Lead

Andy has over 20 years experience in delivering IT based solutions across a

diverse range of industries, and has been leading and coaching software teams in

lean and agile software development for the last decade. He has presented at

conferences and industry events, and regularly contributes to groups such as Agile

Yorkshire and Agile Testing in Finance. Andy joined HML in 2008, initially working

with the Business Intelligence team before joining the IT department to lead the

lean-agile implementation.

Andy has been nominated in the Agile Coach or Mentor of the Year category for UK

Agile Awards 2012.

Technology

Agile IT - A value driven approach to IT delivery final