Administering a substantial fpc program in a large organisation

ACOSM2009, Sydney Australia, 13 November 2009 1

ACOSM2009 The Australian Conferenceon Software Measurement

“Achieving Positive Outcomes in a Changing World”

ACOSM 2009 – Presentation Abstract

• Presenter: Graeme Prescott - Australian Government Department of Education, Employment & Workplace Relations

• Title: Administering a Substantial FPC Program in a Large Organisation

• Abstract:

‏ Organising metrics recording in a large IT organisation working out of multiple sites, developing and maintaining applications and delivering training to end users, where over 40 projects deliver more than 10,000 FPs from work done on up to 25 applications each quarter. A comprehensive function point counting regime is used to support estimation and to help justify funding proposals and restrictions on work programs. The operational model utilises a qualified metrics cell, supporting trained counters in every project team. Techniques applied to support fluctuating skill levels and availability of the counting group includes continually monitoring staff movements and providing regular information and training opportunities. Managing the counting of multiple projects, impacting the same applications, provides challenges managing possible conflicts and corruptions of the counts and baselines. Managing the time critical consolidation of project counts into the application baselines before the next round of counts are required. Techniques to automate matching the reported function points to effort recording outcomes to provide defensible performance metrics. Progressing the metrics program into a management tool that enables the use of industry metrics to drive process and performance improvements.




The DEEWR Experience

Graeme PrescottAIMM CFPS MACS

Administering a Substantial FPC

Program in a Large Organisation




• An Overview of the Employment Systems Group

Presentation Roadmap

• Implementing and Managing the Counting Program

• Problems and Outcomes

• Where to from here ?




An Overview of ESG

• The ESG environment

– Location and Structure

– The Pattern of Work in ESG

– The visibility of metrics in ESG and DEEWR

• The history of FPCing in

– DEEWR ESG

– Other DEEWR IT Areas




Employment Systems Group

Application Development(Application Development, Project

Office, Training))

Application Systems Support(Help Desk, Production Support, Test

Management, BA Support)

Application Systems Technical

Support(Release, DBA, Internal Client

Applications)

Business Intelligence(Data Warehouse Development,

Reporting & Support)

Application Development(Application Development)

Canberra Sydney

Location and Structure




The Pattern of Work in ESG









W2W (2006-2007)

Working Nation (2007-

2008)

ESC4 (2008-2009)




The Current ESG Portfolio




• Within ESG

– Weekly management project progress

reviews

The‏Visibility‏of‏ESG’s‏Metrics

• Outside ESG

– Business area Board meetings

– Departmental annual report to

Parliament

– Departmental budget proposals




• Within ESG– A constantly maturing metrics program

• Initially one count per physical environment

• Changing the counting boundaries to business area based

• Maturing the management of counts as workload increased

The History of FP Counting in ESG

• Outside ESG – No other IT development area, has a metrics program in

place

– The ESG governance model may be exported to other IT areas as part of the best practice approach to IT in DEEWR




The History of FP Counting in ESG

ESC4 (2008-2009)

W2W (2006-2007)

Working Nation (2007-2008)




Implementing and Managing the Counting

Program

• The Structure

• The Workflow

• Managing the Process

• The Metrics




• Projects implement government initiatives

– Often bracket business areas

– Often impact multiple IT applications

ESG’s‏Organisational‏Structure

• Branches consist of multiple project teams

– Project teams focus on business areas (e.g.

Indigenous)

– Project teams run concurrent multiple projects




The Structure of Counting in ESG

FP Count Files

Business

Area A

Business

Area B

Business

Area C

Proposal M

Proposal N

Proposal O

Project

1

Project

2

Project

3

Project

4

Contract

Type W

Contract

Type X

Contract

Type Y

Contract

Type Z

IT APP

A

Project

team

Alpha

Project

team

Beta

Project

team

Delta




The Structure of Counting in ESG

• Every business area has a count

– Client business areas (generally contract types)

– IT business areas (generally service applications)

• Branches consist of multiple project teams

– Project teams focus on business areas (e.g.

Indigenous)

– Project teams run concurrent multiple projects




• Project initiation– Establish project infrastructure

The Counting Workflow

• Initial counts– Validate bottom up estimates

– Sanity check on requirements

• Final counts– Validate initial count, modify as needed

– Feed count into ESG metrics matrix

• project, quarter and FY




• Identifying counts required

– Input from counters and project specialists

Managing the Counting Process

• Creating & distributing count files

– Project Office manages count creation & maintenance

• Reviewing counts

– Certified counter oversights and reviews counts

• Consolidating counts

– For projects

– For business areas (applications)




• Metrics are compiled for each project, all projects in a quarter and for the FY

Managing the Metrics

• The‏KPI’s‏are‏reported‏by:

• $s per FP

• Defects per 1,000 FPs

• Proposals estimated from requirement counts

• Establishing program of work size

• Picking‏‘winners’

• Justifying estimates for funding




Problems and Outcomes

• The Model

• Strengths

• Weaknesses

• Challenges




The ESG Model• Distributed counters

– Ideally at least one counter in each project team

– At worst one counter per project manager




The ESG Model

• Training and Mentoring

– Certified counter mentors during counts, runs

information sessions, etc

• Central management– Certified counter manages the creation and distribution

of count files, tracks progress, performs count reviews, manages reporting

• Distributed counters– Ideally at least one counter in each project team

– At worst one counter per project manager




The Strengths

• Project experts as counters

• Proximity to business knowledge

• Personal knowledge of project

developments




Medium experience Counter

High experience CounterThe Strengths

Group Manager

Dev Br #1 Tech Support BrDev Team #3 Bus Intel BrDev Br #2

Dev Team 2 Developer 1

BA

Developer 3

Developer 4

BA

Developer 6

Dev Team 3Developer 1

Developer 2

Tester

Developer 4

Developer 5

BA


BA

Developer 3

Developer 4

BA

Developer 6


Developer 2

Developer 3

Developer 4

Developer 5

BA


BA

Developer 3

Developer 4

Tester


Developer 2

Developer 3

Developer 4

BA

Tester


Developer 2

Developer 3

Tester

Tester

BA


Developer 2

BA

Developer 4

Lead BA

Developer 6


Developer 2

Developer 3

BA

Developer 5

Tester


Developer 2

Developer 3

Developer 4

BA


Developer 2

Developer 3

Developer 4

Tester

BA


Developer 2

BA

Developer 4

Developer 5

BA


Tester

Developer 3

Developer 4

Tester

BA


BA

Developer 3

BA

Developer 5

Developer 6

Dev Team 1Tester

Developer 2

BA

Developer 4

Developer 5

DW Dev Team BA

Developer 2

Developer 3

Developer 4

Developer 5

BA

DW Analysis Team

Analyst 1

Analyst 2

Analyst 3

App Dev Team Developer 1

BA

Developer 3

Developer 4

Tester

Developer 6

Bus Liaison Team

BA 1

BA 2

BA 3

BA 4

Training TeamTrainer 1

Trainer 2

Coder 1

Coder 2

BA

DBA Team

DBA 1

DBA 2

DBA 3

DBA 4

Special Projects

Team

BA 1

BA 2

BA 3

BA/Tester Mgmnt

Team

BA Mgr

Tester Mgr

Test Tool

BA Tool

Project Office

FP Coord

Schedules

Governance

PM Liaison

Release TeamTech Supp 1

Tech Supp 2

Tech Supp 3

Tech Supp 4

Tech Supp 5




The Weaknesses• Part time counters

• Lack of control over– timing of counts

– counters time (competing priorities)

– counter distribution

• Counters variable backgrounds– BA’s‏are‏best,‏testers‏close,‏developers‏tend‏to‏a‏

technical view

• Potential for PM pressure




High experience CounterThe Weaknesses

Medium experience Counter

Group Manager

Dev Br #1 Tech Support BrDev Team #3 Bus Intel BrDev Br #2


BA

Developer 3

Developer 4

BA

Developer 6


Developer 2

Tester

Developer 4

Developer 5

BA


BA

Developer 3

Developer 4

BA

Developer 6


Developer 2

Developer 3

Developer 4

Developer 5

BA


BA

Developer 3

Developer 4

Tester


Developer 2

Developer 3

Developer 4

BA

Tester


Developer 2

Developer 3

Tester

Tester

BA


Developer 2

BA

Developer 4

Lead BA

Developer 6


Developer 2

Developer 3

BA

Developer 5

Tester


Developer 2

Developer 3

Developer 4

BA


Developer 2

Developer 3

Developer 4

Tester

BA


Developer 2

BA

Developer 4

Developer 5

BA


Tester

Developer 3

Developer 4

Tester

BA


BA

Developer 3

BA

Developer 5

Developer 6

Dev Team 1Tester

Developer 2

BA

Developer 4

Developer 5

DW Dev Team BA

Developer 2

Developer 3

Developer 4

Developer 5

BA

DW Analysis Team

Analyst 1

Analyst 2

Analyst 3

App Dev Team Developer 1

BA

Developer 3

Developer 4

Tester

Developer 6

Bus Liaison Team

BA 1

BA 2

BA 3

BA 4

Training TeamTrainer 1

Trainer 2

Coder 1

Coder 2

BA

DBA Team

DBA 1

DBA 2

DBA 3

DBA 4

Special ProjectsTeam

BA 1

BA 2

BA 3

BA/Tester MgmntTeam

BA Mgr

Tester Mgr

Test Tool

BA Tool

Project Office

FP Coord

Schedules

Governance

PM Liaison

Release TeamTech Supp 1

Tech Supp 2

Tech Supp 3

Tech Supp 4

Tech Supp 5




The Challenges

• Managing competition for counters (with PM)

• Adherence to standards

• Consistency between counts

• Succession planning

• Support‏for‏counters‏‘in‏the‏field’

• Managing and consolidating project counts

• The provision of useful metrics to management




The Challenges – Competition for Counters




The Challenges – Adherence to Standards

• Establishing standards

• Documentation guidelines

• Recommendations on tool use

• Inclusion of reference material

• Reviewing all counts

• Counselling counters




The Challenges – Consistency

• Establishing a culture

• Improve‏‘dodgy’‏counts

• Regular training updates

• Review all counts

• Counsel counters

• Review all counts when the rules change




The Challenges – Succession Planning

• Participate in team construction

• Train for more staff than are needed

• Monitor location and experience of counters




The Challenges – Supporting Counters

• Stay in touch

• Participate in their counts

• Mentor, train, support

• Liaise with the project managers

• Set up an FP counter network

• Support further training and certification




The Challenges – Managing & Consolidating

Counts

• Keep track of the counts

• Police the local standards

• Document deviant counts

• Stay calm, stay calm, stay calm




The Challenges – Management Metrics

• Invest in a tool to store & report data

• Keep project data up to date

• Subscribe to ISBSG, then use it

• Get an estimation tool that uses your data




Where to from here ?• Hopes

– Input to the (re)distribution and tasking of counters

• Plans

– Upgrade the recorded metrics so they can be readily

exported to the ISBSG data base

• Dreams

– Build comprehensive predictive metrics baseline, and

maintain it, and use it




Key Takeaway Points

A model for managing the sizing projects in a complex

development environment

The importance of maintaining counting standards in a

distributed counting model

Remotely managing distributed metrics gathering and

reporting

A workable metrics reporting model for a complex

environment

Things to consider when building a metrics program so

its’‏outcomes‏are‏comparable‏to‏ISBSG‏data




Finishing Up

• Questions ?

• Observations from the floor

• Suggestions

• Discussion

Technology

Administering a substantial fpc program in a large organisation