Software metrics in contracts

Status update Contract Metrics

CommitteeNesma 2016

Hans BerninkHans Kuijpers

KK

Who is whoHans Kuijpers

▪Software Improvement Group▪Senior Consultant▪Certified Scope Manager▪Professional Scrum Master I

▪Nesma:▪Contract Metrics Committee (chair)

h.kuijpers@sig.eu

software metrics in contractsK K

Who is whoHans Bernink

▪Metrieken.nl▪Senior Metrics Consultant▪Senior Sizing Analyst▪Certified Function Point Analyst

▪Nesma:▪Contract Metrics Committee▪Basis of Estimate (BOE) Committee

software metrics in contracts

hans.bernink@metrieken.nl

K K

Background / History▪Nesma wg Metrics in Contracts▪Start July 2013▪7 people▪Hans Kuijpers, Software Improvement Group

(SIG)▪Hans Bernink, Metrieken.nl▪Rini Scholten, Kadaster▪Frans Schoot Uiterkamp, BrainCap▪Jolijn Onvlee, Onvlee Opleidingen & Advies▪Marcel Pereboom, Mediaan▪Tim Schijffelen, Sysqa

software metrics in contractsK K

Assignment

“Draw up a NESMA document that serves as a guideline (or perhaps even more!) for client organizations and suppliers in order to use function points and quality

metrics in RFP’s and in contracts regarding software development

and maintenance.”

software metrics in contractsK K

Why a Guideline▪Outsourcing software projects is a challenge

▪Organizations struggle to select the best supplier▪Attempt to use software metrics often fail▪Lack of standards / guidelines / best practices▪(Wrong) software metrics are used in a ‘wrong’

way

▪Some organizations select a ‘wrong’ supplier, often resulting in ‘failing’ projectssoftware metrics in contracts

K K

Standish benchmark

software metrics in contractsK K

Low maturity of the industry▪Lack of knowledge ▪Low maturity in cost engineering practices▪Wish: ▪ select ‘the right’ partner▪ based on objective criteria

▪Even companies that use metrics based on functional size often use these in a wrong way▪Many suppliers don’t measure completed projects and have

no idea about their performance

software metrics in contractsK B

Example▪RFP framework question: ▪What is your price/FP

for a Java project of 500 FP and 5 month duration?

FPA and quality metrics in contractsB B

Scope of the guideline▪IFPUG/COSMIC/NESMA function points▪Technical quality metrics (ISO 25010)▪Software development + Software

maintenance▪Waterfall, Agile/Scrum, DevOps et al

▪Contracting phase (e.g. RFI / RFP)▪Single project / contract▪Maintenance contracts▪Framework: many projects / applications

▪Execution phase: development / maintenance▪All technologies software metrics in contracts

B B

▪The guideline will result in 13 guides;1. Guideline for the use of software metrics in contracts

(ready)2. Mini Guide for Development Methodologies

(2016)3. Mini Guide for Software Metrics in Maintenance Contracts (2016)4. Mini Guide for Management

(ready)5. Mini Guide for RFP questions

(ready) 6. Mini Guide for Functional Quality

(2016)7. Mini Guide for Pricing Mechanisms8. Mini Guide for Technical Quality

(ready) 9. Mini Guide for Assessing Suppliers Performance

(ready) 10. Mini-Guide for Software Metrics in Development Contracts (2016)11. Mini Guide: Requirements for Supplier organizations12. Mini Guide: Requirements for Customer organizations

(2016)13. Mini Guide for Measurement Standards and Definitions

(2016)

Current state of Guideline

software metrics in contractsB K

[2] Development methodologies▪Several ways to develop software▪Most prominent methods:

Waterfall, Agile, DevOps, Continuous delivery ▪Performance indicators should: ▪ Focus on the development method, and▪ Support the way of working

FPA and quality metrics in contracts

2016

K B

[3] Maintenance▪80% of system lifecycle cost is maintenance

related▪Focus: how to demand for maintainable

software▪Metrics for contracting and execution ▪Performance indicators are defined and

measured▪Supplier performance measured & improved

2016

B K

[4] Management▪Why are FPA and quality metrics useful for

successful projects?▪Select and contract a professional partner▪Stay in control during the contract phase▪Maintenance phase▪Objective metrics and benchmarking

▪The ‘which and how’ questions are answered in the other MGs

software metrics in contracts

READY

K B

[5] RFP Questions ▪Customers often ask questions that won’t

enable them to make objective comparison▪This MG shows: ▪Models from literature that are relevant in composing RFP questions▪Effect of duration on an estimate▪Effect of selecting an optimistic offer versus realistic and pessimistic offers

▪Possibilities to objectively compare suppliers’ bids using benchmarking ▪and more

software metrics in contracts

READY

B K

[6] Functional Quality ▪Outsourced projects often deliver too early▪How can the customer determine that the

requested functionality is ready for production?

▪This MG helps in managing expectations:▪Project delivers on time with the agreed quality

▪Useful metrics:▪Defect potentials▪Defect Removal Efficiency▪Trend▪Bonus/Malus scheme for Early delivery/Low quality software metrics in contracts

2016

K B

[7] Pricing Mechanisms ▪MG shows the different pricing mechanisms that

are used in the industry, such as:▪Price/FP▪Fixed price▪Time and material

▪Pricing mechanisms advantages and disadvantages▪MG discusses the way software metrics can be

used to overcome the disadvantages▪Example: use productivity targets in T&M

contracts software metrics in contractsB K

[8] Technical Quality ▪Main metrics: time, budget, functional quality▪The 4th metric for outsourcing is technical

quality▪Good code quality improves:▪Defect Resolution Time▪Lifecycle Cost

▪KPI’s for maintainability, security, reliability, performance

READY

K K

[9] Assess Suppliers Performance▪Some organizations outsource all of their

software development and maintenance to 1 supplier▪It’s crucial to select the right supplier

▪This MG shows a model on how to compare different suppliers in an objective way▪Basic idea: suppliers send in data of x

completed projects. The model then assesses:▪Compliancy to requirements of projects▪Reality value of data submissions▪Productivity/Quality (to each other and to external Benchmarks)

software metrics in contracts

READY

K B

[10] Metrics in Development Contracts

▪Use of metrics in project contracts▪RFI / RFP▪Execution / Monitoring▪Completion

▪FPA and other metrics used in all lifecycle phases▪Project Estimating▪Supplier selection▪Project Control: cost, time and quality▪Benchmarking

software metrics in contracts

2016

B B

[11] Requirements for Suppliers

software metrics in contracts

▪Suppliers need to implement Performance measurement processes

▪Using standards is crucial:▪Project classification▪Size measurement (ISO standard)▪Data collection (standard WBS in effort

registration)▪Definitions should match benchmarks, e.g.:

▪Hours/FP, Price/FP: which activities included/excluded per project type?▪Defects/FP: Which defects included/excluded?▪Delivery Speed: Per manmonth or calendar month?

B K

[12] Requirements for Customers▪Business – IT demand – Supplier▪Customers need to have knowledge of how

to▪Demand for successful projects▪Governance the contracts

▪If not available in your organization ➔ invest in it

▪Using standards is crucial:▪Estimation based on size measurement (ISO

standard)▪Technical quality is measurable (ISO standard)▪Governance by KPI’s▪Benchmark supplier performance

software metrics in contracts

2016

K B

[13] Standards and Definitions▪This MG describes Base measurements and

Metrics used in the other MG’s

▪Analyzation of Common problems with metrics ▪Using metrics the right way▪How to construct Metrics:▪Define Base Measurements

▪Base measurements need to be objective▪Relevant (ISO) standards, definitions, guidelines▪Data collection

▪Define Explicit Metrics calculation rule ▪Usage of metrics described▪Lots of examples

software metrics in contracts

2016

B B

Questions ?

contract-metrics@nesma.org