Tracey Clavell Head of Estimating and Pricing – Corporate Defence Cost Estimating Conference Melbourne Oct 2012 1 Commercial in Confidence

Tracey ClavellHead of Estimating and Pricing – Corporate

Defence Cost Estimating Conference Melbourne Oct 2012

1Commercial in Confidence

2

Estimation Approaches, Methods &Techniques

Structure/Approach

Top Down

Method Technique

Bottom Up

Parametric

Analogy

Expert Judgement

Delphi Forums

Algorithmic

Build up

Historical

Single Point

Two Point

Three Point

Commercial in Confidence

Accuracy V’s Technique V’s Requirement


Case Study 1 - Background

• Software product only (integration onto the IT infrastructure was not in scope)

• Intelligence data gathering and presentation system• High visibility product• Some COTS software, some developmental• Two iterations:

– Initial parametric estimate to verify team’s ROM

– Formal response, short response period, parametric estimate to verify bottom up


Engagement Process

“We should brief the team in Canberra on the current state of parametric modelling and bid support”

“Thanks for the briefing. Now that you are here, I have a job you can help us with. Are you free to pop upstairs to meet the team?”

Commercial in Confidence 5

Initial Model Development

End User Requirements Document

1.Extract Transform and Load 2.Mine data to extract meaning3.Store data and meaning4.Construct specialised interactive presentations5.Present developed materials6.Replay presentations7.Package presentations in a canned non interactive format • Bid Support Engineer came up with the initial PBS

based on the general user statements• Nature of each component modelled from Engineer’s

understanding of solution, and from probing the team• DBMS added – as we would need such underlying

technology• System Admin added – user rights management etc


Next Iteration

• Team saw the first cut– Yes it matches the user needs

– There are a couple of things that don’t quite match how I think I will build this

• Team had a draft system architecture document– Broke up the product into 3 separate products

– Had a little more clarity around what each component did

• So version 2 of the PBS was created


Version 2 of the PBS

1. Three separate products;2. Components will be COTS;3. Admin required for more than just users


Engagement Process – Part 2

“Here’s your parametric estimate report. Let us know if you have any questions, or if you want us to do more for you.”

“Thanks. That stacks up really well to our ROM. That’s all we need for now, but we’ll be back in touch later.”

“Guys, are you able to update your parametric model for us? We have just received the formal requirements, and our response time is really short.”


CIFS Document stores

Objective Document

stores

Data Bases

READ from Source

Application as a Service space

Extract Transform

Load

Data Store

Meaning Extraction

Extract Transform

LoadGraph Creation Graph Storage

VIEW Data Sets

HMI Views

Presentation creator

Presentation Delivery

VDIJVM & Browser

? Other code

Possible replace by NGD

DSN SOE 125 Client

HTTP or RDP

Static Presentation packages – pdf / html

DSN file store or

web serverFile Transfer

DIGO Map & Image

LDAP Authenticate

Access

Read

and

lim

ited

agre

ed w

rite

NGDDefault

HTTP or RDP

Team Had Developed a Product Architecture


Mapping Architecture to Our PBS


Objective Document

stores

Data Bases

READ from Source


Extract Transform

Load

Data Store

Meaning Extraction

Extract Transform


VIEW Data Sets

HMI Views



VDIJVM & Browser

? Other code


DSN SOE 125 Client

HTTP or RDP


DSN file store or


DIGO Map & Image

LDAP Authenticate

Access

Read

and

lim

ited

agre

ed w

rite

NGDDefault

HTTP or RDP

Not in our scope


This Wasn’t Quite Right

• And also …– They changed the delivery strategy

– Some COTS products had now been identified and chosen

– And not everything was going to be COTS


A New PBS Emerged

• Three Phases (where Phase 1 was system analysis and design)

• Team structure was solidified• The team’s requirements of us

changed– They needed resource loading as

well

• We got a little more clarity on the COTS products– This slightly changed the PBS, as

some items needed to be divided up or merged

Commercial in Confidence13

Now How Does the Mapping Look?


Objective Document

stores

Data Bases

READ from Source


Extract Transform

Load

Data Store

Meaning Extraction

Extract Transform


VIEW Data Sets

HMI Views



VDIJVM & Browser

? Other code


DSN SOE 125 Client

HTTP or RDP


DSN file store or


DIGO Map & Image

LDAP Authenticate

Access

Read

and

lim

ited

agre

ed w

rite

NGDDefault

HTTP or RDP

Compared to architecture•“HMI Views” was split – 2 products used•Presentation creator and delivery merged•Otherwise maps pretty well – in fact better than my first attempts


Summary Case Study 1

• It was iterative – which was expected• The first cut at understanding the end product, while not exactly

accurate, prompted discussion, and in fact influenced design decisions– Or at least made the team make a decision

• The final PBS reflected not just the end product but:– The delivery strategy of 3 phases

– The team structure that they had now decided upon

– The division of functionality among COTS products, which had previously not been chosen

• Not shown here is how the parameters of each item were developed– Bid Support Engineer made a start, from the architecture description, and the

team’s explanation of what each component did


Background Case Study 2

• Distributed systems, loose integration in most cases• COTS software throughout• Request for Proposal

– Customer asked for a ROM to ± 30% accuracy

– Team planned to use parametric estimate as the submitted ROM

– Early parametric model used to guide a Design to Cost workshop

• Design to Cost Workshop identified 4 separate trade offs to perform, each requiring parametric modelling to quantify the differences


Engagement Sequence

Initial Request for

ROM

Initial Parametric

Model

Design to Cost

WorkshopFour Key

TradeStudies

Baseline Parametric

Model

Updated Solution

Option 1Parametric

Model

Option 2Parametric

Model

Option 3Parametric

Model

Option 4Parametric

Model

Initial Solution

ROM Submission


Initial Model Development – One Page Architecture

Page 1

Preliminary Architecture

Intranet

Launched From

Ethernet

Sensor data

DGST REA System

Mission Pack

ENC/AML/Images

GIS ServerESRI ArcGIS

GIS ClientESRI ArcPAD

REA Laptop REA Processing

Portable Mediaw/ Sensor Data

Launched From

METOC Laptop

MMT System

Air sensors

GIS Viewer/Decision Aids

Model and satellite data

Ships network

Development

COTS Integration

COTS Mature

In scope

Out of scope

Existing

Fixed Support Cell

Classified Geo Support Cell

GIS System

Bathymetry and CTD data

Deployed From

Air sensors

Deployed From

Host Vessel

Leeuwin Class / OCV

AUV System

AUV C2 & PMA

Hydroid REMUS VIPw/ SeeTrack Military

REMUS 100w/ CTD and Bathy Sonar

Water sensors

YSICastaway CTD

REA Support Vessel

Zodiac7.2m


Generated Model Structure – Compressed

Page 1

Preliminary Architecture

Intranet

Launched From

Ethernet

Sensor data

DGST REA System

Mission Pack

ENC/AML/Images

GIS ServerESRI ArcGIS

GIS ClientESRI ArcPAD

REA Laptop REA Processing

Portable Mediaw/ Sensor Data

Launched From

METOC Laptop

MMT System

Air sensors

GIS Viewer/Decision Aids

Model and satellite data

Ships network

Development

COTS Integration

COTS Mature

In scope

Out of scope

Existing

Fixed Support Cell

Classified Geo Support Cell

GIS System

Bathymetry and CTD data

Deployed From

Air sensors

Deployed From

Host Vessel

Leeuwin Class / OCV

AUV System

AUV C2 & PMA

Hydroid REMUS VIPw/ SeeTrack Military

REMUS 100w/ CTD and Bathy Sonar

Water sensors

YSICastaway CTD

REA Support Vessel

Zodiac7.2m


Structure Details

• Clearly the PBS required a bit more than the one page diagram• We built an initial PBS, and it refined over a short discussion into the one

on the previous slide

• The RFP documentation also arrived just after the initial PBS, and helped expand and/or change the initial structure

• All through this period, the Bid Support Team had weekly video conferences with the technical team


Engagement Process – Design to Cost Workshop

“Now that we have an initial model, we believe that the Engineering Function’s Design to Cost workshop would be of use”

“Sounds like a good idea. We can also help refine the details of the model while you are here”


Engagement Process – Design to Cost Workshop

“Now that we have an initial model, we believe that the Engineering Function’s Design to Cost workshop would be of use”

“Sounds like a good idea. We can also help refine the details of the model while you are here”

“Hello, we’re from the Engineering Function, and we’re here to help!”


DTC Worksheet – How the Parametric Model Was Used

Generate some pie charts to understand % breakdown of costs


DTC Worksheet – How the Parametric Model Was Used (2)Need a cost target to start everything

All yellow cells in this column are from pie charts or model data

All yellow cells in this column are from pie charts or model data

Percentages are used to derive the targets

Percentages are used to derive the targets

All green cells in this column are from model

data – shows where target < estimate

All green cells in this column are from model

data – shows where target < estimate

And this data is not from the parametric model, but from company past project

data and rules of thumb


What Did the DTC Workshop Uncover

• Our estimated price was higher than the target (but not by much)• We identified where the greatest “bang for buck” would be – this is the

major point of the DTC workshop– This led to 4 specific technical trade studies

– Each trade study option meant a separate model

– Each new model was to be compared to the baseline solution, giving information to inform decisions


Summary Case Study 2

• Parametric estimation was pivotal to this RFP in two ways:– It was the primary estimating technique for the submission

– It was central to the Design to Cost process

• This particular solution was quite volatile, so weekly video conferences were helpful, BUT …

• Being able to visit (for the DTC workshop) was even more useful


Summary

27

• Left Shift • Engage face to face initially • Design to Cost Workshops• Lever Analysis • What if Analysis • Ensure Independence• Ensure Requirements / Scope is correctly modelled

• Analyse, question, reiterate

• Quick Turn around Times


Accuracy V’s Technique V’s Requirement

28

• Decision support tool which works well if supported by a sound process

• Don’t be afraid to utilise Parametric Analysis

• Imagine how powerful this technique could be when combined with Uncertainty Cost Analysis ?

• Imagine how powerful this technique could be to compare to Bottom Up?

• Determine the Accuracy needed at what point in the process you are– This may change your view on what technique or techniques are needed– Ensure you are modelling the Requirement / Design – Utilise Design to Cost


Documents

Tracey Clavell Head of Estimating and Pricing – Corporate Defence Cost Estimating Conference Melbourne Oct 2012 1 Commercial in Confidence