Guidelines for Successful ITS Procurement T3 Session, March 15, 2006 Hosted by Mac Lister FHWA Presented by Phil Tarnoff University of Maryland

Guidelines for SuccessfulITS Procurement

T3 Session, March 15, 2006

Hosted by Mac Lister

FHWA

Presented by

Phil TarnoffUniversity of Maryland

The Problem

• Success rate for intelligent transportation systems life cycle is very low

• Problems can be traced to:– Incorrect contracting approach– Inexperience of the agency– Failure to follow appropriate procedures– Inadequate commitment of project

management and systems engineering resources

Systems Acquisitions

53%

16%

31%

Late

Okay

Cancelled

Source: Patterns of Software Systems Failure and Successes, Jones, Capers, 1996

Success is Influenced by the Purchaser

• Purchaser’s experience has greater impact on project success than any other factor

• Success rate has increases by more than 25% for purchasers with prior experience

• ITS implementation must be led by experienced staffSource; Patterns of Software Systems Failure and Successes, Jones, Capers, 1996

The Contracting Model


• Defines the entire contracting form and process

• Includes four dimensions:– Work Allocation – Responsibilities– Method of Award – Contractor selection– Contract Form – How is work organized– Contract Type – Methods of reimbursement

• Terms and conditions apply to all dimensions


Terms and Conditions (payment, cancellation, disputes, etc.)

Procurement

Work Allocation

Method ofAward

ContractForm

ContractType

Low Bid ContractorSystems ManagerSystems IntegratorDB(OM)Commodity (COTS)Consultant ServicesOutsource ContractorOther Services

Low BidNegotiatedSole Source

PhasedTask OrderPurchase Order

Fixed PriceCost ReimbursableIncentiveTime and Materials

Use of the Contracting Model

• Identifies dimensions to be included in selection process

• Procurement guidelines define the process for choosing appropriate option for each dimension

• Process is simplified by assembling combinations of options – procurement packages

• Project may include multiple contracts and procurement packages

System Procurement Packages (1 of 2)

1) Commodity Supplier– Low-bid selection of prequalified packages– Fixed price contract– Applicable only for unmodified off-the-shelf

software and hardware

2) Low-Bid Contractor with Design Consultant (for 100% design)– Low-bid selection– Fixed price contract– Can use incentives– Can use phased contracts– Useful if the predominant software is off-

the-shelf

System Procurement Packages (2 of 2)

3) Systems Manager– Negotiated procurement– Fixed price, cost plus or time & material contracts– Can use incentives– Can use either phased or task-order contracts– Separate low-bid procurements required for

construction and equipment

4) Design-Build Contractor with Design Consultant (for 30% design)

– Best value selection– Fixed price. Cost plus or T&M may be acceptable– Can use incentives– Can use phased contracts

Other Procurement Packages

5) Consultant– Negotiated– Fixed price, cost plus or time & material contracts– Can use incentives– Can use either phased or task-order contracts

6) Outsourcing either an activity (such as maintenance) or an entire function (such as traveler information)– Low-bid selection may be based on rates– Fixed price or time & material contracts– Can use incentives

Contracting Considerations

Selecting the Best Procurement Approach

Organizational Level•Experience•Resources•Personnel

Project Category•New or replace.•Size & complex.•Uniqueness

Characteristics Contracting Solutions

System Development Process

System Development Process

Procurement Package

Terms and Conditions

Defining Organizational Level

CharacteristicImmature

OrganizationMature Organization

Personnel ExperiencePart time, personnel have no prior experience

Full time responsibility of experienced personnel

Organizational Experience

Never done it before Experienced with 1 or more complex projects

Organizational Structure

ITS responsibilities undefined

Single organizational unit responsible for all ITS

Resources No defined ITS budgetITS budget for systems and personnel

Management SupportModest mid-level support

Considered a priority by senior management

Expectations Not definedIncluded in agency’s planning process

Defining Project Complexity

Characteristic Simple ProjectVery Complex

Project

Newness Off-the-shelf solutions Invention(s) needed

Scope Single function Multi-function system

Interfaces NoneBoth internal & external

Maturity Many similar systems Never been done

StabilityRequirements well defined

Not sure what is needed

InstitutionalBeing developed for single agency

Many agencies involved

The Outputs

• Systems development process– Waterfall– Evolutionary– Spiral

• The procurement package• Lists of terms and conditions

Time

Conceptof Operations

High LevelRequirements

DetailedRequirements

High LevelDesign

DetailedDesign

Implementation

Operations &Maintenance

SystemVerification

SubsystemVerification

Integration &Test

The Development Processes are Based on the Systems Engineering Life Cycle

Systems Development Processes

• Waterfall – A linear process for well-defined programs including all the activities of the “V” diagram

• Evolutionary – Development of a project in a series of well defined phases in which the “V” diagram is repeated for each phase

• Spiral – Not certain of what is needed, so project development accompanied by much prototype development and many planning steps

The Decision Process

Initial Decisions – Step 1

Does the agency intend

to outsource?

Other services being procured.

Not covered by Decision Model.

Use consulting process

(procurement package #5)*

Start

No

Yes

Yes

Are traditional

consulting services being procured?

Yes No

No

Does the project

include a system development?

Use outsourcing process

(procurement package #6 or #7)*

Go To Step 2

Decision Model

Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES

Send Individual Projects through

the Model

Step 5 Select

applicable systems

engineering process(es) &

candidate procurement package(s)

Step 6

Apply Differenti-

ators

Step 7

Package Assessment

and Final Selections

Step 8 Define

Contract Scope and Terms and Conditions

Schedule ConstraintsStart

End

The Fundamental Activities of the Following Process

• Separate the project into individual contracts

• Determine project and agency characteristics

• Select procurement package• Apply discriminators to multiple

solutions• Confer with Contracts• Add terms and conditions

Decision Model – Step 2

Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

Initial Work Allocation (Step 2)• General rules:

– In general, it is best for all work to be contained in a single prime contract (exceptions noted below)

– Minimize interdependency of contractors (i.e. where one contractor depends on another contractor’s completion)

– Multiple contracts require increased project management resources

• Reasons to consider multiple prime contracts:– Significant software and systems development, but

largest dollar amount is in construction (i.e. systems contractor would not be prime under a single contract)

– Unlikely that a satisfactory prime contractor can be identified

– Significant outside expertise required

Decision Model Step 3

Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

Identify Project Category (Step 3)

Category 1

Category 2

Category 3

Category 4

Complexity

Cells of table contain description of the characteristic appropriate to each category

Development

Technologies

Interfaces

Evolution

Requirements

Risk

Examples

Example of Cell Content:“Primarily application of proven well-known technology. May include non-traditional use of existing technolog(ies)”


Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

Select Organizational Levels (Step 4)

Level 1 Level 2 Level 3

Personnel

Cells of table contain description of the

characteristic appropriate to each category

Organizational Experience

Resources

Organization

Management Support

Expectations

Example of cell content:“Experience with at least one category 2 project or greater.”


Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

The Decision Matrix (Step 5)Project Category Organizational Level

Level 1 Level 2 Level 3

1 – Low •Waterfall•SM or DB*

•Waterfall•Low Bid*, commodity, SM or DB

•Waterfall•Lob Bid, Commodity, SM or DB

2 – Moderately Complex

•Evolutionary•SM or DB*

•Waterfall or evolutionary•Low Bid*, SM or DB

•Waterfall or evolutionary•Low Bid, SM or DB

3 – Complex Not recommended •Evolutionary•SM or DB

•Evolutionary or spiral•SM or DB

4 – Extremely Complex

Not recommended •Evolutionary or spiral•SM or DB

•Evolutionary or spiral•SM or DB

Notes:First line is the systems engineering technique, second line is the procurement packageDB = Design-BuildSM = Systems Manager* - Consulting services should be used while project is underway


Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

Procurement Differentiators (Step 6)

• Systems manager is preferred to design-build for significant new software development

• Design-build preferred over systems manager, only for major projects and when significant amounts of field construction are involved

• If project includes both new software and field construction, consider using multiple contracts

Procurement Differentiators continued (Step 6)

• Low-bid contracting should only be used if:– Required by agency policy (rarely true)– Projects consist of field construction and off-

the-shelf equipment

• Commodity procurement applicable if an available packages do not require modification except for:– New software drivers for interface with

communications and field equipment– New database reflecting system configuration– New graphics (maps, etc.) for local

environment

Impacts of Step 6 Decisions (Step 6 continued)

• Low-bid contract will require design consultant & low-bid contractor

• Systems manager contract will require low-bid contractors for field construction and field equipment supply

• Design-build contract will require design consultant and design-build contractor

• Commodity procurement may require systems integrator for system implementation and test


Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

Considerations When Making the Final Decision – Step 7

• Involve the agency’s contracts personnel (They should have been involved from the beginning)

• Compatibility with the systems engineering and project management principles

• Agency’s relative familiarity with the recommended alternatives


Step 2

Work Allocation

Step 4

Determine Agency

Step 3

Define Project

Categories

NO

YES


the Model

Step 5 Select

applicable systems



Step 6

Apply Differenti-

ators

Step 7

Package Assessment


Step 8 Define



End

It is Then Necessary to Define Terms and Conditions

Contract Terms and Conditions (Step 8)

• Some are common to all procurement packages, others are unique for each package

• Requires contracting expertise• To a certain extent, by state law,

agency policies, and federal law• Defined by the NCHRP documentation

Sample Terms and Conditions (Step 8)

Commodity Supplier Terms and Conditions

Low-Bid Contractor with Design Consultant

Systems ManagerDesign-Build

Contractor with Design Consultant

Consultant Outsourcing Agency Activity

Outsourcing Agency Function

Contractor Inspection Requirements

Design within Funding Limitation

Negotiation Negotiation Negotiation Negotiation Negotiation

Inspection of Supplies Redesign Responsibility for Design Errors or Deficiencies

Commercial Computer Software Restricted

Design within Funding Limitations

Notice of Cost Comparison

Fixed Fee Fixed Fee

Option for Increased Quantity

Deficiencies Rights Redesign Responsibility for Design Errors

Allowable Costs and Payment

Incentive Fee Incentive Fee

Ordering Fixed Price Fixed Fee Work Oversight Fixed Fee Work Oversight Work Oversight

Definite Quantity Incentive Fee Incentive Fee Suspension of Work Incentive Fee Execution and Commencement of Work

Execution and Commencement of Work

Indefinite Quantity Performance/Payment Bond

Rights in Data Fixed Fee Performance Based Payments

Performance/Payment Bond


Brand Name of Equal Allowable Costs and Payment

Incentive Fee Delivery Orders (Task Orders)

Allowable Costs and Payment

Allowable Costs


Performance Based Payments

Execution and Commencement of Work

Specifications Performance Based Payments

Modifications

Delivery Orders (task orders)


Delays and Extensions of Time

Modifications Rights in Data

Specifications Specifications and Drawings

Modifications Rights in Data

Summary of the Seven Steps to Implementing the Process

Step (1) Initial Decisions

Step (2) Allocate the work

Step (3) Select project category

Step (4) Determine agency level

Step (5) Using organizational levels and categories, select Procurement Package(s) for each project

Step (6) Apply procurement differentiators to make final selection

Step (7) Make final selection

Step (8) Define contract terms and conditions

Contract process is complete

Key to a Successful Procurement

• Procurement approach is dependent on project type and agency capabilities

• We’ve always done it that way is not a good excuse

• The right procurement approach may not guarantee success, but the wrong approach will guarantee failure

For More Information

• http://www.citeconsortium.org on home page under highlights, select “Guide to Contracting ITS Projects” to access web-based step-by-step selection process

• http://www.pcb.its.dot.gov/Catalogs/ITSCurriculum.htm for courses on ITS procurement, systems engineering and project management

Contact

Mac Lister

Phil [email protected]

[email protected]

Documents

Guidelines for Successful ITS Procurement T3 Session, March 15, 2006 Hosted by Mac Lister FHWA Presented by Phil Tarnoff University of Maryland