Task Team 13.5

Background

Task Identified for Team 13.5

• Review the current methodology for the workload, compatibility and location and criteria and make recommendations for either a subjective or analytical (revised) method and review the General Scoring Criteria

Team 13 Approved Business Objective

Developed by Team 13.1

Select a firm based on Select a firm based on qualifications that qualifications that

best meets the needs best meets the needs of the project scope.of the project scope.

Measurement

The objective is being met when at least 85% of the firms doing business with DOTD have past

performance ratings of “meets requirements” or above.

The target date for meeting this goal is 3 years after implementing the “To be” process. From that time forward, this measurement will be taken annually to ensure the selection process is

achieving the stated objective.

Desirable Outcomes

• Encourage Teams and Teamwork among consultants.

• Maximize the opportunity for firms to perform DOTD work for the purpose of maintaining a healthy resourceful consultant community.

– This can be achieved by easing the entry of small firms through the Pre-qualification Selection Program

Law - Weighting Factors

• The current weights for the rating factors are: – Firm Size (3)– Firm experience (3) – Staff experience (4) – Location (4)– Previous work (6) – Workload (5).

Firm Size (3) Definition/Objective

• Existing Definition: – The intent of this category is to optimize the firm’s size (total numbers of full-time

employees) to the size of the project (total estimated cost of the project). A small firm responding on a large project will not score well in this category. Likewise, a large firm responding on a small project will not score well. The “penalty” for being too small is greater proportionately than for being too large. Prime consultant’s compatibility rating will be used for the sub-consultant’s rating when the percentage of the work to be performed by the sub-consultant is small. The same compatibility rating may also be used for all respondents for small retainer contracts. The optimum project size (O) for the firm (in $) is calculated as:

O = (Z) ($3000/person/mo.) (contract time) where Z is the firm size entered on the consultant record.

The cost of the work to be performed by the firm (FEE) is computed as follows:

FEE = sum of (c(i) * p(i)) where c(i) is the project cost of category i, the percentage of category i to be done by the firm

The ratio of the optimum project size to the fee:

     RATIO = O / FEE

Firm Size Recommendation

• Size of firm will be defined as the number of transportation-related employees

• Firms will not rated below 2 for being too big for moderate to complex projects, but can be rated a 0 if too small.

• $3000/person/month will be defined as the salary of an engineer and 4 staff for statewide average.

Determine Firm Size Pros/Cons

• Pros• Will match firm size

with project size or complexity

• Does not penalize larger firms for moderate to complex projects

• Cons

Firm Experience (3) Definition/Objective

Existing • Each committee member independently

evaluates each respondent for firm experience relevant to the project advertised. This category is also concerned with the experience of the top-level managers, as a unit, for the responding firm. This rating is generally obtained from Section 12 of DOTD Standard Form 24-102. For reference, the ratings are based on the following: 0 – very weak; 1 – weak; 2 – acceptable; 3 – strong; and 4 – very strong. The weighting factor for this category is three.

Staff Experience (4)Definition/Objective

Existing• Each committee member independently

evaluates each respondent for key personnel experience relevant to the project advertised at all levels within the firm performing the work on the project. Experience gained with any employer is “counted”. This rating is generally obtained from Sections 10 and 11 of DOTD Standard Form 24-102. For reference, the ratings are based on the following: 0 – very weak; 1 – weak; 2 – acceptable; 3 – strong; and 4 – very strong. The weighting factor for this category is four.

Experience of the Firm and Key Staff – Brainstormed Ideas

Proposed • Combine experience of the firm and key staff as one

criterion with a weighting of 7Instruction for CEC

• Review the relevant projects the firm has completed and ensure the resumes of the employees that have worked on these projects are still employed by the firm.– Firm experience - On the form the firm should only submit similar

projects performed by the employees that are still employed by the firm.

– Key staff - Should submit only information on relevant projects with specifics of their involvement, dates and if they worked for the submitting firm (yes or no) performing that work.

Recommendation

• Combine experience of the firm and key staff as one criterion with a weighting of 7

Instruction for CEC• Review the relevant projects the firm has completed and

ensure the resumes of the employees that have worked on these projects are still employed by the firm.– Firm experience - On the form the firm should only submit similar

projects performed by the employees that are still employed by the firm.

– Key staff - Should submit only information on relevant projects with specifics of their involvement, dates and if they worked for the submitting firm (yes or no) performing that work.

Determine Combining Pros/Cons

• Pros• Hard to separate• The staff is the firm• Evaluating past

projects with different staff is not relevant for a firm

• Cons• May have to change

law• Lessens higher

ratings for firms that have been in business vs. new firms with powerful staff

Location (4) Definition/Objective

Existing • The location factor is based on the distance between the firm’s

office(s) to the project site. The firm’s office will be the location where the key staff and the majority of the personnel performing the work for the project are located. Basically, the closer the firm’s office is to the project site, the higher its score. A coordinate system or a GIS-based procedure is used to compute the distance to the project site. For district- and state-wide projects, coordinates for the District Office or Marksville (for state-wide) will be used for the project location unless otherwise noted in the advertisement. The weighting factor for this category is usually four. For Urban System projects, the factor is increased to six. The table below describes the rating. – Distance Rating 0 To 50 Miles = 4

50 To 450 Miles = Interpolated between 4 and 0More than 450 Miles = 0

Location Recommendation

– For statewide and specialty projects this factor will be neutralized and everyone will receive the same score if recommended by the PM or use Marksville, LA or the district office as the project location.

– For other specialty projects the weighting factor can be increased in the advertisement to ensure a local firm has more consideration if recommended by PM.

Determine Location - Pros/Cons

• Pros• More input from PM

• Cons• Statewide contracts

will have out-of-state firms with same advantage as in-state firms

Past Performance (6) Definition/Objective

Existing• This rating is based on the results of input from the Project Managers at various

project milestones. They evaluate the firms that they have managed within the last five years. The weighting factor for this category is six.

The firms are evaluated in five categories: knowledge and technical expertise, quality of plans and other deliverables, completion of work within the terms of the contract, cooperation and problem resolution, and communication and documentation. Project Managers rate on the same 0-4 point scale that is used by the CEC. For reference, the ratings are based on the following: 0 – unsatisfactory; 1 – needs improvement; 2 – meets requirements; 3 – exceeds requirements; and 4 – outstanding. The ratings are given for several categories of work. Project specific ratings may also be obtained from the pertinent department personnel. Each firm receives the average rating from all Project Managers who have evaluated them. Project Managers’ ratings are subject to a review by their direct supervisors. The Committee screens the Project Managers’ ratings for: those who may grade either too strictly or too leniently; consistency; grading which conflicts markedly with other coordinators, or with the Committee’s perceptions; etc. The Committee reserves the right to investigate any discrepancy they may identify, and change or eliminate an old rating, if justified. To date, changes have been rare.

Only those firms who have had projects monitored by DOTD within the last five years are rated. Firms who have not received a rating for a work category are given the average of all respondents who have a rating, subject to a maximum limit.

Past Performance Recommendation

– Leave as 5-years pending the recommendation from 13.2 - 3-year existing to 5-year proposed phased in over 2-years

– Use the average of the other firms submitting on the project for firms having no Past Performance rating in system

Determine Past Performance Pros/Cons

• Pros• Allows new firms to have

an opportunity• Rewards firms that have

performed well for 5-years

• The phase-in allows the consultant to be on notice to closely monitor their ratings without being penalized for the past

• Cons• A bad rating within the

past 3-years could penalize the firms that have worked to rectify the problem

Workload (5) Definition/Objective

Existing • This category considers the firm size (as explained in

COMPATIBILITY), and the workload with the Department in dollars. Projects that are “counted” are: all contracts with others on behalf of the Department using Department administered funds; all contracts with others with supervision provided by the Department; and all contracts with the Department. Note that a project is counted as soon as the selection is made. For retainer type contracts, only the remaining balances on open Task Orders are counted. The maximum workload for a firm with DOTD equals the firm size multiplied by $75,000. The greater the firm’s workload is, the lower the rating in this category. A firm with no workload with DOTD would receive a rating of four. The weighting factor for this category is five.

Workload Recommendation

• Segregate by complexity of project – Simple and moderate projects leave formula as is. – Complex projects neutralize by assigning same value to all

• Review $75000 and recommend to increase – Ed will review history and make recommendation (It was $75,000 in 1990)

• If a project is projected to be on “hold” or suspended for 6 months or greater, the remaining fee is not “counted” against the consultant in section 13 of the 24-102 (with approval of the DOTD PM)

Determine Workload Pro/Con

• Pros• Increase $75,000 -

Firms could carry a higher $ value on books

• Segregate Work – Complex projects would encourage a QBS selection

• Cons• Penalizes firms for

having work• Allows firms that have

work to get more