Source Selection TrainingFor ServicesInstructor: Leslie S. Deneault

Plan of the HourWhat is unique to services source selection

Consider the New SS Procedures Review some GAO Cases

*

Director, Defense Procurement & Acquisition Policy Memorandum*

Pre-Solicitation Activities Acquisition PlanningThe risk assessment done as part of Acquisition Planning will be critical in developing evaluation factors.Review of the Acquisition PlanningAcquisition Strategy for non IT services > $1B approved by AT&L or designees> $500M for IT services DPAP Peers Reviews if value > $1BNon-competitive contracts just changed to > $500M*

Pre Solicitation ActivitiesStresses the need for market research and industry involvementIndustry Days highly recommended for all acquisitionsFollowed up with One on One Meetings (Not in SSP) but permitted by FAR

Draft Requests for Proposals highly recommended for all acquisitions

*

Prepare the Request for Proposal

Evaluation Criteria must include cost or price and the quality of the serviceQuality is considered byPast performanceManagement capabilityPersonnel QualificationPrior Experience Compliance with the solicitation requirementsTechnical excellence*

Past Performance is more important in ServicesGood performance is a strong indicator of future performanceServices work is frequently commercialNot inventing something newIt just makes sense to evaluate their previous work performanceMust prepare PP evaluations >1M FY 2012 NDAA; timely and complete PP data Must evaluate PP in source selections of services > $1M

*

*Evaluation and Decision Past Performance (Relevancy on Recent Effort)

*Evaluation and Decision Past Performance (Performance Confidence Assessments)

Importance of Evaluation CriteriaOften there are many suppliers of the serviceWrite evaluation criteria that differentiates between the many contractors in order to pick best value Must make the effort to:Understand the risks in the workUnderstand the marketplace

*

Importance of Evaluation CriteriaWith a Performance Work Statement, we will get differing proposalsEvaluation criteria must be written to evaluate results, not one method of doing the work

*OR

In reviewing protests of an agencys evaluation and source selection decision, even in a task order competition as here, we do not reevaluate proposals but examine the record to determine whether the evaluation and source selection decision are reasonable and consistent with the solicitations evaluation criteria and applicable procurement laws and regulations. GAO Perspectives

GAO CaseB-401889 Best Value Source Selection: Overall rating of non-cost factors ( Qualifications, Mgt, Technical, Past Performance, Small Business Participation) are more important than price.

OfferorsOverall RatingOverall RiskEvaluated PriceProposal AGoodLow Risk$94,705.72Proposal BGoodLow Risk$91,721.82Proposal CMarginalHigh Risk$88,747.12Proposal DAcceptableLow Risk$88,274.00Proposal EMarginalHigh Risk$86,375.74Proposal FMarginal High Risk$85,456.74Proposal GAcceptableLow Risk$77,271.28Proposal HAcceptableModerate Risk$66,535.87

Case 4 - Coastal EnvironmentsThe Army SSA picked Proposal A, Ecological Communications Corporation (ECC) because of ECCs technical superiority.Proposal G, Coastal Environments protested because they were low cost and had an acceptable rating with overall low risk.The protest is sustained because the SSA conducted a tradeoff between the two highest-rated , highest priced proposals, but did not consider in its tradeoff decision the lower prices offered by other lower-rated offerors, whose proposal were rated technically acceptable with low risk.Lesson: Cannot ignore lower-priced technically acceptable offerors. A proper tradeoff decision must provide a rational explanation of why a proposals evaluated technical superiority warrants paying a premium.

Some Other Lessons Learned from Protest CasesPrice Evaluation must meaningfully consider priceAccess Systems Inc, B-400623.3Task order on a GWACs contractProtest sustained because documentation did not justify paying the price premium for superior capabilities. If you are going to pay a higher price for technically superiority, the SSDD must explain why.

Some Other Lessons Learned from Protest CasesCannot exclude a proposal from competitive range without considering priceARC Tech Inc, B-400325.38 (a) set aside for multiple award ID/IQ contract10 proposals; cut the 7 lowest scored technical proposals from the competitive rangeProtest sustained because: An agency may not exclude from the competitive range a proposal that has not been determined technically unacceptable without taking into account the proposals price.

Some other Lessons Learned from Protest CasesA mechanical comparison of points is not good enough documentation.Midland, B-298720One-paragraph summaries of the proposalsCharts showing agencys assignment of raw scores, the calculation of weighted scores, & the total point score for each proposalThe record lacks any documentation reflecting a meaningful comparative analysis of proposalsNeed to explain why awardees lower technically rated, lower-priced proposal was selected for award over Midlands higher technically rated, higher-priced proposal. Agency will do the evaluation again

If you would like to get a better understanding of why the Government loses protests, you should take a look at GAO Bid Protest Overview (GAO-12-520SP).Last updated in December 2011, GAO Bid Protest Overview looks at: Evaluation and Source Selection Price and Cost Evaluations Past Performance Federal Supply Schedule (FSS) Purchase Organizational Conflicts of Interest (OCI) Unfair Competitive Advantage Protests, Task and Delivery Orders OMB Circular A-76 Miscellaneous IssuesInsight Into GAO Decisions

Defining the requirement is more difficult in servicesNew SSP says The leadership of Program Management/Requirement Office shall: ensure the technical requirements.are approved and stable, establish technical specifications, and develop a SOW, SOO, or PWS

Easier said than done! These people are usually not experts in what they are buying or trained in acquisition! *

Defining the RequirementWriting is hard! Very few people in most organizations have the skill and interest to write good requirements. Find these people, praise them, and promote them so they can never leave.

*

Defining the RequirementThe sooner the parties come to an understanding of the requirements, the better (FORCES THE GOVT TO PREPARE AND AGREE!)Before AwardDraft proposalsOne on one meetingsSite surveysProvide as much historical data as you haveAfter AwardPost award conference*

Poor Requirement ExampleArmy lost to InfraMap in GAO ProtestEstimates of future work was not reasonableRFP work statement was based on 2010 data plus 10%Ktrs to propose fixed annual priceInformation available indicated a 17 to 19% increase from 2010 data

10% increase was the increase used by the agency the last time they solicited this work *

Manage the Risk Evaluate Risk so you can manage it Require and evaluate Contractors Quality Control Plan. Put it on contractUse incentives to ensure contractor attention on performanceAssign a COR early in the development of the RFPSame COR develops the requirement and monitors the performance

Use IncentivesNeed to select the appropriate contract type and incentivesFY 2012 NDAA: Meaningful incentives to services contractors for high performance at low cost BBP: Use FFP type, CPFF, CPIF for servicesMy opinion: We used T&M and Award Fee so much that our KOs dont know how to use various contract types and incentivesT&M and Award Fee are not in favor*

COR Training and Certification The standard identifies competencies, experience and minimum training needed for successful performance as a COR for:Type A: fixed-price, low performance risk requirementsType B: other than fixed-price, low performance risk requirementsType C: unique requirements that necessitate a professional license, higher education, or specialized training (Matrix at Attachment A)*

Contracting Officers Representative Training

4.5 class days CLC 106 COR with a Mission FocusCLC 206CORs in a Contingency EnvironmentCLC 222Contracting Officers Representative Course32 hrs online3 hrs onlineCOR 222Contracting Officers Representative CourseCOR 206CORs in a Contingency Environment3 Class HoursTraining is determined by the Type of Contract the COR is assigned to monitorCOR 222 and CLC 222 are equivalent coursesCOR 206 and CLC 206 are equivalent courses

COURSES AVAILABLE:KEY:8 hrs onlineONLINE COURSESCLASSROOM COURSES

Help: CORT ToolElectronic nomination and termination process COR has the ability to self nominateElectronic supervisor and contracting officer/specialist approval of COR nomination/termination request Identifies: Name, career field, certification levelCOR supervisor and contracting officer/specialist by name and other contact informationAll training completed by the COR by complexity of the work/requirement (Type A/B/C), including basic and refresher training requirementsTracks:Contracts by COR or CORs by contract Add and review documents, such as a Status Report, COR trip report, correspondences and other miscellaneous documents to one or more of the COR appointed contracts

*

Help: Service Acquisition Mall - SAMIntegrates Sourcing Process and Learning assets with Product Service Code KnowledgeUtilizes same sourcing process contained in SAW and ACQ 265Aligns with DPAP Service Portfolioshttp://sam.dau.mil

Builds your documents Performance Work Statement (PWS) Quality Assurance Surveillance Plan (QASP) Performance Requirements Summary (PRS)Easy to use Step by Step Approach Wizards provide guidance to help build documentsPerformance Based requirements Proven methodology for building better requirements.Help: ARRT Automated Requirements Roadmap Tool*A job aid using standard templates for PWS, QASP and PRS to help you organize and write performance requirements following the Requirements Roadmap process. Over 600 downloads since 15 Aug Runs on Microsoft Office applications Generates Microsoft Word documents for use in your acquisitionARRT Community of Practice on DAUs ACChttp://sam.dau.mil/arrt 1.1 Online today at*

Help: DAU Online Training Resourceswww.dau.mil

CLC007, Contract Source Selection

Acquipedia Articles Source Selection Fair and Reasonable Price Determination

Source Selection Community of Practice (ACC)

*

Source Selection for Services*

This guy is an African Albino Guitarist. He is presenting unique.

WE are going to look at source selection of services contracts. I will be using information fro the March 4, 2011 SSP and some fairly recent GAO casesIssued in March 2011, became effective on July 1, 2011, incorporated into the DFARS in Sept 2011. *This comes out of the new source selection procedures of March 4, 2011. These procedures are designed for uniform source selection guidance within DoD and to simplify the source selection process. It says that appropriate acquisition planning is paramount for a successful source selection. (FAR/DFARS 7)*I mention One on One, because market research is so important in services. As long as you give no preference to a particular supplier in the one on one discussions, they are a great way to find out the weaknesses and strengths of the market place

Industry Day: This is our current environment and requirement, we give a presentation to interested vendors. Due Diligence: The Government releases a draft description of the requirement and opens up the doors to interested parties, helping each side clarify the requirement

Regarding DRFPs: Everyone does this is the Weapon Systems Environment. Still do not see the interest in Services. Maybe it is because so many services are commercial or commercial-like.

Quote from the SSP Exchanging information on upcoming acquisitions improves understanding of Govt requirement and Govt understanding of industry capabilities. *Point out that there are many ways to consider quality and much is in terms of services. The first 4 on the list are very service descriptors *Past performance information get evaluated and collected at the $1M or greater threshold for services and IT. It $5M for systems and operational support, so the need for a lower threshold for services and IT is recognized. But source selection teams should consider the value of evaluating at a lower level. Deviation 99-0002, Jan 29, 1999

I just a class of CON360 students. They told me that they dont bother with CPAR. Its too hard to do. If they dont put it in CPARs, how are we supposed to pull it out. This should be reduced. The PP threshold is too high.

**These are the definitions of past performance found in the new Source Selection Procedures guide. Criteria shall be unique to each source selection and stated in the solicitationConsider aspects of offerors contract history that would give greatest ability to measure whether offeror will satisfy current procurementCommon aspects:Similarity of service/supportComplexityDollar valueContract TypeDegree of subcontract/teaming

New case in Past Performance : This case proposes to implement section 806(c) of the NDAA for FY 2012. The provision (1) provides contractors a maximum of 14 calendar days to add comments/rebuttals to a past performance evaluation after the date the contractor is informed the evaluation is available for comment and (2) requires the past performance evaluation to be made available to source selection officials, etc., 14 days after the date that the contractor is informed it is available for contractor comment, whether or not the contractor has made comments. The due date for the draft proposed rule is being extended to March 28, 2012, because of IT issues regarding compliance with the second part of the statutory mandate. A meeting with Mr. Ginman will take place March 20th, after which the committee should be able to proceed with the draft.

*How well contractor performed on contractsEvaluation does not change existing record/history of offerors past performance on past contractsGathers information from customers on how well offeror performed past contractsRequirements on history of small business utilization outlined in FAR 15.304(c)(3)(ii) and DFARS 215.305(a)(2)

We dont want to train our contractors to do the work. Nor do we have the manpower to monitor them all the time. So start out by getting the best contractor to do the work that your money can buy. As a team, determine the high risk areas of the project. Is it timelines and schedule because the work supports emergency relief, is it the hazardous material that will be brought across country, is it customer service or lack of it. Have the contractors prove with past performance, proposal explanation, internal quality control systems, how they will decrease the risk of performing this work. The Govt must understand what will differentiate the contractors and what work is really significant so they can write the evaluation criteria. *For example: If our PWS says that we need someone to teach students about Services Contracting, then we cant expect that all proposals will offer teachers with brick and mortar buildings. Some contractors will propose FOLE, some Distance Learning, some Resident. Our market research and our clear understanding or our customer requirement will make the source selection team prepared to write evaluation criteria that will fairly evaluate all the proposals.

*Information from Coastal Environments, Inc, File B-401889, December 2009

This is a small business set-aside contract to provide professional marine, terrestial, and historic cultural resource investigation services

Proposal A is ECC. They were awarded the contract by the SSA. Proposal G is Coastal, the protestor. *___________________________________________________________________________________The SSA did not document the analysis of all the proposals. She improperly limited her tradeoff analysis to a comparison of the two highest rated, highest-priced proposals. Army was recommended to perform and document a new technical/price tradeoff analysis. Coastal was reimbursed their costs.

New 2010 case made same mistake, System Engineering Intl Inc. B-402754*Access Systems Inc: Protest is sustained in a competition for the issuance, on a best-value basis, of a task order to a higher-priced vendor under 8(a) Streamlined Technology Acquisition Resources for Services (STARS) government-wide acquisition contract for IT Services, where the record does not show meaningful consideration of price in the agencys selection of the higher priced quotation.

Govt must perform the cost/technical tradeoff again. . * Agency may not exclude from the competitive range a proposal that has not been determined technically unacceptable without taking into account the proposals price. DECISION Arc-Tech, Inc., of Ashburn, Virginia, protests the exclusion of its proposal from the competitive range under request for proposals (RFP) No. 263-2008-P(GG)-0238, issued by the Department of Health and Human Services (HHS) for custodial services at National Institutes of Health buildings located in Bethesda, Rockville, and Poolesville, Maryland. The protester argues that the agencys evaluation of its proposal was unreasonable and that the agency improperly failed to consider price in its competitive range determination. We sustain the protest. The RFP, which was set aside for competition among 8(a) firms,1 contemplated the award of a fixed-price, indefinite-delivery/indefinite-quantity contract for a base and 4 option years. The procurement was conducted using the procedures under Federal Acquisition Regulation (FAR) part 12 for the acquisition of commercial items. * Under solicitation providing for award to offeror whose proposal is found to be the most advantageous to the government based on past performance, delivery, and price, selection of lower technically rated, lower-priced proposal is improper where the record shows that selection decision was based on a mechanical comparison of offerors total point scores and lacks any documentation indicating that a price/technical tradeoff was made. Need words with meaning, not just numbersDECISION Midland Supply, Inc. protests the award of a contract to Danaher Tool Group under solicitation No. 6FLS-G3-050327-N, issued by the General Services Administration for quantities of 50 commercial line items in the 5120 Federal Supply Class (handtools, non-edged, non-powered). Midland challenges the agencys award for line item No. 1 for socket wrench sets (an item Midland has provided to the government since 2001) to Danaher, a firm submitting a lower technically rated, lower-priced proposal for this line item. We sustain the protest.

The propriety of a price/technical tradeoff turns not on the difference in the technical scores or ratings per se, but on whether the agencys judgment concerning the significance of the difference is reasonable and adequately justified in light of the evaluation scheme. Opti-Lite Optical, B-281693, Mar. 22, 1999, 99-1 CPD 61 at 4.

An award decision is not reasonable where there is no documentation or explanation to support the price/technical tradeoff and where the agency makes its award decision based strictly on a mechanical comparison of the offerors total point scores. Universal Bldg. Maint., Inc., B-282456, July 15, 1999, 99-2 CPD 32 at 4; see also FAR 12.602(c), 15.308. **Market research, requirements generation, preparation of the SOO/PWS and QASP, and the source selection is usually accomplished by individuals not trained in Acquisition or source selection, nor are they experts in the requirements that they need. Ever had to develop a PWS with metrics? Evaluation criteria? Its hard

Untrained people are preparing the core documents for source selection.

*Im not kidding. Certainly industry knows this. *Manage the expectations. Talk to each. Before award: As quickly as possible let the contractors see what you are thinking of doing. Develop a web site where you post draft proposals. Let the contractors comment on everything before you put out the proposal. Dont wait to you award to see that there are going to be problems. Call in contractors and talk to them about the requirements as you see them. Ask for their input. Dont issue requirements that dont make sense. Let the contractors tell you what is reasonable, what is going to cost a lot of money. They might tell you if you give them 3 months to complete, it will cost $2M but if you give them 5 months to complete it will cost $1M because they can get material and labor at a more reasonable pace and price. For construction and services, let the contractors see the site, see the GFP and GFE. Let them see as much as possible of the working environment. How are they supposed to proposed the running of an IT help desk if they dont see where they are to be working, understand the equipment, understand how many calls they would expect to get.

After Award: For the smallest projects, the CO needs to meet with the COR/QA and go over the contract. As complexity builds, so should the post award. On your large complex projects, you should plan on 2-3 days of discussion. Program Managers should meet. Every element of the SOW and Specification should be reviewed, parties need to understand payments, schedule and milestones, incentives, Govt surveillance plans, contractor quality control plan

*Protest # B-405167, 6 Feb 2012: Aberdeen Proving Ground involving electronically locating, physically marking, and mapping buried utilities prior to issueing excavation permits. The offerors had to propose a fixed annual price for performing the UUL (underground utilities location)The Armys estimates of future underground utilities location work did not reasonable account for the expected workload. Failing to provide reasonably accurate estimates deprived the protester and other offerors of information required to assess the likely cost of performance.

The 10% was based on copying the last solicitation package. We all could have guessed that. Specifically, the proposal was due on Oct 31, 2011. On Oct 27th, a representative from InfraMap spoke with the agencies chief engineer who said the UUL work would include underground utilities location services for a new electrical privatization effort at Aberdeen. When InfraMap asked the KO for clarification, they go no response. InfraMap immediately protested. **We cant monitor everything. We just dont have enough people to monitor every move of the contractor. So we should ensure that we evaluate a contractor quality control plan. I recommend that it be submitted with their proposal so that it is evaluated and then put on contract. Then after award, the Govt can work/re-work/ and continue to change its own surveillance plan based upon risks We can always use award and incentive fees to ensure contractor attention. Probably more appropriate for complex, large dollar procurements.

We teach COR classes to people who are COR. The first thing we ask is if they have a copy of the contact and know who their CO is. Do you know what they say? Almost always the answer is NO on both questions. I base this opinion on teaching. Our students dont know how to use the other contract types.

FY 2012 NDAA took the recommendation of the Defense Science Board on Improvements in Services Contracting

BBP, Dr. Carter, Undersecretary of Defense for AT&L 3 Nov 10, SUBJECT: Implementation Directive for Better Buying Power - Obtaining Greater Efficiencyand Productivity in Defense SpendingLimit the use of time and materials and award fee contracts for services:I will issue further detailed guidance for establishing taxonomy of preferred contract types in services acquisition, but starting immediately, you will ensure that services acquisitionsunder your control are predisposed toward Cost-Plus-Fixed-Fee (CPFF) or Cost-Plus-IncentiveFee (CPIF) arrangements when robust competition or recent competitive pricing history does notexist. This practice will be used to build sufficient cost knowledge of those services within that market segment. You will employ that cost knowledge to inform the "Should Cost" estimates offuture price and contract type negotiations. When robust competition already exists, or there is recent competitive pricing history, you will ensure that services acquisitions under your controlare predisposed toward Firm-Fixed-Price (FFP) type contract arrangements. FFP should also be used to the maximum extent reasonable when ongoing competition is used in Multiple AwardContract scenarios.

**Properly train your COR before contract award. They frequently write our requirements, write the evaluation criteria, sit on SS Boards, monitor our contracts..for heavens sake train them. All training requirements for the DOD COR Certification is available on line:CLC 106, 8 hrsCLC 222, 32 hrsCLC 206: 3 hrsCLC 003: 1 hr

DOD is doing some things to improve monitoring of contractor performance All services contracts must have COR, prior to award. Recognition that CORs help write the requirementsAll CORs must be trained.New DFARs and PGI specify COR responsibility (201.602-2)Management review process and acquisition planning for services

*Instructor: The details on the slide are not very important. What is important is that contracting people have to control the COR nomination process in a data base called the CORT.

Carter memo released March 21, 2011 announcing this tool. Pilots beginning Oct 2010.From the Carter memo:I am pleased to announce the deployment of the DoD CORT Tool. This capability,initially developed by U.S. Army, Communications and Electronics Command, and subsequentlyadopted by the Army leadership, has been adapted for use by the DoD. The Army will be theexecutive agent to maintain this electronic capability until further notice. This capability willallow components and defense agencies to electronically track Contracting OfficerRepresentative (COR) nominations, appointments, terminations and training certifications acrossDoD.The DoD CORT Tool is a Web-accessible management application designed to: Be Common Access Card-enabled and available to all members of the DoD with internet access and an Army or Defense Knowledge Online account (AKOIDKO). Nominate, appoint, track, and terminate, if necessary, an individual as a COR against a DoD contract or order or actions issued on behalf of a DoD assisting agency. Allow a prospective COR to create a profile, document their training and experience,and process a nomination package for one or multiple contracts and orders. Provide DoD personnel a Web-based portal for all relevant COR documents to include their training and experience,and process a nomination package for one or multiple contracts and orders.*New Tools to help the requirements generation** This is only a sampling of DAU resources that address source selection.

Relate the Conflicts of Interest training module to the importance of impartiality during the source selection process*Questions?