-SAMPLE-

Professional Services for SemioticMonitoring RFP

FAIRNESS MONITOR’S REPORTOF BID EVALUATION

OUTCOMES

Mmmm dd, yyyy

Prepared by:RFP SOLUTIONS - Procurement Strategies for GovernmentOttawa, CanadaTelephone: (613) 728-1335

ABCDepartment ofSemiotics

Page 1

-SAMPLE-Attestation of Fairness Monitor

I, Paul Probity, having served as the Fairness Monitor during the evaluation ofABC’s Professional Services RFP, and having participated in the proposalevaluation process associated with this RFP, do hereby attest that:

1. The evaluation of the proposals presented to the Evaluation Committee wasundertaken in accordance with the pre-established evaluation criteriadescribed in the RFP, and that the pre-established evaluation criteria weredeveloped in advance of the RFP being placed on GETS;

2. Any and all proposals considered by the Evaluation Committee which weredeemed to be non-compliant for failing to meet any one of the establishedmandatory requirements as set out in the RFP, were duly analyzed,documented and verified prior to rendering this decision;

3. Where clarifications/questions were submitted to bidders by ABC during theevaluation process, that these requests for clarification/questions weresubmitted by ABC in writing for dissemination to the appropriate bidders;

4. The proposal evaluation methodology used by the Evaluation Committeewas based on the “rules of evidence approach”, with determinations beingmade on the basis of whether or not evidence was supplied by bidders intheir proposals against each of the previously established proposalevaluation criteria described in the RFP; and

5. The proposal evaluation process was conducted in a consistent, fair, even-handed, objective and transparent manner, in accordance with the criteria setout in the RFP, and that the specific criteria and tests of evidence were

Page 2

consistently and fairly applied to all proposals considered by the EvaluationCommittee.

Page 3

-SAMPLE-In consideration of the above, and in consideration of my efforts as FairnessMonitor to verify the process followed in evaluating responses to this RFP and toseek clarification, as required, for the rationale behind any Evaluation Committeedecision, I am satisfied that the evaluation process associated with this RFP wasconducted in a fair, open, consistent and transparent manner.

Signed at LOCATION, this 25 day of June, 2003:th

Paul Probity, M.B.A.,Assistant DirectorFairness Ltd.Tel:

Page 4

-SAMPLE-

FAIRNESS MONITOR REPORTPART A: OVERVIEW

PurposeThe purpose of this document is to describe the process and outcomes associatedwith the solicitation and evaluation of proposals in relation to ABC’s RFP # 12-34-5678, as well as the work undertaken by the Fairness Monitor to verify thefairness and probity of the process and outcomes.

Mandate of the Fairness MonitorThe services of a Fairness Monitor (Fairness Ltd.) were retained by ABC to act asan independent and impartial third-party observer of the procurement processassociated with this RFP.

The Fairness Monitor (Paul Probity, M.B.A.) was mandated to oversee andobserve the proposal evaluation process and to provide the Department with areport outlining his impartial observations and findings with respect to thecompliance of the process with both the principles and practices of fairness.

In all respects, the Fairness Monitor served as a neutral and objective third-partyduring the process, with no interest (financial or otherwise) in the outcome of theproposal evaluation exercise other than ensuring that an open, fair and transparentprocess was followed.

The Fairness Monitor attended and participated in all of the deliberations and

Page 5

-SAMPLE-discussions held by the Evaluation Committee to determine the compliance ofbidders, and participated in discussions to obtain clarification on any outstandingpoints or issues. All of the documentation submitted by the bidders was madeavailable to the Fairness Monitor.

The Fairness Monitor is the Assistant Director with Fairness Ltd., an Ottawa-based firm specializing in RFP development and proposal evaluation services. Paul Probity is the former Head of Inquiry at Examination Inc. Mr. Probity holds avalid Secret-level government security clearance.

Fairness Ltd. also maintains a strict corporate ethics policy which, among otherthings, requires that Mr. Probity divulge nothing regarding the RFP process tooutside parties, before, during or after the RFP process, without the explicit writtenauthorization of ABC.

Profile of RFP Issued by ABCThe RFP’s (see below) were posted on the Government Electronic TenderingSystem on mmmm dd/yy, with a closing date/time of mmmm dd/yy, 00:00 hrs.,EST (including extensions).

RFP Number RFP Name

12-34-5678 Professional Services for Semiotic Monitoring

Page 6

-SAMPLE-Members of the Evaluation CommitteeThe members of the ABC Evaluation Committee evaluating proposals received inresponse to the RFP were:

1. Adam Administration, ABC2. Patricia Proposal, ABC3. Paul Probity, Fairness Ltd. (Fairness Monitor)4. Carla Contract, ABC (Recording Secretary - Non-voting)

Summary Outcomes of the BidEvaluation ProcessFor all bids deemed to be in compliance with the mandatory requirements, the BidEvaluation Committee completed a detailed evaluation in order to assess theevidence presented in their proposals against the rated criteria stipulated within theRFP. The summary-level results of this analysis are presented in the tables below:

1a. Professional Services:Bidder Mandatory

CriteriaMet (Y/N)

TotalPoint-RatedScore

Ranking # ofQualifiedResources

Connie Contractor Yes 85/100 1 1

Peter Procurement Yes 70/100 2 1

Adele Acquisition No n/a n/a n/a

Page 7

-SAMPLE-Mandatory Criteria Within the RFPBidders’ proposals needed to meet ALL of the following mandatory criteria fortheir submissions to be considered for further evaluation. Failure on the part ofbidders to meet these requirements resulted in their proposals being deemed non-compliant, with the proposal being given no further consideration.

MANDATORY CRITERIA

1. Bidders must include within their bid a detailed curriculum vitae (c.v.) for eachproposed Professional Services resource named in their bid; c.v.’s must includechronological work experience (indicated in years/months) as well as a detailed listingof the educational and professional designation attainments, as well as all otheracademic credentials for each proposed Professional Services resource.

2. It is mandatory that all proposed Professional Services resources must be able todemonstrate within their c.v.’s, a minimum of twelve (12) months previous workexperience during the past ten (10) years (on a cumulative basis) in the provision ofProfessional Services in the specific subject area of Semiotic Monitoring. ProposedProfessional Services resources lacking these respective experience requirements willnot be considered by the Department.

3. Bidders must include within their bid the names of at least three (3) impartial referenceswho have knowledge of the bidder’s experience in the provision of professional services in thespecific subject area of Semiotic Monitoring. At a minimum, the name, address, and telephonenumber for each named reference must be included. Note: It is not necessary to providereferences for each proposed resource.

Page 8

-SAMPLE-Point-Rated Evaluation MethodologyBidders’ proposals were evaluated and point-rated against the following criteria,using the evaluation factors and weighting indicators specified for each criteria. Only those bidders whose aggregate point-rating score on the whole exceeded thepassmark threshold (65/100) were considered as responsive.

Certain of the following point-rated criteria apply to information submitted bybidders at the firm/bidder level (#’s 3, 4, 5 & 6), while other criteria relatespecifically to information provided by bidders concerning each of their proposedresources (#’s 1 & 2). Each proposed Professional Services resource submitted bythe bidder will be evaluated on a category-by-category basis against criteria # 1 &2, and an average point-rating score derived for each category. The average point-rating score for criteria # 1 & 2 (by category) will be added to the bidder’s scoreon criteria # 3, 4, 5 & 6 and totalled, leading to an aggregate point-rating score forthe proposal on the whole.

Page 9

-SAMPLE-POINT-RATED CRITERIA

POINT-RATEDCRITERIA

WEIGHT EVALUATIONFACTORS/METHODOLOGY

1. For each proposed Professional

Services resource, bidders

should indicate their knowledge

of the historical, legal, political

and cross-cultural elements

affecting Semiotic Monitoring

in Canada, as well as their

knowledge of the documentary

and evidentiary requirements of

the Semiotic Monitoring

process, as evidenced through

previous work experience

and/or formal education in this

general subject area.

/20 2 points (or part thereof) for each year of previous

work experience and/or formal education in this

general subject area, as listed in the c.v. of each

proposed Professional Services resource, to a

maximum of 20 points (calculated as an average per

category, based on the rating score of all proposed

resources by category). Evaluations will be

undertaken for each proposed Professional Services

Resource on a category-by-category basis.

Based on evidence contained within proposals, the

Evaluation Committee determined the total

relevant experience and education. Bachelor

degree (any field) - 3 years; Bachelor degree with

Honours (any field) - 4 years; Relevant Masters

degree - 2 years; Relevant PhD degree - 1-3 years

(based on evidence supplied; Experience as per

evidence supplied of relevant experience.

(X months education and experience/12),

multiplied by 2 = points awarded up to a

maximum of 20.

Page 10

2. For each proposed

Professional Services

resource, bidders

should describe the

proposed resource’s experience in the

general area of Professional Services as

evidenced through previous work

experience in this general area.

a) Professional Services:

responsible for the planning,

managing and direction of

research projects and teams

working to develop complete

and factual briefs for semiotic

codification (e.g. Research

Plans, Research Reports,

Statements of Claim/Reference

Analysis, Background

Documents, Briefing Materials);

b) responsible for identifying

primary documents and other

materials of relevance to

Semiotic Monitoring; extracting

relevant data from primary

documents, transcripts and

secondary sources; identifying

distinct issues of relevance to

monitoring in such sources;

compiling annotated

bibliographies; writing document

summaries and file summaries;

producing annotated document

indices; reading and transcribing

hand-written documents;

working with computerized

database information systems

used for monitoring; and

entering data derived from

research into such

/20

2 points (or part thereof)

for each year of previous

work experience in this

area, as listed in the c.v.

of each proposed resource, to a maximum of 20

points (calculated as an average per category, based

on the rating score of all proposed resources by

category). Evaluations will be undertaken for each

proposed Professional Services resource, on a

category-by-category basis.

Based on evidence contained within proposals, the

Evaluation Committee determined the total

relevant experience per resource, per category.

Relevant Professional Services experience (a) was

considered to include only that which was directly

related to work experience Professional Services

activities (i.e. planning, directing, managing) in

the specific area of Semiotic Monitoring (defined

to also include Semiotic Monitoring equivalents

such as codification, transmission and education).

Relevant Professional Services experience (b) was

considered to include only that which was directly

related to work experience Professional Services

activities (i.e. identifying primary documents,

compiling annotated bibliographies, etc.) in the

specific area of Semiotic Monitoring (defined to

also include Semiotic Monitoring equivalents such

as codification, transmission and education).

(X months experience/12) multiplied by 2 = points

awarded up to a maximum of 20.

-SAMPLE-

Page 11

3. Bidders should

indicate,

through a

written

description of no more than

1,000 words, their

understanding of ABC’s

requirement as stated in this

RFP, together with their

proposed approach and

methodology which would be

used to manage any

tasking/call-up assignments

awarded to them.

/10

Up to 2 points for

demonstrating evidence

of having understood

ABC’s requirement

Up to 4 points for describing an approach and

methodology that is consistent with the principles

and practices of sound Semiotic Monitoring

Professional Services, including investigation and

analysis techniques, research planning, the

preparation of clear, concise and objective historical

reports, document/file summaries, report forms,

annotations and database entries; accurate source

attribution; effective document management; and

sound resource budgeting/scheduling

Up to 4 points for describing an approach and

methodology that is consistent with sound project

management, quality control and reporting practices

4. Assessments from named

references.

/5 1/3 point per affirmative response, to a maximum of

five (5), based on five (5) standardized questions

asked to each of the three (3) supplied references.

5. Proposal Quality /5 5 points for presenting proposals in a clear and

logical fashion, and in a manner which facilitates a

clear and straightforward evaluation, based on the

information requested in the RFP, as evidenced by

the following:

2 points for ordering/structuring the proposal to

match the order and sequence of the mandatory and

point-rated factors in the RFP;

1 point for including tabs/separations between

sections of the proposal;

2 points for the overall clarity and ease-of-use as it

relates to the structure, presentation, layout and

design of the proposal.

-SAMPLE-

Page 12

6. Bidders should

indicate the

applicable all-

inclusive per

diem rate ($CAN) for an 8.0

hour day, for each proposed

resource during the term of the

Standing Offer. Quoted per

diem rates which vary over the

period of the SOA must be

specified and will be evaluated

on a weighted average basis.

/40

The average of the

Professional Services per

diem rates quoted will be

used for evaluation

purposes on a category-by-category basis, with full

points going to the bidder with the lowest weighted

average rate per category, and lesser points being

awarded to all other bidders on a pro rated basis

based on the percentage differential of their

weighted average rate per category from that of the

bidder with the lowest weighted average rate per

category. Per diem rate evaluations will not be

undertaken for the Document Management

category.

Average per diem rates per category were

calculated by the Evaluation Committee on a

category-by-category basis, using the quoted

average per diem rates for only those proposed

resources who were deemed to have met

Mandatory Requirement #2 for their category.

Only the average rates of these qualified resources

were used to establish the weighted average per

diem rate per bidder.

TOTAL: /100

-SAMPLE-

Page 13

-SAMPLE-PART B:Profile of Proposals ReceivedThe table below provides a listing of the proposals received by ABC in response tothe RFP:

Name of Bidder MandatoriesMet?

ProfessionalServices

Document Support

Connie Contractor Yes T T

Peter Procurement Yes T T

Adele Acquisition No T

Non-Compliant BidsThe bid received from Adele Acquisition was deemed as non-compliant for failingto meet the requirements of Mandatory Criteria #3. The Adele Acquisition bid didnot contain any references.

Page 14

-SAMPLE-Outcomes of Evaluation ProcessThe Evaluation Committee completed a detailed evaluation of each proposal inorder to assess the evidence presented within the proposals against the mandatoryand point-rated criteria stipulated in the RFP. All proposals which failed to meetone or more of the stated mandatory requirements were set aside by the EvaluationCommittee and given no further consideration (see above).

Page 15

-SAMPLE-Summary of Point Rated Evaluation

Bidder: Connie Contractor

Point-Rated Criteria ProfessionalServices

Rationale

#1 Knowledge (Education &

Experience)

20/20 40 months relevant experience and 80 months

relevant education

#2 Specific Work Experience 8.5/20 40 months relevant experience applicable to both

categories

#3 Understanding of

Requirement

9/10 Understanding too narrowly defined to one

specific type of monitoring activity

#4 References 5/5 n/a

#5 Proposal Quality 2.5/5 Tabs missing; ease-of-use and layout was lacking

#6 Cost 40/40 Professional Services: $400.00

TOTAL: 85/100

Bidder: Peter Procurement

Point-Rated Criteria ProfessionalServices

Rationale

#1 Knowledge (Education &

Experience)

20/20 60 months relevant experience and 65 months

relevant education

#2 Specific Work Experience 8/20 50 months relevant experience

#3 Understanding of

Requirement

8/10 Transmission activities understanding not fully

documented

#4 References 5/5 n/a

#5 Proposal Quality 4/5 Tabs missing

#6 Cost 25/40 Professional Services: $800

TOTAL: 70/100

Page 16

-SAMPLE-- Appendix 1 -

Requests for Clarification

During the evaluation process, the Evaluation Committee sought clarification fromeach of the following bidders with respect to information which could not belocated within the bidders’ proposals. Questions were presented in writing tobidders by ABC, with bidders being given two (2) working days to respond inwriting.

1. Peter Procurement.Mandatory Criteria #3 within the RFP required bidders to include within their bid thenames of at least three (3) impartial references who have knowledge of the bidder’sexperience in the provision of professional services in the specific subject area ofSemiotic Monitoring. At a minimum, the name, address, and telephone number for eachnamed reference must have been included. The RFP also noted that it was not necessaryto provide references for each proposed resource.

Among the references provided by the bidder with respect to their proposed resources, theEvaluation Committee is unable to identify which of these references are in compliancewith the requirements of Mandatory Criteria #3, and therefore which of these referencesshould be contacted by the ABC Evaluation Committee to obtain an assessment of thebidder organization’s experience and capabilities.

From the references provided by the bidder with respect to their proposed resources, canthe bidder specifically identify the three (3) references to be contacted by the ABCEvaluation Committee?