25
Critical Perspectives on Accounting 19 (2008) 1321–1345 The actual evaluation of school PFI bids for value for money in the UK public sector Iqbal Khadaroo School of Management and Economics, Queen’s University Belfast, Northern Ireland BT7 1NN, UK Received 21 September 2006; received in revised form 1 April 2007; accepted 15 May 2007 Abstract Much of the literature on value for money (VFM) evaluation of private finance initiative (PFI) contracts has concentrated on the financial methodology of constructing the public sector comparator (PSC) and its limitations. In contrast, this paper focuses on examining how ‘PFI bids’ are actually evaluated for VFM in the case of three secondary school PFI contracts. It is a unique study which presents the Northern Ireland school PFI decision-making processes, within the broader UK context. It is based on the full business cases of three PFI schools and interviews with key actors involved in the PFI processes. This study shows that the methodology used to evaluate and benchmark the design, facilities management, financial and contractual aspects of PFI bids for VFM is subjective and problematic-changes in the assumptions used may easily shift the balance in favour of conventional procurement and/or competing private sector service providers. It further highlights the importance of risk transfer; how non-financial factors are taken into consideration; and the public sector’s reliance on consulting firms and ‘accounting measures’ in the PFI decision-making processes. © 2007 Elsevier Ltd. All rights reserved. Keywords: Private finance initiative; Public private partnerships; New public management; Value for money; Schools; Northern Ireland; UK 1. Introduction The private finance initiative (PFI), introduced in 1992 by the UK Conserva- tive government, involves a contractual agreement for the provision of infrastructure Tel.: +44 2890971366. E-mail address: [email protected]. 1045-2354/$ – see front matter © 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.cpa.2007.05.001

The actual evaluation of school PFI bids for value for money in the UK public sector

Embed Size (px)

Citation preview

Page 1: The actual evaluation of school PFI bids for value for money in the UK public sector

Critical Perspectives on Accounting 19 (2008) 1321–1345

The actual evaluation of school PFI bids forvalue for money in the UK public sector

Iqbal Khadaroo ∗

School of Management and Economics, Queen’s University Belfast, Northern Ireland BT7 1NN, UK

Received 21 September 2006; received in revised form 1 April 2007; accepted 15 May 2007

Abstract

Much of the literature on value for money (VFM) evaluation of private finance initiative (PFI)contracts has concentrated on the financial methodology of constructing the public sector comparator(PSC) and its limitations. In contrast, this paper focuses on examining how ‘PFI bids’ are actuallyevaluated for VFM in the case of three secondary school PFI contracts. It is a unique study whichpresents the Northern Ireland school PFI decision-making processes, within the broader UK context.It is based on the full business cases of three PFI schools and interviews with key actors involvedin the PFI processes. This study shows that the methodology used to evaluate and benchmark thedesign, facilities management, financial and contractual aspects of PFI bids for VFM is subjectiveand problematic-changes in the assumptions used may easily shift the balance in favour of conventionalprocurement and/or competing private sector service providers. It further highlights the importance ofrisk transfer; how non-financial factors are taken into consideration; and the public sector’s relianceon consulting firms and ‘accounting measures’ in the PFI decision-making processes.© 2007 Elsevier Ltd. All rights reserved.

Keywords: Private finance initiative; Public private partnerships; New public management; Value for money;Schools; Northern Ireland; UK

1. Introduction

The private finance initiative (PFI), introduced in 1992 by the UK Conserva-tive government, involves a contractual agreement for the provision of infrastructure

∗ Tel.: +44 2890971366.E-mail address: [email protected].

1045-2354/$ – see front matter © 2007 Elsevier Ltd. All rights reserved.doi:10.1016/j.cpa.2007.05.001

Page 2: The actual evaluation of school PFI bids for value for money in the UK public sector

1322 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

related public services by a private sector consortium, in return for a stream ofpayments payable by the public sector. In 1997, it was adopted by the successiveLabour government as part of the broader public private partnerships (PPP) reformagenda to modernise public services (Broadbent and Laughlin, 2005). The governmentposits that PFI offers better value for money (VFM) than conventional procurement,achieved through ‘optimal’ risk-sharing between the public sector and the privatesector, and innovative design and delivery of public services by the private sector(HM Treasury, 1997, 2006). VFM, as illustrated later in the paper, is demonstratedthrough the ‘accounting quantification’ and comparison of the costs and benefits ofthe conventional public sector procurement option with the PFI option. Governmentdepartments mostly rely on HM Treasury guidance (for example, HM Treasury, 2003)and consulting firms to legitimise their preferred procurement option and demonstrateVFM.

Since its introduction, the PFI policy has been subject to many theoretical and empiricaldebates. Most studies have explored the PFI decision-making processes and have raisedconcerns over accountability and VFM (see for instance, Broadbent et al., 2003, in press;Broadbent and Laughlin, 1999; Edwards et al., 2004; Edwards and Shaoul, 2003a, 2003b;Heald, 2003; Pollock et al., 2002; Shaoul, 2002, 2003, 2004, 2005). For example, Heald(2003) explored the implications of risk transfer on VFM and the accounting treatment ofPFI contracts on the public sector’s balance sheet. From a distribution of economic wealthperspective, Shaoul (2005) critically examined the financial methodology used to appraisePFI projects in the case of hospitals. Recently, researchers have started to explore post projectevaluation of PFI contracts to assess VFM and accountability in the underlying processes(see for example, Broadbent et al., 2004; Edwards et al., 2004; Ismail and Pendlebury,2006).

To date, much of the literature on VFM evaluation of PFI contracts has concentratedon the public sector comparator (PSC) and its limitations. In contrast, this paper focuseson examining how school ‘PFI bids’ are evaluated for VFM after the initial decision toproceed with PFI has been made. It is a unique study of the Northern Ireland’s experienceof school PFI contracts, within the broader UK context. It is important to understand themethodology and the criteria used to evaluate PFI bids for VFM because it is in theseprocesses that performance expectation regarding the delivery of PFI services are formedand transformed into binding legal agreements. The financial and non-financial criteria usedto benchmark PFI bids for VFM may need to be followed by the respective governmentdepartments to monitor the progress of the PFI contract over its term of 25 years or more(Broadbent et al., 2003).

This paper is organised into six further sections as follows. The next section providesa brief review of prior studies on VFM evaluation of PFI contracts to highlight the gapin the literature that this paper has attempted to address. The third section explains theresearch methodology used for collecting empirical data. The fourth section explains thePFI decision-making processes in the context of secondary schools in Northern Ireland.The fifth section discusses the risk-adjusted PSC used by the public sector to appraise PFIcontracts for VFM. The penultimate section illustrates the methodology and criteria used toevaluate PFI bids for VFM. The final section discusses the empirical findings and concludesthe paper.

Page 3: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1323

2. Literature review: PFI and VFM

The PFI policy has been mimicked across many countries as part of the respectivegovernment’s philosophical commitment to the new public management reform agenda,driven by the institutional economic theory rhetoric of ‘rationalising’ public sector activitiesand improving public services delivery through contracts and competition (Hood, 1995;Osborne and Gaebler, 1993). For example, the UK Treasury’s budget report (HM Treasury,2006) highlights the government’s commitment to modernising the public sector by usingPFI as follows:

With PFI, the public sector defines what is required to meet public needs and ensuresdelivery of the outputs through the contract. Consequently, the private sector canbe harnessed to deliver investment in better quality public services . . .. In assessingwhether PFI is appropriate, the Government’s approach is based on its commitment toefficiency, equity and accountability, and on the Prime Minister’s principles of publicservice reform (HM Treasury, 2006, p. 295).

At the macro level, PFI has been examined in the context of the UK government’smodernisation agenda to improve public services stemming from “its desire to achieveefficiency and build parallels with the private sector” (Broadbent and Laughlin, 2005, p. 83).Consideration of the NPM arguments and philosophical debates for PFI is beyond the scopeof this paper (for detailed discussion of these issues, see Broadbent and Laughlin, 2005;English, 2005; English and Guthrie, 2003; Osborne and Gaebler, 1993). The governmentclaims that PFI offers better VFM than traditional procurement, achieved through leveragingprivate sector expertise and creativity in PFI projects, the transfer of risk to the private sectorand through better scope for innovation by the private sector contractors (HM Treasury,2000, 2003; Public Accounts Treasury Committee, 2000, para 25). The PFI policy hasbeen “operationalised through appropriate technologies at the micro organisational level”to ‘steer’ its course in an attempt to deliver its VFM objectives (Broadbent and Laughlin,2005, p. 83; English and Guthrie, 2003). HM Treasury controls the PFI processes throughits policies and guidance. It states that “value for money will need to be demonstrated bycomparing the private sector PFI bids with a detailed public sector comparator (PSC)” (HMTreasury, 1997, para 3.10).

However, there is widespread agreement in the academic literature that VFM is a difficultconcept primarily because of the difficulties involved in ‘measuring’ outcomes and the factthat its meaning is rarely made explicit in public policies (Heald, 2003). This problem iscompounded in PFI due to the lack of transparency in the PFI processes—the commercialconfidentiality veil puts PFI ‘contracts’ beyond public scrutiny. The extent to which VFM isachieved is tested through private sector capital investment appraisal techniques. It involvesthe construction of a PSC which is a discounted cash flow of the costs to the public sectorof procuring the project traditionally (English and Guthrie, 2003, p. 504). The cost of riskstransferred to the private sector is added to the PSC to adjust for the risks transferred underthe PFI option. The resulting risk-adjusted PSC is used to benchmark ‘PFI bids’ submitted bythe private sector for VFM. This paper briefly revisits some of the problems associated withthe construction of the PSC which have been debated in depth elsewhere (see for example,

Page 4: The actual evaluation of school PFI bids for value for money in the UK public sector

1324 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Froud, 2003; Shaoul, 2005) because the ‘PFI bids’ are eventually evaluated for VFM againstthis benchmark, albeit, the PSC is fraught with many conceptual and practical difficulties.

The conventional procurement and PFI comparison is based on comparing the discountedfuture costs of each approach. Prior studies have criticised the use of investment appraisalto appraise PFI projects primarily because this private sector technique considers financialfactors and does not effectively deal with qualitative and social factors (Broadbent andLaughlin, 2005; Shaoul, 2005). The difficulties involved in forecasting future cash flowsand choosing an appropriate discount rate in the public sector are widely acknowledged(see for example, Froud, 2003; Shaoul, 2005). Sustained critiques of using discount ratesof 6%, stemming from the fact that the cost of public sector borrowing is less than the costof borrowing in the financial markets, may have prompted the Treasury to reduce it to 3.5%(Gaffney et al., 1999, p. 116; HM Treasury, 2003). Higher discount rate makes the PFIoption more attractive because under conventional procurement the costs of the projects arepaid for in the first few years as compared to the PFI alternative where the cost is spreadover the 25-year life of the contract. This drastic reduction in discount rate suggests that theproper discount rate used in the public sector is contestable and many PFI projects, includingthe three PFI case studies examined later in this paper, which have shown marginal VFM,would most likely not have gone forward at the current discount rate of 3.5%.

Similarly, the problems associated with incorporating risks and uncertainty in PFIappraisals to calculate the risk-adjusted PSC have also been the subject of much debate andcontroversy. Froud (2003) posits that there is little consensus on the meaning of risk anduncertainty although scholars recognise that these two concepts are different. Froud (2003,p. 570) explains that the term ‘risk’ is used when probabilities can be estimated and ‘uncer-tainty’ is used when they cannot be, that is, risk refers to the likelihood of something goingwrong, and uncertainty means that the outcome of a course of action is indeterminate or sub-ject to doubt. Froud (2003, p. 586) concludes that by locking itself into PFI contracts, the statehas reduced its own powers to act because to effectively deal with “uncertainties requires aresponsible state which has the rights to initiate changes in the use of assets, and whose auton-omy is not limited to the robustness of contracts enforced within private markets”. Broadbentet al. (2008) further argue that because of the pervading power of ‘accounting logic’ aimedat finding ways to capture everything of importance through measurement technologiesand marginalising those which cannot, quantitative risk estimation is given a privilegedposition in PFI decision-making processes thereby ‘silencing’ consideration of qualitativeuncertainties (Aerts, 1994). Indeed, as the empirical case studies later illustrate, ‘accountingmeasures’ have been used to score the qualitative factors in the cost–benefit analysis of thePFI alternative to make decisions. Despite these difficulties, many PFI contracts have beenfound to rely on risk transfer to ‘beat’ the PSC and ‘demonstrate’ VFM (Froud, 2003, p. 576;Shaoul, 2005, p. 457). Moreover, in the case of the UK National Health Service PFI contracts,Broadbent et al. (2008) found that the PSC and PFI comparison was the ‘dominant element’in the decision-making process and only some limited non-financial analysis was conducted.

Heald (2003) argues that there might be a conflict between attempting to achieve ‘max-imum’ risk-transfer to the private sector to keep PFI assets off the public sector’s balancesheet and ‘optimal’ risk-transfer to achieve VFM—this is because the government is in abetter position to assume certain risks than the private sector. Consideration of all risks anduncertainties involved in PFI contracts rather than focusing on risks actually transferred to

Page 5: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1325

the private sector was argued to be important in the VFM evaluation process. Moreover,Heald states that if government departments know that public funds are not available andPFI is the only option that would be funded, then the VFM evaluation process becomesless than real as they might be motivated to engage into ‘gaming’. Similarly, a survey ofPFI school headteachers by Ismail and Pendlebury (2006, p. 400) provides some supportto this argument that if “PFI is generally accepted to be ‘the only show in town’ thenthere is not much point in concluding that the PSC alternative gives better VFM if thismeans that it will not go ahead” and “under such circumstances there is a clear incentivefor all those involved in the evaluation to ensure that the PFI alternative is the definitewinner”.

While the preceding studies have pointed out the difficulties and subjectivities involvedin preparing the PSC and assessing risk transfer and VFM, no study has examined howthe PFI alternatives proposed by the private sector bidders are actually evaluated for VFM.This study attempts to fill this gap in the literature by examining how school PFI bids areevaluated for VFM and how school PFI decisions are made in Northern Ireland. In thisrespect, Broadbent and Laughlin (1999) stated that there is a need for research to explorehow PFI decisions are made in different areas of the public sector to better understand themerits and worth of the PFI policy. The next section discusses the research methods usedto achieve the objectives of this research.

3. Research methods

This paper draws from the full business cases of the three PFI schools and interviewsconducted with 42 key stakeholders involved in the PFI decision-making processes.1 Themotivation for collecting qualitative data by conducting semi-structured interviews is basedon the premise that when the objective of a research is to understand a complex process, itis more appropriate to adopt a research strategy which allows this process to be understoodin the same terms in which the participants understood them (Blumer, 1969; Connollyand Hyndman, 2006; Garfinkel, 1967). Similarly, Humphrey and Scapens (1996, p. 96)state that one of the merits of conducting case study research is that it “can provide richerinterpretations of the research issues through the particular synthesis which the researcherdevelops to encompass the cases under consideration”. Broadbent and Laughlin (1999, pp.110–111) argue that when examining “how are PFI decisions made in different areas of thepublic sector . . . it is important to include an analysis of not only the public sector partners butalso the private sector”. Moreover, in the case of school PFI contracts, Ismail and Pendlebury

1 Upon request, the author would provide the list of questions asked to guide the semi-structured interviews.The questions asked to guide the semi-structured interviews were tailored to take account of differences in termsof the interviewees’ position (e.g. teachers, project manager or consultant) and the different types of organisationthey represented (e.g. school, local education authority or public sector). It is interesting to also note that all threeschools’ headteachers (and some of the project managers) who were involved in the evaluation of the design andthe facilities management aspects of the bids, had retired or moved elsewhere at the time this study was carried out.The long-term nature of PFI implies that the stakeholders involved in the post implementation PFI processes maynot necessarily be those who were actually involved at the procurement stage, although they may well understandhow PFI decisions are made.

Page 6: The actual evaluation of school PFI bids for value for money in the UK public sector

1326 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

(2006, p. 403) posit that many of the PFI “themes would require examination through themore detailed research methodology of case studies or possibly in-depth interviews”.

At the school level, headteachers, deputy headteachers, teachers and private sector facili-ties managers and contractors were interviewed because of their involvement in the facilitiesmanagement processes. Some of these participants were also ‘consulted’ during the PFIprocurement processes. At the local education authority level, the head of property ser-vices, project managers, contracts managers, and chief finance officers responsible for theprocurement and post-implementation monitoring of PFI were interviewed. Private sectorconsultants were also interviewed because they advised the local education authorities onthe bid evaluation methodology, including the preparation of the PSC. At the central gov-ernment level, representatives from the PPP (or PFI) units in the Department of EducationNorthern Ireland (DENI), the Department of Finance and Personnel Northern Ireland andthe Strategic Investment Board were interviewed primarily because of their responsibilityfor implementing HM Treasury policies and developing policies specific to the NorthernIreland context. The auditor general and the PFI audit manager from the Northern IrelandAudit Office who prepared an audit report examining the quality of the buildings of the PFIschools and services delivered were also interviewed (see NIAO, 2004).

Data from secondary sources came mostly from the full business cases and from pressreleases published on the respective government departments’ websites. A request for thefull business cases of the three secondary schools was made to the respective local educationauthority under the Freedom of Information Act 2000. All three local education authoritiesadvised that they would need to consult with the private sector contractors and obtain legaladvice, and as a result would require an extension beyond the 20 days limit provided by theAct to respond to the information request on the grounds that the information contained inthe full business cases was confidential. The full business cases were eventually releasedbut the amount of information disclosed varied across the three schools. The PFI bidsevaluation section in the final business case for school 2 was completely redacted by thelocal education authority which quoted the clauses of Section 41 (covering informationprovided in confidence) and Section 43 (covering information likely to prejudice a party’scommercial interest) exempting ‘commercially sensitive information’ from being disclosed.However, no information was redacted from the full business cases of school 1 and school3. The later section of this paper which illustrates the PFI bid evaluation methodology istherefore based on the information provided on these two schools.

All interviewees allowed for the interviews to be tape-recorded. The tapes were tran-scribed and coded in a qualitative software package (NVIVO). The software was mainly usedto facilitate content analysis and searches of ‘key terms’ across all the interview transcripts.

The next section explains how school PFI decisions are made in the education sector inNorthern Ireland. The results in relation to the main research question of how school PFIbids are evaluated for VFM are subsequently presented.

4. The school PFI decision-making processes in Northern Ireland

Northern Ireland has a population of about 1.71m and forms part of the UK. Underdevolution, the Northern Ireland Assembly (and the UK Parliament in its absence) has the

Page 7: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1327

right, power and responsibility for the administration of the public sector. The sources ofPFI policies are from HM Treasury but are tailored and administered by the Departmentof Finance and Personnel Northern Ireland, to suit the local context. PFI policies specificto the education sector are developed by the DENI and approved by the Department ofFinance and Personnel Northern Ireland. The Northern Ireland Audit Office reports to theNorthern Ireland Assembly on the proper use of public funds through its VFM audits andreports. Public Accounts Committee hearings on these reports are held at the NorthernIreland Assembly and, when suspended, at the UK Parliament.

Education is administered locally by five Education and Library Boards and the Councilfor Catholic Maintained Schools2—one of the three schools examined (labelled as school1, 2 and 3 to maintain anonymity) is managed by CCMS whereas the remaining two schoolsare managed by the Education and Library Boards. The local education authorities have theprimary responsibility to prepare proposals (or economic appraisals), detailing the planningand development work required in their respective geographical area, and prioritise andsubmit them to the DENI to be included on a ‘Schools Capital Priorities Planning List’,and eventually on a ‘Contenders List’ (DENI, 2002). The case for a PFI project is madeat the local education authority level as opposed to the agency (or DENI) level, meaningthat project appraisals are carried out at a narrow local education authority level and not aspart of a wider government policy level or geographical level where ‘catchment areas’ mayoverlap (see similar observation by Shaoul (2005) on the lack of central planning in theNHS and the NIAO (2004) report on the lack of central planning in the Northern Irelandeducation estate). The head of PPP unit from the DENI highlighted that his team has ‘justprepared’ a database of schools in response to recommendations made by the NorthernIreland Audit Office. However, the database still lacked information on the condition of theschools:

We have got an estate’s database, which we have just completed, but it does notstretch as far as assets management planning . . .. And that would be part of therecommendations in the NIAO report . . . (Head of PPP unit, DENI).

Similarly, an advisor from the Strategic Investment Board highlighted the need for betterasset management at the central government level in order to get the schools ‘back in shape’as follows:

There is no proper asset management planning in the Department of Education. Ayear ago, they did not know how many schools they have and the conditions of thoseschools, in terms of minor and major works required. You need to know the state ofyour own assets for proper asset management. It is a bit like knowing your own bankaccount (Strategic advisor, Strategic Investment Board).

An economic appraisal would include matters regarding the age and conditions of exist-ing buildings, the lack of sufficient classrooms to meet the requirements of the curriculum,the inadequacy of facilities for the pupils and staff, among others. It identifies the preferred

2 Under the review of public administration in Northern Ireland (DENI, 2005), the education minister hasannounced its plan to modernise education administration by creating a new education authority to replace the fiveeducation and library boards. The new education arrangements are expected to come into force by April 2008.

Page 8: The actual evaluation of school PFI bids for value for money in the UK public sector

1328 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

option, for example, new build or refurbishment, but does not identify alternative procure-ment routes. The various economic appraisals submitted by the local education authoritieswould be used by the DENI to prepare a ‘Schools Capital Priorities Planning List’ whichlists schemes requiring major works under five categories of prioritisation—category 1 rep-resenting the highest prioritisation (for example, insufficient places to accommodate pupils),and category 5 representing the lowest prioritisation (for example, reliance on temporaryaccommodation). Categories 4 and 5 are usually not considered for funding by the DENIbecause of severe backlogs in capital work and financial constraints (DENI, 2002a). Theeconomic appraisals in categories 1, 2 and 3 are then placed in a ‘Contenders List’ to com-pete for funding. The next stage is to prioritise the schools on the ‘Contenders List’ in termsof those which have the most pressing educational need in the light of the funds available.

A yearly announcement of the successful PFI and conventional procurement schemesis then made by the minister of education (DENI, 2004). After a project is approved forprocurement by the DENI, the local education authority forms a project board and a projectteam3 to take the PFI project forward. Consultants are commissioned by the project team toprepare the outline business case. The outline business case typically includes a needs andobjectives analysis; a review of the procurement options; a draft PSC developed from theeconomic appraisal submitted to the DENI; a risks allocation matrix; an outline user outputspecification developed in consultation with the school representatives; a legal viabilityreport stating that there are no issues which would fundamentally impact the PFI project.The outline business case would also need the DENI’s approval prior to the project beingadvertised in the Official Journal of European Community.

As negotiations progress, the PSC is revised in the light of the solutions proposed bythe private sector. The revised PSC, including details of the procurement and bids evalua-tion processes would form part of the financial business case and is used as the basis forbenchmarking PFI bids received from the private sector. The final PSC may substantiallydiffer from the original economic appraisal and outline business case which won approvalin principle—for example, in the case of school 2 and school 3, the preferred option wasrefurbishment but the private sector offered to provide a new build at refurbishment prices.The audit manager from the Northern Ireland Audit Office (see also, NIAO, 2004) high-lighted that the fact that the private sector contractors have provided the local educationauthorities new build at refurbishment prices may have affected the quality of the buildingsdelivered:

. . . how is the private sector going to provide the public sector with what it determinesto be a more expensive solution and at a lower price. The only way that this can happenis by reducing quality and that is why we made a comment in the Northern IrelandAudit Office report that because of that quality suffered (Audit manager, NorthernIreland Audit Office).

3 The composition of project team varies across projects. A typical project team for the school PFI contractsis made up of a project manager from the Library Board (or Council for Catholic Maintained Schools); LibraryBoard’s solicitor, architect, finance and service managers; trade union representatives. The project team wouldconsult with a user assurance group comprising of headteachers, teachers and pupils and report to a project boardcomprising senior decision makers including, chief executive from the local education authority, project managerand representatives from the Department of Education and the school.

Page 9: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1329

The government seems to have created an in built bias in favour of PFI, raising doubtson the extent to which the local education authorities can objectively prepare the PSCto benchmark PFI bids and ‘demonstrate’ VFM (see also, Heald, 2003 and Ismail andPendlebury, 2006). The following comments made by the interviewees typically reflectedthis concern:

We were told, PFI or nothing. So, we were not going to turn down a new school,no matter what way we are getting it. Waiting was not an option for us, because ourbuilding was so bad. I cannot describe to you, how appalling the building was (Deputyheadteacher, school 1).

I think there is no problem with DENI’s commitment to getting work done, whetherit be conventional or PFI, but there the general interest is being shifted to PFI, becausethere are more resources coming down that way (Head of PPP unit, DENI).

The board of governors can decide that they don’t want to go through PFI, in whichcase, they would be taken out of the PFI list and would go down to the bottom ofthe pile again for consideration for a new school which could be another 10 years.So, effectively they have no real choice and won’t say we don’t want PFI (ProjectManager 4, local education authority’s PPP unit).

Although school teachers, headteachers and pupils are ‘consulted’ during the later stagesof the PFI procurement processes, they do not seem to have a say in the choice of theprocurement alternatives. Moreover, teachers (mostly in school 2 and to some extent inschool 3) highlighted that their views were dismissed during the consultation process:

. . . our opinion was asked but what we had to say was totally disregarded. We didnot have any power or authority whatsoever . . .. Those [enough cloakroom spaceand corridor width] were the two main concerns we had here and we relayed thoseconcerns, from the plans we saw, to the PFI architects. But their attitudes were a verydismissive one (Teacher 1, school 2).

In contrast, the headteachers and teachers in school 1 stated they were actively involvedin these consultation processes and their views were listened to:

We were involved in the process of planning, and given opportunities to identify wherewe wanted specific resources, such as benches, cabinets placed and the setups in theclassrooms. We all had an input into these processes and were well pleased (Teacher4, school 1).

The main document from the consultation process is a Statement of User and ServiceRequirements which is ultimately used to evaluate the PFI buildings and services proposalsfor VFM. We now turn to the financial PSC criteria used to benchmark PFI.

5. Appraisal of PFI projects for VFM: the PSC and risk transfer

Table 1 shows the calculation of the ‘risk-adjusted PSC’ for the three PFI schools. In thecase of school 2, although the PFI option was more costly (£9.711m) than the conventional

Page 10: The actual evaluation of school PFI bids for value for money in the UK public sector

1330 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 1Calculation of the risk-adjusted PSC

Net Present Cost (discounted at 6%) School 1 £000 School 2 £000 School 3£000

Capital costs 12,008 6,025 15,993Operating costs 5,543 2,692 3,943

Total costs before adjustment for risktransfers

17,551 8,717 19,936

Risk transfers (e.g. construction, timeoverrun, site, obsolescence,planning, maintenance, insurance,operating and maintenance,regulatory and force majeure risks)

1,420 877 1,526

Total risk-adjusted PSC 18,971 9,594 21,462

PFI price 17,193 9,711 21,500

Difference = financial VFMbenefits/(disadvantage)

1,778 (117) (38)

Source: PricewaterhouseCoopers and Chesterton (1998), PricewaterhouseCoopers (1999a, vols. 1–3),PricewaterhouseCoopers (1999b, vols. 1 and 2).

procurement alternative (£9.594m), PFI was chosen because the disadvantage was consid-ered to be marginal and the project was a ‘pathfinder’4. According to the final business case,the initial bid submitted by the preferred bidder was £10.169m, and was not considered ‘avalue for money solution’ because it was 6% in excess of the PSC. The private sector wastherefore advised to ‘work down’ its bid (to £9.711m). In the case of school 3, an original bidof £21.6m was submitted by the consortium but it was eventually worked down to £21.5mat the ‘best and final offer stage’ to better stack up with the PSC.

The Net Present Cost figure was calculated by discounting the capital and operatingcosts at the rate of 6% (HM Treasury, 1997). Had the current discount rate of 3.5% beenused (HM Treasury, 2003), the conventional procurement option would have become moreattractive than PFI.

The risks associated with planning, design, construction, finance, maintenance and oper-ation of the project over its life are identified, valued and allocated to the contracting partieswhen calculating the PSC (see Table 1). However, the methodology for identifying, valuingand allocating risks is problematic. It involves making assumptions regarding uncertainfuture events and changes in these assumptions can easily shift the balance between con-ventional and PFI procurement. In the case of hospitals, some researchers have found that“risk transfer almost equals the amount required to bridge the gap between the PSC andthe PFI option” (Pollock et al., 2002, p. 1208). Shaoul (2005) further found that even afterincluding risks, the differences between PFI and conventional procurement are marginaland do not provide a robust criterion for making such important decisions.

4 The three school PFI contracts examined in this paper are termed ‘pathfinders’ because they were the first PFIschools built in Northern Ireland. According to the Northern Ireland Audit Office (2004) report, the ‘pathfinder’PFI projects were undertaken in the context of a major capital and maintenance backlog in the education estateand limited budgets.

Page 11: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1331

In the case of the three PFI schools examined, the consultants have used ‘point estimates’to calculate an expected value for the various types of risks. More specifically, a monetaryvalue and a probability of occurrence were assigned to the different categories of risks.These risks were (subjectively) allocated to specific years in which they were expected tocrystallise (for example, year 10 or year between years 10 and 25). The infinite numberof possible occurrences was arbitrarily restricted to high, medium and low. The monetaryvalue of the risks was multiplied with the probability of occurrence and discounted toarrive at a discounted expected value figure which was subsequently added to the PSC. AsBroadbent et al. (2008) point out, whilst the nature of the risks transferred to the privatesector are given numerical estimates, those risks and uncertainties that are shared or remainwith the public sector are not. These risks and uncertainties are often ignored or ‘burieddeep’ in the ‘commercially confident’ PFI contracts which are not available for publicscrutiny.

Furthermore, many interviewees perceived that the risks actually transferred under thePFI contract were difficult to assess and enforce because the PFI contract is between theprivate sector and the local education authority, not the school which had limited inputin the PFI process. Yet, both contribute to the unitary payments—the capital element isborne by the local education authority and the service element is borne by the school. Animportant VFM implication of PFI contracting is that if the private sector finds it financiallyadvantageous to take a penalty rather than meet the standards as laid down in the contract, theschool and pupils may suffer. Although headteachers monitor their respective PFI school, itis the local education authority that needs to enforce the contract. Nevertheless, the schoolshave no recourse to the local education authority and the private sector in cases of default.

One important consequence of PFI, which is largely ignored in the appraisal process, isthat it may lead to inequity in the education estate. For example, in times of budget cuts,the reduction in funding is borne by conventional schools only. Moreover, because of theinflexibility associated with PFI contracts, in times of falling enrolment, the local educationauthority may be forced to close down conventional schools and transfer pupils to PFIschools to increase their ‘catchment areas’ and sustain them. These concerns were typicallyexpressed as follows:

I am a little bit concerned about the PFI process in that it mortgages our future . . .. Forinstance, in this current financial year, on the 1st of April, we have just got a budgetfrom the DENI and we have had to cut out our budget by £3.6m out of about £250m.Now, the biggest area where we are going to cut is on maintenance and buildings andwe will spread that over the whole estate, except for one building, School 2 (Head ofcorporate services, school 2, reference to school name has been removed).

We still have to pay the unitary payment no matter what, and that is one of thebiggest disadvantages of PFI compared to an ordinary school . . . this school mustbe kept afloat and the bills must be paid, no matter how many pupils we have . . .. Itwould be in their [the local education authority and the Department of Education]best interest that our school is kept full of pupils (Headteacher, school 1).

The UK government has used VFM as the key rationale for using PFI to modernise thepublic sector. However, VFM is a problematic concept which is hard to operationalise inPFI contracts and by using the PSC to ‘demonstrate’ VFM, the UK government has opened

Page 12: The actual evaluation of school PFI bids for value for money in the UK public sector

1332 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

itself to numerous criticisms. The PSC is subjective and sensitive to changes in cash flows,discount rates and risks assumptions—the balance between private or public procurementmay shift easily if these assumptions were to change. Furthermore, as illustrated in thenext section, it is quite difficult to take account of non-financial factors in the PFI appraisalprocess to justify its VFM benefits vis a vis conventional procurement. It is equally difficultto assess the reputation and long-term survival of the PFI contractors and the wider socialand environmental consequences of PFI.

The next section examines the methodology used to evaluate the PFI bids for VFM inorder to justify the selection of the preferred bidder and the PFI option.

6. Bids evaluation and selection of the preferred bidder

As previously mentioned, the local education authority for school 2 was of the view thatthe methodology for evaluating bids and the selection of preferred bidder was commerciallysensitive and was therefore redacted from the full business case. This section thereforediscusses the broadly similar bids evaluation methodology used in the case of school 1 andschool 3 by drawing from the schools’ full business cases and interviews. More specifically,this section highlights the financial and non-financial criteria used to evaluate the PFI bidsfor VFM and how these criteria have been ‘measured’ to select the preferred bidder.

The PFI proposals received for both school 1 and school 3 were evaluated on the basisof the merits of four primary criteria—quality of design of the buildings and technicalproposals; facilities management services; contractual arrangements; and financial viability.Four bid evaluation teams were convened by the local education authority to evaluate thebids received from the short listed bidders. Their findings were coordinated by a projectmanager responsible for the procurement of the PFI contract and reviewed by a Bid (or User)Assurance Team before being finally approved by a project board (PricewaterhouseCoopers,1999a, 1999b). The project managers from the local education authority explained thesearrangements as follows:

I would work with the schools to assist them in preparing the output specificationand tell them, basically, what they are going to get. The project team would have theresponsibility for drawing the documentation for the output specification telling thebidders what we want . . . and then to evaluate the bids when they come in (Projectmanager 1, local education authority).

The financial panel in particular will be looking at VFM by comparing PFI withthe PSC. I will be on the design panel to assess the design standards and qualityof the schools. There will be a facilities management panel that will be looking atthe operational aspect of the building. There will be a financial panel and a legalevaluation panel (Project manager 4, local education authority).

Whereas in the case of school 1 an equal weight of 25% was assigned to evaluate each ofthese four aspects of the bids, in the case of school 3 the weightings varied. No justificationwas provided for the differences in the weightings used, albeit, the final business caseswere prepared by the same consultant. Table 2 shows the primary evaluation criteria andweighting used in the case of school 1 and school 3. School 1 had placed equal importance on

Page 13: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1333

Table 2Primary evaluation criteria and weighting

Primary evaluation criteria Weighting school 1 (%) Weighting school 3 (%)

Quality of design and construction 25 35Quality of services provision 25 25Financial viability 25 20Contractual arrangements 25 20

Source: PricewaterhouseCoopers (1999a, vols. 1–3), PricewaterhouseCoopers (1999b, vols. 1 and 2).

the building design, facilities management services, financial and contractual considerationswhereas school 3 has placed more importance on the quality of design.

Bids which scored highest in these four primary evaluation criteria, with due considera-tion to their costs, were considered to offer better VFM (PricewaterhouseCoopers, 1999b,para 7.2)—a cost per unit of benefit was computed for the purpose of ranking the bidsreceived from the private sector.

Each of the four primary criteria was further divided into various sub-criteria which wereweighted and scored out of a maximum of 25 points. Table 3 shows the scoring bands usedto score the sub-criteria and their associated description.

6.1. Evaluation of building design and construction

The design and construction proposals presented by the bidders were evaluated on thebasis of the extent to which they met or exceeded the requirements specified in the Statementof User and Service Requirements issued at the invitation to negotiate stage. The sub-criteria for design and construction were scored and further weighted based on their relativeimportance. One of the consultants explained that all design variations to the Statement ofUser and Service Requirements proposed by the private sector needed to be ‘quantified’ fordecision-making purposes and approval by the local education authorities:

If there are material differences in terms of design, we would ask them to quantifythese to give us an estimate of the value of these differences, because we need toreduce everything to financial terms (Consultant, school 2).

Tables 4 and 5 show the specific sub-criteria considered for the purpose of evaluating thequality of design and construction and the weights applied in the case of school 1 and school3, respectively. Interestingly, both the sub-criteria and the weightings applied to evaluatethe quality of design and construction proposals were different for the two schools. The

Table 3Scoring bands for best and final offer evaluation

Score Description of score band

0–5 Poor—proposals do not meet project requirements6–10 Barely adequate—some serious shortcomings in meeting requirements

11–20 Generally satisfactory—project requirements are met21–25 Good—project requirements are exceeded

Source: PricewaterhouseCoopers (1999b, vols. 1–3).

Page 14: The actual evaluation of school PFI bids for value for money in the UK public sector

1334 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 4School 1—sub-criteria used to evaluate the quality of design and construction proposals

Sub-criteria Weight1 (%)

Quality of the overall design proposals in meeting the Trustees’ user requirements 15Quality of the overall design response to the Trustees’ service requirements 10Design impact, in terms of prestige, scale and image 10Operational control, in consideration of adjacencies and efficiency 10Totality of accommodation provision, in terms of Trustees’ vision, education,

spirituality and community20

Potential for flexibility of use in response to curriculum requirements 7Degree to which the design solution complies with the DENI environment and

energy policies3

Siting of the building(s) in relation to other buildings, roads, noise sources, etc. 8Appropriateness and quality of the finishes to each area within the buildings 7Appropriateness and quality of the fixtures, fittings and specialist furnishings 7Likelihood of obtaining planning approval and any other necessary consents 3Extent to which the construction method statement and programme meets the

Trustees’ requirements5

Feasibility of bidders’ proposals in terms of disruption 3

Source: PricewaterhouseCoopers (1999a, vols. 1–3).1 The sum of the weights in the financial business case, as prepared by PricewaterhouseCoopers, adds up to

108% instead of 100%. No explanation was provided for this discrepancy.

criteria used to evaluate the building design for school 1 seem to be more detailed thanschool 3. Note that the latter had placed more weights to the evaluation of the architecturalproposal (35%) as opposed to school 1 which considers ‘design quality’ mostly in terms ofconformance to user/Trustee requirements (15, 10, 20 and 5%).

The headteachers and the teachers interviewed seem to have mixed views on the designof their respective school buildings actually delivered—in general, while they were verysatisfied with the ‘external’ design of the schools, they had mixed views regarding theinternal design of the school. Some of the positive sentiments relating to design issues weretypically expressed as follows:

I am not an expert on design but we have been over 4 years now and it is a goodquality build and there has been no problem with it. I certainly am more happy withthe layout, the design and the functionality of it and basically it seems to do the jobit was built for (Teacher 2, school 3).

Table 5School 3—sub-criteria used to evaluate the quality of design and construction proposals

Sub-criteria Weight (%)

Quality of the architectural proposals 35Quality of the structural engineering proposals 10Quality of the mechanical and electrical proposals 15Quality of construction proposals 20Quality of decant and transfer proposals 15Quality of insurance proposals 5

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

Page 15: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1335

As compared to school 2 where teachers felt that they did not have adequate input in thePFI procurement processes, teachers in school 1 were generally very satisfied with the PFIbuilding (and services) primarily because they perceived that their views were listened to:

We were not expert in the design and decor but in the end, we had a good say in termsof the purchasing of quite a lot of things which were put inside the school. We hadinput and we could decide what we wanted in terms of resources (Teacher 3, school1).

The dissatisfaction (mostly in school 2 and, to some extent, in school 3) stemmed fromthe lapses in the quality of finishes, acoustics, ventilation, and corridor width. These negativesentiments were exemplified as follows:

[In the case of school 2] one of the main weaknesses is not providing good qualityfinishes and the acoustics of the classrooms are not great. Externally, the buildingis fine; we would not wish anything different (Project manager 4, local educationauthority).

. . . experts have found that the natural ventilations in this building are not workingvery well. In this weather, some of the classrooms become very hot and don’t ventilatewell enough. But the building design was approved by the education and library boardand the DENI . . . in terms of official specification (Headteacher, school 3).

Many interviewees pointed out that the PFI procurement processes and the design spec-ifications were heavily controlled by the local education authority and the DENI ‘standardbuilding handbook’ which dictated the size and content of a typical school with certain pupilnumbers. In particular, the standard building handbook was perceived to be out of date andlacked certain essential requirements, such as provision for pastoral care and restrictivecorridor width:

. . . there are issues about carbon dioxide levels in classrooms, inadequate day lighting,and corridors’ width. If you want to blame anything, don’t blame these providers,blame two things. One, the building and education handbook, which is wonderfullyout of date. Second, blame the model, through which PFI was run for these pathfinders.They were very heavily input driven and defined by the DENI and the education andlibrary board (Headteacher, school 3).

. . . nowhere in the handbook provision for pastoral care is mentioned. This is prettyironic in a sense because you need a very strong pastoral system in a school to run itproperly to support the children in the school (Teacher 3, school 1).

Although, as explained by the interviewees, there were certain problems with the designof the schools (mostly in school 2, to some extent in school 3—see NIAO (2004) for moredetail), it is interesting to note that all the schools had undergone similar evaluation pro-cesses, met the standards laid down in the DENI’s ‘building handbook’, and were approvedby the local education authorities and DENI. Headteachers and teachers in school 1 whofelt that they had a greater input in the procurement processes were more satisfied with thebuildings (and the facilities management services) actually delivered.

Page 16: The actual evaluation of school PFI bids for value for money in the UK public sector

1336 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 6School 1—sub-criteria used to evaluate the quality of facilities management services proposals

Sub-criteria Weight (%)

Quality of the overall service proposals in terms of Trustees’ educational, spiritualand community requirements

17

Overall approach to flexibility and the degree of innovation offered in the serviceproposals

10

Quality of the proposed management of the bidders’ operational regime 10Extent of impact of non-Trustees usage on the operation of the school 8Quality of proposed service methodologies 10Extent of service delivery (scope of service) 15Quality of bidders’ proposals for establishing monitoring and liaison procedures 10Quality of service assurance through performance measurement, etc. 5Extent to which the bidder is prepared to be responsive and adaptable to varying

Trustees requirements10

Level of acceptance of performance standards and proposals for the agreement ofservice provision

5

Source: PricewaterhouseCoopers (1999a, vols. 1–3).

6.2. Evaluation of facilities management services

The bidders’ proposals for services provision were also evaluated on the basis of theextent to which they met or exceeded the requirements specified in the Statement ofUser and Service Requirements issued at the invitation to negotiate stage. Tables 6 and 7show the specific sub-criteria and the corresponding weightings applied when evaluatingthe quality of facilities management services for schools 1 and 3, respectively. Again,the sub-criteria and weightings used to assess the quality of facilities management ser-vices proposals differed in the case of both schools—no rationale was provided for thesedifferences.

It can be observed that school 1 has made a greater effort than school 3 to frame itsevaluation criteria in terms of its desired educational outcomes by trying to be more specificabout the meaning of these criteria. In contrast, school 3 used the word ‘quality’ many timeswithout indicating what that word actually meant (see Tables 4–7).

Table 7School 3—sub-criteria used to evaluate the quality of facilities management services proposals

Sub-criteria Weight (%)

Quality of catering proposals 15Quality of cleaning proposals 10Quality of building maintenance proposals 10Quality of grounds maintenance proposals 5Quality of caretaking proposals 10Quality of furniture and equipment proposals 15Quality of information technology proposals 15Quality of sporting proposals 15Quality of other service proposals 5

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

Page 17: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1337

At the post-implementation stage, considerable attention is placed on the monitoring offacilities provided by the PFI contractor. This is carried out through a ‘helpdesk system’which mostly involves the headteachers and teachers placing ‘ad hoc’ requests for serviceprovision (for example, drink spillage or set up of classroom) and ‘planned’ maintenance(for example, cleaning, testing of equipment, grass cutting and painting). The headteachersand teachers had mixed views on the effectiveness of the facilities management system.While headteachers of PFI schools spend less time dealing with facilities managementissues, they spend a considerable amount of time monitoring the services delivered. Viewson the positive aspects of the facilities management were typically expressed as follows:

Previously, say for example, if we have broken windows in the school, I was involvedin that. Under PPP, the facilities manager would be dealing with those issues. So,from this point of view, it is beneficial (Deputy headteacher, school 2).

Maintenance and building services provided by the private contractor are good.Cleaning and caretaking, after initial difficulties, are much better now (Headteacher,school 1).

The facilities managers seem to have significant autonomy and control over the moni-toring system and preparation of the helpdesk report. One contract manager from the localeducation authority explained the reliance on the private sector as follows:

. . . if services fall below the level specified in the contract, then the contractor willhave to bear for deductions from the unitary payments. We rely on the consortiumwhich is responsible to monitor its own performance standard based on those specifiedin the contract (Contract Manager, local education authority).

Whereas in the case of school 1 the headteacher and teachers ‘sign off’ the helpdesk reportafter the service provider has fulfilled their requests, in the case of school 3, headteachers andteachers do not. Concern over the accuracy of the helpdesk reports produced was expressedby a headteacher as follows:

If a piece of flooring is slightly raised, that is a trip hazard and should be addressedimmediately. The output specification would specify the length of time the contractorhas to deal with that. Unless you check that it is done within that specific time frame,how can you be sure? You can be told that something has been done, but unless you goand check that out, how do you know? How do you know their logs are accurate? Theoperational review report needs to be checked upon. Teachers are not informed whentheir job requests have been completed. That does not happen here (Headteacher,school 3).

Although the facilities management services were evaluated on the basis of certain cri-teria, in practice, there seem to be varying level of satisfaction with the services actuallydelivered. The service providers have considerable autonomy and control over the servicemonitoring process and the types of ‘helpdesk system’ used (for example, school 3’s helpdesk is located ‘offsite’ and it caters for other PFI projects as well, whereas the helpdesksfor school 1 and school 2 are located ‘onsite’ and they cater for the respective PFI schoolonly).

Page 18: The actual evaluation of school PFI bids for value for money in the UK public sector

1338 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 8School 1—sub-criteria used to evaluate the contractual aspects of the proposals

Sub-criteria Weight (%)

Effect of comments on project 30Level of detail response 25Integration of comments with bid 10Evidence of funder’s involvement 15Deliverability 20

Source: PricewaterhouseCoopers (1999a, vols. 1–3).

6.3. Evaluation of the contractual arrangements

The contractual aspects of the bids were evaluated by considering the bidders’ commentson the contract terms in the Draft Letters of Agreement prepared by the local educationauthority and their proposals for the transfer of staff from the public sector to the privatesector. Comments were assessed in terms of the proposed amendments suggested, theirimplications for risk transfer, and the overall relationship between the local educationauthority and the proposed contracting party. The sub-criteria used to evaluate thecontractual aspects of the bids for school 1 and school 3 are set out in Tables 8 and 9.School 1 was evaluated on the basis of five sub-criteria whereas seven sub-criteria wereconsidered in the case of school 3. The weightings used were also different.

The PFI contract is usually not referred to by the service provider and the school as thebasis for delivering PFI services, primarily because of its complexity and the difficultiesinvolved in operationalising them. The complexity of interpreting and enforcing PFI con-tracts was typically highlighted by a project manager from the local education authority asfollows:

The contracts for PFI are there in those two large boxes. The PFI contracts are verythick and very hard to interpret. The public sector does not possess good corporatelawyers. These documents are drawn up by corporate lawyers for lawyers. Theyare not drawn up by lawyers to be used by ordinary people like us. If you look atthe terminology, if you look at the phraseology, it is heavy going (Project manager,School 2).

Table 9School 3—sub-criteria used to evaluate the contractual aspects of the proposals

Sub-criteria Weight (%)

Impact of comments on project 30Level of detail response 20Integration of comments with bid 10Evidence of funder involvement 10Deliverability 15Quality of TUPE (transfer of undertakings protection of employment) proposals 5Consortium arrangements 10

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

Page 19: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1339

In practice, trust and good working relationships, rather than the contract, seem to playa vital role in the service delivery process. The contract is perceived to be important onlyin cases of major dispute. Both the headteachers and facilities managers have highlightedthe need for mutual understanding and good working relationships:

I suppose, in the end, it [delivery of PFI service] all depends on how good yourworking relationship is with your facilities management team (Deputy headteacher,school 2).

. . . the headteacher was not going to penalise us. He does not hit us with a qualityfailure and vice versa, we are not going to charge him if the pupils vandalise thewall. Before, in the case of the previous facilities manager, service delivery was bythe contract and this was very unpleasant and it did not work, hence, that facilitiesmanager was made to go . . .. The contract is there for a purpose, but the relationshipand partnership need to be built, otherwise the PFI will never work (Facilities manager,school 3).

Negotiation for items not included in the contract was a major concern to the schoolsand local education authorities because of their implications for risk transfer—for example,the private sector contractor assumes the risks associated with the building and servicesdelivery and changes to the initial contract (referred to as ‘contract variation’) may lead torisk-reversal to the public sector. This issue was highlighted as follows:

. . . whenever you want anything done to the building, for instance, negotiating theinstallation of data projectors and interactive whiteboards with the service provider,has been a nightmare. And that’s because the information technology network andinfrastructure were not included in the PFI . . . we are very careful of risks reversal(Headteacher, school 3).

For example, we have bought 36 interactive whiteboards but because they are beinginstalled in a PFI school, we have to go via the facilities manager to get their permissionand to install them correctly, although the money is the school’s. This is because they haveassumed the risks of the building, so they have to make sure that the whiteboards are putup safely and correctly (Teacher 2, school 3).

6.4. Evaluation of financial aspects of the bid and selection of the preferred bidder

The financial viability of the bids was evaluated by paying “particular attention to therobustness of the financial projections and the capacity of the proposed contracting party toabsorb risk” (PricewaterhouseCoopers, 1999a, para 7.6; PricewaterhouseCoopers, 1999b,para 7.3). The overall financial strength of the proposed deal was determined by a rangeof financial ratios and measures. The sub-criteria used to evaluate the financial aspects ofthe bids for school 1 and school 3 are set out in Tables 10 and 11, respectively. In the caseof school 1, eight sub-criteria were used to evaluate the financial strength of the proposalwhile in the case of school 3 only five sub-criteria were used. The weightings applied alsodiffered—school 1 seemed to have paid more attention in the weightings of the financialsub-criteria whereas school 3 applied an equal weighting of 20%. The financial sub-criteriawere also more specific in school 1 as compared to school 3.

Page 20: The actual evaluation of school PFI bids for value for money in the UK public sector

1340 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 10School 1—sub-criteria used to evaluate the financial viability of the proposals

Sub-criteria Weight (%)

Estimate of deliverability and robustness of financial proposals 40Evidence of support from funders—term sheet and letter of support 25Term of loan 5Covenants attaching to loan 5Annual debts service cover ratio 5Loan life cover ratio 5Strength of investors and deliverability of equity contribution 10Strength of Special Purpose Vehicle–debt/equity ratio 5

Source: PricewaterhouseCoopers (1999a, vols. 1–3).

The financial evaluation group was mostly concerned with the deliverability and robust-ness of the proposals, including evidence of financial support. Whereas the potential foroff balance sheet accounting for PFI contract was considered in the case of school 3, in thecase of school 1, this sub-criterion was dropped. HM Treasury’s (1999, para 1.8) positionon PFI accounting is that “value for money, and not the accounting treatment, which is thekey determinant of whether a project should go ahead or not”. More specifically, the publicsector should focus on sharing risks in a way that optimises value for money and not transferrisks to the private sector to get PFI assets off the public sector’s balance sheet. This adviceseems to have been followed in the VFM evaluation of school 1 but not school 3.

In the case of both schools, best and final offers were received from three short listedconsortia. Following a preliminary evaluation of the design, facilities management, contrac-tual and financial aspects of the bids by the four evaluation teams, the private sector bidderswere asked to further clarify certain aspects of their bids. The evaluation groups were thenre-convened to formally evaluate the best and final offers against the criteria approved bythe project board. Tables 12 and 13 present a summary of the aggregate scores assigned bythe evaluation teams to the bids received from the three consortia for school 1 and school3, respectively.

In the case of school 1, consortium M scored highest in terms of financial criteria (19.65)whereas consortium B obtained the highest overall score (70.90). Table 12 present the‘quantification of the qualitative criteria’ for school 1 and may be considered as the benefits(or value) of the different PFI proposals. However, the proposal with the highest score doesnot necessarily mean that it offers the best value for money as ‘costs’ have not been takeninto consideration.

Table 11School 3—sub-criteria used to evaluate the financial viability of the proposals

Sub-criteria Weight (%)

Adequacy of debt and equity of finance proposals 20Degree of commitment of debt and equity finance 20Financial strength of special purpose vehicle 20Robustness of financial projects 20Potential for off balance sheet accounting 20

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

Page 21: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1341

Table 12Aggregate bid scores for school 1

Main criteria Maximum score B N M

Design and construction 25 21.40 11.15 17.92Facilities management 25 11.76 8.94 7.97Financial 25 18.90 17.15 19.65Contractual 25 18.90 16.89 18.65

Total 100 70.90 54.13 64.19

Ranking 1 3 2

Source: PricewaterhouseCoopers (1999a, vols. 1–3).

Table 13Aggregate bid scores for school 3

Main criteria Maximum score A D N

Design and construction 35 19.3 25.4 22.2Facilities management 25 12.9 16.3 14.5Financial 20 15.2 13.2 13.6Contractual 20 10.5 12.6 11.9

Total 100 57.9 67.5 62.2

Ranking 3 1 2

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

Table 14 shows how the Net Present Cost of the PFI proposals submitted by the threepotential bidders’ stack up with the PSC. While the lowest PFI proposal may be attractive,it may not necessarily deliver the best value for money—a cost per unit of benefit needsto be computed by dividing the Net Present Cost payable with the preceding bid scoresto justify the ‘winning bid’. Consortium B’s bid resulted in the lowest cost per unit ofbenefits (17.193/70.90 = £0.24m) as compared to consortium M (17.695/64.9 = £0.27m)and consortium N (17.551/54.13 = £0.32m) and was nominated as the preferred bidder onthis basis.

In the case of school 3 (see Table 13) all three consortiums scored close to the mean scoreof 62.5—consortium D obtained the highest overall score (67.5) followed by consortiumN (62.2) and A (57.9). As regards the financial VFM consideration, only consortium N’sproposal demonstrated better VFM compared to the PSC (see Table 15).

Consortiums N’s bid resulted in the lowest cost per unit of benefits (21.5/62.2 = £0.35m)as compared to A (25.9/57.9 = £0.45m) and D (30.5/67.5 = £0.45m). Consortium N was

Table 14Economic analysis for school 1

Bidder Net Present Cost (£000) PSC (£000) Difference (%) Rank

B 17,193 18,971 9.4 1N 17,551 18,971 7.5 2M 17,695 18,971 6.7 3

Source: PricewaterhouseCoopers (1999a, vols. 1–3).

Page 22: The actual evaluation of school PFI bids for value for money in the UK public sector

1342 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

Table 15Economic analysis for school 3

Bidder Net Present Cost (£m) PSC (£m) Difference (%) Rank

N 21.5 21.5 0 1A 25.9 21.5 20 2D 30.5 21.5 39.5 3

Source: PricewaterhouseCoopers (1999b, vols. 1 and 2).

therefore nominated as the preferred bidder for school 3 and was eventually awarded thecontract.

In sum, although the bid evaluation methodology for school 1 and school 3 were broadlysimilar, there were variations in terms of the criteria actually used and their weightings,the degree of specification of the criteria, and users’ involvement in the decision-makingprocesses. The analysis also reveals an additional limitation of the PSC—while a cost perunit of benefit was computed for the ‘PFI option’ to justify the winning bid, an equivalentcost per unit of benefit was not computed for the conventional procurement option. Assess-ing the ‘benefits’ of the conventional procurement option in terms of the design, facilitiesmanagement, financial and contractual considerations would require the conventional pro-curement option to be subject to the competitive bidding process as well. This may not bepractical, as Grimsey and Lewis (2005) point out, because this alternative involves con-siderable commitment in terms of time and resources that the VFM evaluation approachcurrently used by the UK government, to some degree, obviates.

7. Discussion and conclusion

PFI is deeply rooted in the government’s ideological commitment to adopt new publicmanagement reform in an era where limited public funds are available for infrastructuredevelopment. The government has constantly used the VFM rhetoric to legitimise theincreasing use of PFI to deliver public services. This paper has provided some empiri-cal evidence on how PFI decisions are made in the education sector in Northern Irelandand how school PFI bids are actually evaluated for VFM at the micro organisational level(Broadbent and Laughlin, 1999). This study further demonstrated the importance of risktransfer; illustrated the use of ‘accounting measures’ to quantify the qualitative factors;and highlighted the public sector’s reliance on professional accounting firms in the PFIdecision-making processes.

More specifically, this paper provides some evidence to suggest that the school PFIdecision-making processes in Northern Ireland were heavily controlled by the local educa-tion authority and the DENI. There was a lack of ‘strategic’ planning of the education estatealthough the DENI is now developing a database detailing the state of repair of the schools inNorthern Ireland. The PFI procurement process was dominated by the PFI and PSC compari-son while inadequate attention was paid to the consideration of the implication of PFI on: thebroader education estate, non-PFI schools, other parts of the public sector, and future gen-erations of users and taxpayers. The fact that interviewees perceive PFI as the only option toobtain a new school casts doubts on the degree of objectivity in the VFM assessment process.

Page 23: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1343

The economic analysis of the PSC supports prior studies (see for example, Shaoul, 2005)that the financial benefits of the school PFI contracts were marginal (and negative in thecase of two schools) and were dependent on the adjustments for risk transfer. Neverthe-less, these projects were allowed to proceed by the DENI on the grounds that they were‘pathfinders’. Moreover, the paper illustrated the process of ‘quantifying’ the various qual-itative considerations used to evaluate PFI bids for VFM. It provided evidence that the PFIand PSC comparison includes assessment of non-financial criteria, at least in the case ofthe PFI option. However, the criteria used and the weightings were subjective and arbitrary;changes in the weightings may easily change the choice of the preferred bidder and/or shiftthe balance in favour of the traditional public sector procurement option. The problematicconcept of VFM was reduced to a ‘cost per unit of benefit’ and formed the basis for makingthe final PFI decision and selecting the preferred service provider. However, an equivalent‘cost per unit of benefit’ could not be computed for the conventional procurement option.The interviews further indicated that the headteachers and teachers in school 1 were moresatisfied with the PFI building and services actually delivered primarily because of their‘active’ involvement in the procurement process. The school has also been more specific interms of framing the criteria used to appraise the PFI bids for VFM.

There are additional complications stemming from the contractual nature of PFI andarrangement for monitoring of services. It is based on the premise that the services deliveredwould be ‘self-monitored’ and reported by the private sector. It also puts the burden forreporting service failures to the schools which are not signatory to the PFI contract. Althoughthe payment mechanism is related to the service level agreement specified in the contract,the schools have no power over the private sector contractor and are unable to question theintegrity of their monitoring reports and the underlying internal control systems. Moreover,the services criteria specified in the PFI contract may themselves become obsolete, forexample, due to changes in the curriculum. Therefore, the public sector may be tied upinto paying for unwanted facilities which in turn would have repercussions on the otherparts of the education estate and the public sector. Moreover, the assessment of VFM inPFI contracts may be hindered by the lack of transparency and public accountability in PFIprocesses (see also, Edwards et al., 2004)—for example, the local authorities had to obtainlegal advice and the approval of the private sector to release the full business cases of thePFI schools. Moreover, the actual PFI contract itself is not in the public domain.

The scope of this research has been limited to studying the bid evaluation processesof school PFI contracts in Northern Ireland and as such any generalisation of the findingsshould be undertaken with caution. There are variations in actual practices in differentparts of the public sector, across similar types of projects (as illustrated in this paper)and geographically. For example, Grimsey and Lewis (2005) pointed out that while ‘fullydeveloped UK-type PSC approach’ is commonly used in Australia, it was not used in the roadPFI projects in Ireland. A further limitation is that this project was not explicitly concernedwith post project evaluation of school PFI contracts, although where relevant, it exploredthe interviewees’ perception of the VFM evaluation criteria at the post-implementationstage. Moreover, it did not explore differences in perceptions across interviewees and overtime, which would have been beyond its scope. Future research might explore how PFIbids are evaluated for VFM in other parts of the public sector and in other countries toidentify the VFM evaluation methodology used and highlight best practices. It might also

Page 24: The actual evaluation of school PFI bids for value for money in the UK public sector

1344 I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345

be useful to further explore how VFM is actually perceived among the different types ofstakeholders involved in PFI projects and how their perception of VFM changes over time.Such a longitudinal analysis would involve the development of “a more robust design forpost project evaluation” and a follow-up of the financial and non-financial criteria used toevaluate PFI at the procurement stages (Broadbent et al., 2008, p. 30). Only through longerterm evaluation that it would be possible to draw definitive conclusions on the VFM meritsof the PFI policy.

Acknowledgements

I acknowledge with thanks the constructive comments received from Maurice Pendle-bury, Rowan Jones, Donal McKillop, Istemi Demirag, Bride Mallon and four anonymousreviewers of this journal. All errors and omissions remain my personal responsibility.

References

Aerts W. On the use of accounting logic as an explanatory category in narrative accounting disclosures. Accounting,Organizations and Society 1994;19(4-5):337–53.

Blumer H. Symbolic interactionism: perspective and method. Englewood Cliffs, NJ: Prentice-Hall; 1969.Broadbent J, Laughlin R. The private finance initiative: clarification of a future research agenda. Financial

Accountability and Management 1999;15(2):95–114.Broadbent J, Gill J, Laughlin R. Evaluating the private finance initiative in the national health service in the UK.

Accounting, Auditing and Accountability Journal 2003;16(3):422–45.Broadbent J, Gill J, Laughlin R. The private finance initiative in the National Health Service. CIMA Research

Report. London: CIMA; 2004.Broadbent J, Gill J, Laughlin R. Identifying and controlling risk: the problem of uncertainty in the private finance

initiative in the UK’s National Health Service. Critical Perspectives on Accounting 2008;19:40–78.Broadbent J, Laughlin R. The role of PFI in the UK Government’s modernisation agenda. Financial Accountability

and Management 2005;21(1):75–97.Connolly C, Hyndman N. The actual implementation of accruals accounting: caveats from a case within the UK

public sector. Accounting, Auditing and Accountability Journal 2006;19(2):272–90.DENI. Capital programme-process of determining allocations. Northern Ireland: Department of Education; 2002.DENI. Jane Kennedy Publishes List Of Contenders To Compete For Capital Funding. Press Release 31/03/04;

2004.DENI. New Procurement and Delivery Arrangements for the Schools’ Estate. Northern Ireland: Department of

Education; 2005.Edwards P, Shaoul J. Controlling the PFI process in schools: a case study of the Pimlico project. Policy & Politics

2003a;31(3):371–85.Edwards P, Shaoul J. Partnerships: for better, for worse? Accounting, Auditing & Accountability Journal

2003b;16(3):397–421.Edwards P, Shaoul J, Stafford A, Arblaster L. Evaluating the operations of PFI in roads and hospitals. ACCA

Research Report. London: Certified Accountants Educational Trust; 2004.English L. Using public-private partnerships to achieve value for money in the delivery of healthcare in Australia.

International Journal of Public Policy 2005;1(1/2):91–121.English L, Guthrie J. Driving privately financed projects in Australia: what makes them tick? Accounting, Auditing

& Accountability Journal 2003;16(3):493–511.Froud J. The private finance initiative: risk, uncertainty and the state. Accounting, Organizations and Society

2003;28(6):567–89.

Page 25: The actual evaluation of school PFI bids for value for money in the UK public sector

I. Khadaroo / Critical Perspectives on Accounting 19 (2008) 1321–1345 1345

Gaffney D, Pollock A, Price D, Shaoul J. PFI in the NHS—is there an economic case? British Medical Journal1999;319:116–9.

Garfinkel H. Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall; 1967.Grimsey D, Lewis MK. Are public private partnerships value for money? Accounting Forum 2005;29(4):345–78.Heald D. Value for money tests and accounting treatment in PFI schemes. Accounting, Auditing

2003;16(3):342–71.HM Treasury. Partnerships for prosperity. London: HMSO; 1997.HM Treasury. Technical Note 5: how to construct a public sector comparator. Treasury Taskforce Guidance.

London: HMSO; 1999.HM Treasury. Public private partnerships: the government’s approach. London: HMSO; 2000.HM Treasury. The Green Book: appraisal and evaluation in Central Government. London: HMSO; 2003.HM Treasury. Building Britain’s long-term future: prosperity and fairness for families Budget report 2007, HC

342. London: HMSO; 2006.Hood C. The new public management in the 1980s: variations on a theme. Accounting, Organizations and Society

1995;20(2/3):93–109.Humphrey CG, Scapens RW. Methodological themes theories and case studies of organizational accounting

practices: limitation or liberation? Accounting, Auditing and Accountability Journal 1996;9(4):86–106.Ismail S, Pendlebury M. The private finance initiative (PFI) in schools: the experiences of users. Financial

Accountability and Management 2006;22(4):381–404.NIAO. Building for the future: a review of the PFI education pathfinder projects. Belfast: Northern Ireland Audit

Office; 2004.Osborne D, Gaebler T. Reinventing Government: how the entrepreneurial spirit is transforming the public sector.

New York: Penguin; 1993.Pollock AM, Shaoul J, Vickers N. Private finance and value for money in NHS hospitals: a policy in search of a

rationale? British Medical Journal 2002;324:1205–9.PricewaterhouseCoopers, Chesterton. School 2, full business case. Belfast: PricewaterhouseCoopers and Chester-

ton Consulting; 1998.PricewaterhouseCoopers. School 1, full business case. Belfast: PricewaterhouseCoopers; 1999a.PricewaterhouseCoopers. School 3, full business case, vols. 1–3. Belfast: PricewaterhouseCoopers; 1999b.Public Accounts Treasury Committee. The private finance initiative, HC 147. London: HMSO; 2000.Shaoul J. A financial appraisal of the London underground public-private partnership. Public Money and Man-

agement 2002;22(2):53–60.Shaoul J. A financial analysis of the national air traffic services PPP. Public Money & Management

2003;23(3):185–94.Shaoul J. Railpolitik: the financial realities of operating Britain’s national railways. Public Money & Management

2004;24(1):27–36.Shaoul J. A critical appraisal of the private finance initiative: selecting a financial method or allocating economic

wealth? Critical Perspectives on Accounting 2005;16(4):441–71.