A personal perspective on Cost-benefit Analysis and the Balance of

Competences Review

Peter Andrews

Chief Economist

UK FCA

All of the views expressed here are my own

They do not reflect any position of the FCA

The FCA is not responding to the Balance of Competences Review,

making formal statements on it or taking any positions on it

Overview of argument

• My issue: is it technically possible to make a realistic CBA of a nation joining a multinational ‘single market’ with some non-economic shared legislation?

• A possible shortcut? The sum of Commission CBAs (depending on nature of policy and what these CBAs capture)

• Failing this, is CBA as practised reliable or in need of reform? Latest evidence from the USA

• How does Commission CBA measure up?• Europe Economics’ “CBA” of UK membership –

what should we make of this?

List of contents

• Background

• The Commission’s use of CBA (‘Impact Assessment’)

• General issues with CBA

• Comments on US CBA reform proposals

• Does Commission CBA meet these proposals?

• Europe Economics’ observations on CBA of UK membership

Background 1

• The review of the balance of competences is an audit of what the EU does and how it affects the UK: https://www.gov.uk/review-of-the-balance-of-competences

• The first part of the review concerns the Single Market

• The consultancy Europe Economics has prepared a report which extensively considers the costs and benefits of the Single Market: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/224579/bis-13-1058-europe-economics-optimal-integration-in-the-single-market-a-synoptic-review.pdf

Background 2

• The Europe Economics report considers what tools promote integration, how to assess whether integration is optimal and how to measure integration

• It also draws conclusions on:– How can the benefits of integration be

measured in specific sectors?– How can the costs of integration be

measured in specific sectors?– How can policy costs and benefits be

assessed according to some objective or transparent standard?

Commission’s use of CBA 1

• If the Commission undertook rigorous CBA that excluded pure transfers (e.g. between citizens of different member states), it could in principle design regulatory initiatives to make the EU as a whole better off

• Methodology looks very good: http://impel.eu/wp-content/uploads/2010/01/European-Commission-Impact-Assessment-Guidelines-iag_2009_en.pdf

Commission’s use of CBA 2

• The governance also looks very good

• Substantial role designed for independent Impact Assessment Board: http://ec.europa.eu/governance/impact/iab/iab_en.htm

• And the Board’s challenges are real/frequent: http://ec.europa.eu/governance/impact/key_docs/docs/iab_report_2012_en_final.pdf

Commission’s use of CBA 3

• But CBA of public policy exists at a confluence of very difficult technical and political issues: Radaelli and Meuwese –Measuring the quality of impact assessment http://centres.exeter.ac.uk/ceg/research/riacp/documents/Measuring%20the%20quality%20of%20impact%20assessment.pdf

• And a lot of literature flows from: Becker, Gary S. (1983)"A Theory of Competition among Pressure Groups for Political Influence, "Quarterly Journal of Economics, 98, 371-400

Commission’s use of CBA 4

• Also, Commission cast some doubt on use of CBA in financial services regulation: Communication concerning the review of Directive 94/19/EC on deposit guarantee systems in the EU suggests requirements would have been stiffer but for ‘better regulation’ (CBA/evidence disciplines)

• But evidence matters! See N Moloney, How to protect investors, Cambridge University Press, page 114

• Overall, not safe to conclude that Commission’s use of CBA means that all member states are made better-off by regulations and directives

General issues with CBA 1• Traditional generalist culture/sceptical view

of CBA in UK: see Baldwin & Cave, Understanding Regulation, Oxford, 1999

• Weak Judicial Review: Associated Provincial Picture Houses Ltd v WednesburyCorporation (1948) 1 KB 223: decision so unreasonable that no reasonable person acting reasonably could have made it – hard to challenge UK CBAs (and challenges rare)

• So traditional view: CBAs that fall short of standards of economic analysis required for academic journals are OK for UK public policy decisions

General issues with CBA 2

• In any case, practical and technical difficulties are considerable. See Irish Government’s CBA Guide: http://publicspendingcode.per.gov.ie/wp-content/uploads/2012/08/D03-Guide-to-economic-appraisal-CBA-16-July.pdf

• Practical issues include biases in estimation

• Technical difficulties include widespread need for ’shadow prices’ when market prices don’t exist or are distorted by imperfections (a common case)

General issues with CBA 3

• In US, financial sector successfully challenged SEC Dodd-Frank rules on CBA grounds, e.g. proxy access rule defeated in DC Circuit Court of Appeals 2011: http://www.risk.net/risk-magazine/feature/2189269/cost-benefit-defeat-weighing-gun-shy-sec

• Also: American Equity Inv. Life Ins. Co. v. SEC, 613 F.3d 166, 178-79 (D.C. Cir. 2010) on inadequate ‘counterfactual’

General issues with CBA 4

• This led to expanded CBA resources and practice at the SEC: http://www.sec.gov/divisions/riskfin/rsfi_guidance_econ_analy_secrulemaking.pdf

• Harvard-based Committee on Capital Markets Regulation (CCMR) recently proposed ‘A balanced approach to CBA reform’: http://capmktsreg.org/2013/10/balanced-approach-to-cost-benefit-analysis-reform/

General issues with CBA 5

• CCMR’s main proposals:

– Mandate consistent standards

– Focus on economically significant rules

– OIRA (independent) review of CBAs

– Utilize outside resources

– Utilize pilot studies

– Prospectively require retrospective review

– Modify standards of judicial review

– Facilitate information collection

– Devote adequate resources

Comments 1

• Comments are on CCMR CBA reform proposals above

• Mandate consistent standards –probably necessary for policing/ accountability but certainly not sufficient: culture, resources and top management support matter most

• See John Howell & Co’s N2+2 Review of CBA at the FSA: www.betterregulation.com/doc/2/4740

• And CBA quality depends on analytical specifics…not high-level standards

Comments 2

• Focus on economically significant rules: yes, for resources and reputation

• OIRA review: yes, little dispute in UK that ‘National Audit Office’ (NAO) adds significant value

• See for example NAO report on FSA under FSMA, section 12: http://www.nao.org.uk/wp-content/uploads/2007/04/0607500.pdf

Comments 3• Utilize outside resources and pilot studies:

yes – especially if extending to field trials

• See pp40+ 52 of FCA Occasional Paper 1 ‘Applying behavioural economics at the FCA’ re RCTs and which remedies work

• Prospectively require retrospective review: yes, very hard to capture resources for this

• See NERA’s report on FSA ex post and cumulative CBA, 2004 (part of N2+2 Review, above)

Comments 4• Modify standard of judicial review – no comment

on US law of course but interesting that DC court thought standards of CBA set by then SEC economists/lawyers were too low

• Facilitate information collection – yes, costs of collection should be low relative to benefits of making markets work well

• Devote adequate resources for proper CBA – yes …if CBA is focused on economically significant rules (see above), if CBA is influential (scope of political economy) and if the degree of policy uncertainty justifies the substantial, research-type projects that may be necessary

Comments 5• FCA sees CBA as useful check of proportionality not

sole decision criterion

• Therefore supportive of CCMR’s paper, subject to caveats above

• But recognise frequent and substantial difficulty in producing CBAs

• Suggests regulators should be set challenging CBA targets but not mission impossible

• Sometimes, alternatives to CBA are enough:– Integrated Analysis of market failures + theoretical model of

intervention

– Long-term research programme on policy issues

– RCTs of remedies can give better cost and benefit data (e.g. consumer response to disclosure must be tested anyway) but not always market responses

Stocktake 1

• Are the common weaknesses in CBA discussed above a barrier to Europe Economics’ CBA for the Review?

• No

• How well does the Commission’s Impact Assessment stand up against the CCMR proposals?

• See next slide

Stocktake 2

• Commission already meets:– Mandate consistent standards– Focus on economically significant rules– OIRA (independent) review of CBAs– Utilize outside resources– Prospectively require retrospective review

• Commission might not meet:– Utilize pilot studies– Right level of judicial review (Commission is too

protected from legal challenge for poor analysis?)– Facilitate information collection– Devote adequate resources

Europe Economics’ observations 1

• On benefits, Europe Economics uses a combination of general reflections upon data, econometrics, and analytical judgement

• It thereby illustrates how one might explore the relationship between integration in the Single Market and trade, competition, efficiency and innovation for the pharmaceuticals, financial services, motor vehicles and energy sectors

• It expresses optimism about the scope for drawing fairly objective quantitative results in respect of these benefits in respect of tradecompetition and efficiency.

Europe Economics’ observations 2

• On costs, Europe Economics explores trade diversion and process costs, including the costs of regulation

• It is confident in principle about measuring regulatory costs

• But notes that there is an inevitable element of judgement involved in determining how much additional regulatory costs the EU creates that would not have been incurred anyway through national regulatory interventions

• It also notes that “the ways regulatory costs are typically analysed in regulatory impact assessments is not precisely suited to our purpose here, in that many EU regulations are intended to be deregulatory or liberalising — the very essence of the stripping away of non-tariff barriers is that doing so should have the net effect of reducing regulatory costs, not increasing them”

Europe Economics’ observations 3• Europe Economics concludes: “We consider policy costs and benefits to be

potentially as large as or even larger than the concrete costs and benefits of the Single Market, bearing in mind especially the point that the key goals of the Single Market project may be political (and hence directly related to policy influence) as much as trade-oriented”

• It also explores whether membership might make a country worse-off (on the basis that a country knows its own interests)by exploring “how measures of dissent, such as occasions the UK has been outvoted via QMV, might supplement more widely-used coarser-grained measures of policy costs, such as the number of lost vetoes”

• Similarly, it suggests that “detailed reflection upon the dynamic of policy-making can reveal the UK’s influence or lack thereof and how this changes through time” and ingeniously explores financial services showing “how the UK’s past influence can be seen through detailed consideration of the structural similarities between key EU and UK financial regulations and the differences between these and previous regulation in other Member States”

• This suggests that benefits were obtained in the past, although Europe Economics notes that similar analysis undertaken today on financial services regulation is less encouraging

What should we make of this 1

• Europe Economics is clearly aware of the difficulties of CBA of something like UK membership of the EU

• It has developed a number of clever work-arounds to draw inferences about what might be true

• But of course such approaches must be considered to have a wide margin of error

What should we make of this 2

• Generally, a CBA of X is the difference in states between the world with X and the world without X

• The latter is the counterfactual

• It is clear that the counterfactual to the UK not being a member of the EU is unobservable and can only be constructed very approximately (what labour force, what trading relationships, what innovations, etc) and with great difficulty

• Europe Economics’ pragmatism is sensible

What should we make of this 3

• The difficulty of making reliable CBAs can be seen in the controversy over the HS2 CBA and in the extent of the caveats in FSA Occasional Paper 42 on the relationship between bank capital and crisis and bank capital and growth (www.fsa.gov.uk)

• The difficulties in these CBAs are small relative to those that arise in any attempt to CBA UK membership of the EU – a fascinating challenge!