NSW Government Evaluation Conference Using Cost-Benefit Analysis: Don’t be Afraid Dr. Peter...

NSW Government Evaluation Conference

Using Cost-Benefit Analysis: Don’t be Afraid

Dr. Peter AbelsonApplied Economics P/L

NSW Treasury

September 2015

Contents

• Cost-Benefit Analysis Basics

• CBA of Community Licensing Scheme Northern Territory

• CBA of Ten Person Unit for People with Severe Intellectual Disability and Mental Disorders

• Conclusions: CBA and Alternative Evaluation Methods

Cost-benefit analysis: Basics• CBA attempts to estimate total benefits and costs of policies,

projects or programs to a defined community

• Benefits are increases in human well-being; costs are reductions in well-being

• Includes “economic”, social and environmental goods: they may be market or non-market goods

• Values benefits in dollars based on individuals’ own valuations

• Values costs in dollars based on opportunity costs

• Aggregates benefits and costs to = total benefit less total cost

• CBA has general application (see next slide)

Process of CBA: key steps1. Identify issues and objectives

2. Develop project / policy options, including base case option

3. Forecast (quantity) outcomes of each option over lifetime of project / policy

4. Estimate value of these outcomes (costs and benefits) in dollars

5. Estimate overall value of project or policy

6. Test for uncertainty and risks

7. Assess distributional issues

8. Prepare report including CBA and qualitative effects

CBA Measures of Net Social Benefit• Concept of present value (PV)

– PV = w1B1 + w2B2 = B1/(1+r) + B2/(1+r)2

• Net present value– NPV = PV(B) – PV(K +C)– NPV > 0 represents positive net social benefit

• Benefit-cost ratio– BCR = PV(B)/PV(K+C) or PV(B-C)/PV(K) – BCR > 1 represents positive net social benefit

• B = benefits, K = capital cost, C = recurrent costs • w = time weighting, r = discount rate

Six critical issues in CBA• Are estimated (quantity) outcomes accurate?

• Are the valuation principles acceptable? • Do the valuation methods provide accurate valuations

of benefits and costs? • How are valuations aggregated and compared?

• How to deal with outcomes over time (discount rates)?

• How should we deal with risk and uncertainty?

CBA of Community Stores Licensing Scheme in Northern Territory

• NT National Emergency Act 2007 prescribed various interventions including income management, alcohol restrictions, community store licensing, etc.

• An estimated 36,500 indigenous people live in 73 community areas. At time of study (mid-2009), 86 community stores were licensed to provide a reasonable range of quality foods under income-managed scheme.

• Main aim of scheme was to provide remote Indigenous communities with healthy food (low on fat, sugar and salt) along with restrictions on expenditure of welfare benefits and to have significant ongoing health benefits.

Major Costs and Benefits• Major costs of licensing schemes were Australian Government’s

expenses in setting up and managing the scheme. This included significant subsidies via publicly owned Outback Stores. And community stores experience compliance costs.

• The main quantified benefits are improved health outcomes. These outcomes are measured in estimated reductions in disability adjusted life years (DALYs) and valued at Australian Government recommended value (then) of $151,000 / DALY.

• This is a general Australian standard which includes most benefits of improved health, including productivity. But possibly not social benefits.

Estimating outcomes: major challenge

• Need to estimate – Changes in behaviour and – Effects of changes in behaviour.

• School of Population Health (Qld. Uni) estimated:– Indigenous people experience 209 DALYs per 1000 people, cf. 86

DALYs per 1000 people nationally. – Inadequate fruit and vegetable consumption responsible for 3.5% of

all DALYs in remote areas, i.e. 360 DALYs for 36,500 people (note a small % of the total DALYs)

– With detailed modelling of demographics, a 20% increase in fruit and vegetable intake would produce a 28% reduction in DALYs due to poor food consumption (i.e. reduction of 100 DALYs).

Economic Evaluation of Current Stores Licensing Scheme: 5 years licensing: excludes OBS: $m, 2009 prices

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2104-15

2015-16

2105-16

Costs Commonwealth expenses Departmental expenses 1.65 2.10 4.50 3.50 2.50 Administered expenses 1.65 2.60 3.70 2.10 2.00 Total 3.30 4.70 8.20 5.60 4.50 0.00 0.00 0.00 0.00 0.00 Compliance costs Set-up costs 0.29 0.29 0.29 Annual costs 0.26 0.52 0.52 0.52 All Expenses 3.59 5.25 9.01 6.12 5.02 0.00 0.00 0.00 0.00 0.00 Benefits Health benefits 0.00 1.51 4.61 6.12 7.17 5.74 4.30 2.30 1.43 0.00 Net benefits -3.59 -3.74 -4.41 -0.01 2.15 5.74 4.30 2.30 1.43 0.00 NPV @ 7% ($0.07) NPV @ 4% $1.42 NPV @ 10% ($1.21) Sensitivity tests: NPV @ 7% Exclude sunk costs first 2 years $7.50 Population in areas 50,000 $8.61

Comments• Other results– Including role of Outback Stores and longer period for

program improved results– Results could vary (better in larger communities)

• Other options (not considered)– Subsidies for fruit and vegetables – could increase prices

and hard to administer– No radical alternative (subsiding community move)

• Valuations– Paternalistic versus individual valuations.

Ten Person Unit for People with Severe Intellectual Disability and Mental Disorders

• Concept: a specialist secure 10 person unit with intensive therapy services for people with severe disorders with aim to enable return residents to community.

• Can view as CBA or cost-effectiveness study. The estimated benefits are reduction in high costs of anti-social costs of behaviour. Thus high costs would be incurred to reduce other costs.

Costs of proposal

• Capital costs including land value = $4.9m

• Recurrent costs = $6.5m p.a.– Just over 50 FTEs including 20 disability workers

and 16 registered nurses (all detailed) = $5.8 m with onc-costs

– About $0.7m in other recurrent costs– i.e. about $650,000 per in-house person p.a.

Costs of alternative pathways

• Included costs associated with:– Police, corrective services, courts, legal aid, disability

services, health, Justice health, Primary health, Housing NSW, NGO housing, Office of the Public Guardian, Mental Health Tribunal, public and private property, victims compensation and Centrelink.

• Based on four detailed case studies and research at UNSW– Average cost was $470,000 per person per annum.

Benefits of program

• Assumed that half of the 10 residents would be able to transit out of the highly intensive unit after 3 years (and this would continue)

• The public costs of the transiting residents would fall by 50% from average $470,000 p.a. to $235,000 p.a.

• This is modelled in spreadsheet next slide.

Economic evaluation of 10 person Unit (2012 $m)

Costs of 20 persons with unitCapital cost of unit 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Land cost / site preparation 1.10 Construction costs 2.62 Design / professional fees 0.86 Other capital costs 0.31 Total capital cost 4.89Operating cost of unit Basic staff costs 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 5.18 Various on costs 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 0.68 Other recurrent costs 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 0.66 Total recurrent costs 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52 6.52Costs of transited clients Clients transiting end year 4 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 Clients transiting end year 7 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 Clients transiting end year 10 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 1.18 Total clients transiting costs 1.18 1.18 1.18 2.35 2.35 2.35 3.53 3.53 3.53 3.53 3.53 3.53 3.53 3.53 3.53 3.53 3.53Total cost for 25 clients 4.89 6.52 6.52 6.52 7.695 7.695 7.695 8.87 8.87 8.87 10.045 10.045 10.045 10.045 10.045 10.045 10.045 10.045 10.045 10.045 10.045

Costs of 20 persons without unit 10 clients entering in year 2 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 4.70 5 clients entering in year 5 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 5 clients entering in year 8 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 5 clients entering in year 11 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35 2.35Total costs without entry 0.00 4.70 4.70 4.70 7.05 7.05 7.05 9.40 9.40 9.40 11.75 11.75 11.75 11.75 11.75 11.75 11.75 11.75 11.75 11.75 11.75

Net benefit of proposed unit -4.89 -1.82 -1.82 -1.82 -0.65 -0.65 -0.65 0.53 0.53 0.53 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71 1.71

4% 7% 10%NPV $0.12 -$2.96 -$4.71

Comments• On above assumptions agreed with client, proposed Unit is

marginally negative, with a NPV of $-2.96 million (with 7% discount rate).

• Results are sensitive to the inputs. If alternative pathways

have an average cost of $500,000, NPV increases to +$1.26 million. More effective transit rates would also increase the economic return to the Unit.

• The evaluation makes no allowance for well-being and quality

of life of the clients. For some the Unit would represent a substantial improvement on their quality of life. Others could find the restrictions and isolation reduce their quality of life.

Other Economic Evaluation Methods

• An “economic evaluation” is an evaluation of the use of scarce resources using a money metric.

• Examples: – Cost-effectiveness analysis– Financial analysis– Maximising the value of output: GDP or GSP (gross

state product)– Economic impact analysis: the value of output or

incomes in a region or local area

• CBA is most complete form of economic evaluation.

Multi-criteria Analysis

Criteria Weight Score Score Score A B C

1 40 25 35 302 30 15 15 253 20 15 10 54 10 10 8 5

Total 100 65 68 60

Multi-criteria Analysis (cont.)• MCA helps to organise issues systematically and transparently. • However, there is subjectivity and arbitrariness at each main

point in the process: – in selecting criteria, – in allocating weights to each criterion, – in allocating point scores to each option for each criterion.

• The core weakness: MCA and other non-economic methods of valuation are not based on testable valuation principles.

• In absence of valuation principles, the evaluation process is wide open to manipulation.

Conclusions• CBA is the most comprehensive form of economic evaluation

and is generally recommended by central government agencies.

• CBA is concerned with welfare.

• CBA is based on principles of value and has reasonable practical capacity to implement these principles.

• CBA can go wrong especially if options are not considered or if forecasts of outcomes are wrong.

Conclusions

CBA is not especially complex or expensive

DON’T BE AFRAID!