View
247
Download
1
Category
Tags:
Preview:
Citation preview
Governance by Targets & Terror: Synecdoche, Gaming & Audit
Gwyn Bevan (LSE) & Christopher Hood (All Souls College, Oxford)Westminster Economic Forum
20 April 2005
Governance by Targets & Terror
New approach to health care governance in 2000s
Decisive breakthrough in governance – or repeat of history of Soviet Union?
Conclusion
Governance by Targets & Terror New approach to health care
governance in 2000s targets & indicators linked to negative
feedback Voltaire: ‘ici on tue de temps en temps un
amiral pour encourager les autres’ Decisive breakthrough in governance –
or repeat of history of Soviet Union? Conclusion
Three phases of governance of public health care systemUK Concordat clinical autonomy & resource
constraints no real command or control from the centre 1980s: various attempts to empower managers
1990s: attempt to control public health-care professionals through quasi-markets
separating providers & purchasers
England only from 2001- new ‘concordat’ of higher spending
accompanied by P.I.s & targets monitored from centre by multiple & overlapping units (& ‘terror’?)
NHS spend as % of GDP
0123456789
10
% GDP
Sources: Office of Health Economics, HM Treasury, & official projection of 9.4 per cent by 2008
Prime Minister’s Delivery Unit22 targets
Treasury130 PSA targets– c. 10 for health
Dept of Health50 targets by trusttype
Healthcare Commission Quality regulator develops & publishes ratings
Health care ‘trusts’ (c.700)
Audit Commission & NAOAudit finance & vfm
money reporting dialogue
Institutional arrangements
Some underlying assumptions Synecdoche: a part can meaningfully
stand for the whole
What is measured is a good indicator of performance (e.g. ‘threshold effects’ at the top of the quality range either will not occur or do not matter)
Gaming effects (the ‘knights-to-knaves’ problem) are either small or unimportant
Defining priorities
Residual domain β
Domain α: government’s priorities
Measuring priorities
Residual domain β
Domain αg: good measures M[αg]: no false positives or negatives
Domain αn:
no measures
Domain αi: imperfect measures M[αi]: large numbers false positives & negatives
The synecdoche assumption: the part can represent the whole
Domain α-: government’s priorities for which good & imperfect measures exist
Residual domain β+: omitted because unimportant or cannot be measured
‘Threshold Effects’ either don’t matter or won’t Happen
Target
Success
Waiting timeIn months
Failure?
Frequency
Knights either will not turn into Knaves or Knavery can be Controlled‘Saints’ may not share mainstream goals public service ethos so high that they voluntarily disclose
shortcomings to central authorities‘Honest triers’ broadly share mainstream goals do not voluntarily draw attention to their failures but do not attempt to spin or fiddle data in their favour‘Reactive gamers’ broadly share mainstream goals, but aim to spin or fiddle data if they have a motive or
opportunity to do so‘Rational maniacs’ do not share mainstream goals aim to manipulate data to conceal their operations
Target & PI systems against 4 types of actors
‘Saints’
‘Honest triers’
‘Reactive gamers’
‘Rational maniacs’
Expected effect of targets
NO CHANGE
What is measured is a good indicator of performance?
Works for saints Problems for honest triers Vulnerable to Reactive gamers Fails for rational maniacs Gresham’s law: saints & honest
triers Reactive gamers?
What is measured is a good indicator of performance? Agent satisfied signals M[α-]
Domain α-
All is well? Domain α- but
domain β+ ? Domain α- ? Failure on M[α-]
concealed by problems of
definition & measurement?
gaming?
Domain β+
Signals M[α-]:
Governance by Targets & Terror
New approach to health care governance in 2000s
Decisive breakthrough in governance – or repeat of history of Soviet Union? Some evidence from the English NHS
Conclusion
Development of star ratings in England
2001 2002from2003
Acute (156)Specialist (20)
Ambulances (31) Mental Health (88)
PCTs (304)
Evidence of impact of key targets
Ambulances 75% category A
calls < 8 minutesHospitals total time in A&E
< 4 hours waiting times for
elective inpatient admission
Reported successes
Problems of measurement & gaming
Reported success: 75%category A calls<8 minutes
0
25
50
75
100
20000
25
50
75
100
2003
Before After
Problems of measurement & gaming: third ‘corrected’ response times
75% < 8 minutes
Source: http://www.chi.nhs.uk/eng/cgr/ambulance/index.shtml
‘Corrections’’ only 2% to 6%
75% < 8 minutes
Impact: A&E total time < 4 hours
60
80
100
Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1
2002-03 2003-04 2004-05
& a 20% increase in numbers in A&E
Source: http://www.nao.org.uk Improving Emergency Care in England
Problems of measurement & gaming in A&E Problems of measurement
Government figures: mid-2004, target met by 96% of patients
Healthcare Commission survey (55,000 patients): 77% of patients stayed < four hours in A&E
Problems of gaming Queues of ambulances outside Moving staff & cancelling operations over
period of measurement
Impact of key targets on hospital waiting times for elective inpatient admission
0 3 6 9 12 15 18
2001
2002
2003
2004
Target waiting time (months)for elective inpatient admission
Impact: hospital waiting times elective admission
0
50
100
150
200
1997 1998 1999 2000 2001 2002 2003 2004
>12 (2003)>9 (2004)
Numbers waiting elective admissions (‘000s)
Star ratings published
Source: http://www.dh.gov.uk Chief Executive’s Report to the NHS – Statistical Supplement (2004)
Impact of key target: hospital waiting times elective admission
0
50
100
150
200
1997 1998 1999 2000 2001 2002 2003 2004
>12 (2003)>9 (2004)
Numbers waiting elective admissions (‘000s)
Star ratings published
Source: http://www.dh.gov.uk Chief Executive’s Report to the NHS – Statistical Supplement (2004)
Impact of key target: hospital waiting times elective admission
0
10
20
30
2000 2001 2002 2003
England
Source: http://www.statistics.gov.uk National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38
% patients waiting for hospital admission > 12 months
Impact of key target: hospital waiting times elective admission
0
10
20
30
2000 2001 2002 2003
EnglandScotland
Source: http://www.statistics.gov.uk National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38
% patients waiting for hospital admission > 12 months
Impact of key target: hospital waiting times elective admission
0
10
20
30
2000 2001 2002 2003
EnglandWalesScotland
Source: http://www.statistics.gov.uk National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38
% patients waiting for hospital admission > 12 months
Impact of key target: hospital waiting times elective admission
0
10
20
30
2000 2001 2002 2003
EnglandWalesScotlandNorthern Ireland
Source: http://www.statistics.gov.uk National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38
% patients waiting for hospital admission > 12 months
Natural experiment in UK countries England (from
2001): annual performance (star) rating Zero to ‘naming &
shaming’ Wales, Scotland,
Northern Ireland no ranking no incentives
90
95
100
105
110
115
England Scotland Wales NorthernIreland
1998-992002-03
Spend per capita on health care (UK = 100)
Source: www.hm-treasury.gov.uk/media//B4887/pesa04_chapter08_190404.pdf
Waiting times: problems of measurement & gaming
Problems of measurement Audit Commission: reporting errors at
least one PI in 19 trusts Problems of gaming
NAO: 9 NHS trusts inappropriately adjusted their waiting lists
Audit Commission: 3 cases of deliberate misreporting of waiting list informationSources:
www.nao.gov.uk/publications/nao_reports/01-02/0102452.pdfwww.audit-commission.gov.uk/health/index.asp?catId=english^HEALTH.
‘Synecdoche’ problems over quality of care 2 failures that resulted in major public
inquiries Bristol case of paediatric cardiac surgery Shipman
All difficult to detect from outside All could plausibly have sent ‘satisfactory’
M[α-] signals under star ratings regime And M[α-] ‘failures’ might nevertheless
provide excellent quality of care
Governance by Targets & Terror New approach to health care governance
in 2000s Decisive breakthrough in governance – or
repeat of history of Soviet Union? Conclusion
How far did English NHS satisfy assumptions? Repeating Soviet history: 1939? 1969? 1989? Policy implications?
How far did English NHS satisfy assumptions? Synecdoche: a part can meaningfully
stand for the whole?
What is measured is a good indicator of performance?
Gaming effects (the ‘knights-to-knaves’ problem) are either small or unimportant?
Repeating Soviet history: 1939? 1969? 1989? 1939: waiting times? 1969: ? 1989: balanced scorecard?
Policy implications: targets retained, limiting gaming? Limits on transparency Designing & resourcing systems of
validation Sources of information other than
data reported by those being monitored
Recommended