Identifying Problems—Outcomes for Judging System Performance

Marc J RobertsProfessor of Political Economy and Health Policy

Harvard School of Public HealthNairobi, Kenya 11 April, 2011

2

How to Begin• Pharmaceutical systems are both complicated and plagued

by numerous difficulties

• Setting a sensible reform agenda means starting by figuring out which outcomes it is most important to improve

• We call this: identifying the problem

• Identifying which causes to try to fix requires a “diagnostic journey” after important performance problems have been identified

• Fixing the causes of poor performance is a way to get better outcomes. It is not as an end in itself

• This is not how most of us think—we identify problems in terms of processes and possible causes

3

Flow Diagram for Pharmaceutical System Outcomes

Inputs

Financing Financial Protection

System/Processes Satisfaction

Outputs Health

Inputs to thePharmaceutical System

Personnel BuildingsEquipment FurniturePharmaceuticals VehiclesSupplies FuelWorking Capital

Components of the Pharmaceutical System

Hospitals (Public/Private) Central StoresClinics (Public/Private) District StoresInternational producers Ministry of FinanceDomestic producers InsurersImporters/Wholesalers Procurement AgenciesRetail drug sellers RegulatorsLaw enforcement

Pharmaceutical System Processes

Registration and Quality TestingManufacturingTracking Inventories and UseForecasting Needs (quantification)Tender/Procurement ProcessesSupply ChainDiagnosis and Prescribing—Inpatient and OutpatientDispensing to and Purchasing by PatientsPatient UtilizationInspection/Enforcement

Pharmaceutical System Financing

• Out of pocket payment• General tax revenue (e.g. VAT)• Social insurance• Community funds• Private insurance• Donor assistance (bi-lateral and multi-lateral)

Outputs of the Pharmaceutical System

Medicines Actually Taken by PatientsWhat is needed for good system

performance?– The right medicines– Taken in the right way– At the right time– By the right individuals

Outcomes of the Pharmaceutical System

• Health Status of the Population

• Citizen Satisfaction with the Pharmaceutical System

• Financial Protection

10

Why These Outcomes?

• Important ethically and philosophically

• Important politically

• Influenced by the health care system

Using these requires us to specify practical measures in more detail

11

How to Measure Outcomes

• Various health measures: DALYs, life expectancy, disease rates, IMR, MMR– Better measures reflect both morbidity and mortality

– Better measures cost more to collect and may be less accurate

• Measuring satisfaction or risk protection requires household survey data

• Beware reporting difficulties

• Measures are not neutral – they reflect values and priorities

12

Citizen Satisfaction

• Economists typically emphasize this goal

• Source of much political concern

• Often depends on non-clinical aspects--availability, service, prices etc.

13

Financial Protection• Two components: routine medicines costs and the

costs of major illnesses (risk protection)

• The impact of any cost depends on household income

• Impact may occur through decreased use

• Lack of risk protection contributes to impoverishment among the poor

14

Cost Is Always Part of the Problem

• No nation has “enough” resources• Cost represents the burden the health sector places

on a country’s resources– Cost can provoke reform – when the burden is too high– Short run cost constraints may be changeable in the long

run: economic growth or external support

• More money can be helpful – but it may not be either necessary or sufficient

15

Cost

Performance

•

• 6

2

74

3

5

8

1

9

B

A

Cost-Performance Trade-Offs

16

Reasonable Cost-Performance Choices at Point A

The system is not efficient

1. Lower cost, even if performance suffers

2. Minimize cost for current performance

3. Improve performance and lower cost

4. Maximize performance at current cost

5. Better performance, even if costs rise

17

Other Cost-performance Choices

The system is efficient—it is at Point B6. Higher cost and higher performance 7. Lower cost and lower performance

Irrational moves no matter where you are8. Higher cost, same performance9. Higher cost, lower performance

18

Variations in Attitudes Toward Cost

• The Ministry of Health typically argues the nation is at A and urges move #6– They don’t want to admit inefficiency exists– They want more money

• The Ministry of Finance generally claims the nation is at B and urges move #3– They don’t believe that the system is efficient – They want to save money

19

What Costs Matter?

• Governments look at budget costs

• Economists include all costs—private and public, monetary and non-monetary

• Patient costs for acquiring medicines even in “free” public systems can be substantial: transportation, lost time, lost income, bribes etc.

20

How to Choose Outcome Priorities

• Those illnesses that contribute to the burden of disease

• Availability of cost effective treatments

• Groups that do poorly

• Political sensitivity

• Organizational feasibility: “low-hanging fruit”

• Benchmarking: regional comparisons, international norms

21

Average Performance May Conceal Big Variations

• Some population groups may do significantly worse than average—on health, satisfaction or financial protection– Rural residents – Certain regions– Urban poor– Refugees

• Upper income residents often can and do finance their own medicines consumption

• Policy makers have to decide which of these variations are important problems for reform to address

22

Improving the Average Versus Helping the Worst Off

• Sometimes helping the bottom is cost-effective– Primary care works well– Basic medicines are cost-effective

• Sometimes helping the bottom is especially expensive– Isolated locations cost more to serve– Social circumstances lead to resistance to change

The largest non-cost-effective in the health system is usually at the top not the bottom. Such unequal spending lowers overall performance.

The Role of Non-Health Goalsin Problem Definition

• Pharmaceutical policy is often asked to serve non-health goals:– Economic development (protecting local manufacturers)– Raising revenue (import fees and sales taxes)– Responding to regional pressures (decentralization)– Responding to the government’s political concerns

• These added goals greatly complicate the ethical and political analysis of reform options

• Pharmaceutical policy makers may face difficult choices between being loyal to their political leaders and acting as sector advocates

International Pressures and National Priorities

• The medicines sector offers international donors the possibility of tangible, significant accomplishments

• Those donors have their own goals and face their own pressures

• Their objectives may not perfectly match local priorities or views

• Aid-dependent governments may feel a need to set poriorities in ways that respond to donor views

• Donor pressures can be useful as a tool for counteracting local stakeholders

25

The Role of Values

• Picking a specific index to measure performance involves both technical issues and values

• Deciding what equity aspects of the distribution of performance matter involves values

• Deciding how to trade off among different (health and non-health) performance goals involves values

The Importance of Problem Identification

• What “problems” reformers identify as priorities to solve has a major impact on all that follows

• Problems should be identified by asking where the system performs poorly in terms of ultimate outcomes: health, satisfaction, financial risk protection

• Once key performance problems have been identified, a “diagnostic journey” can be undertaken to identify the relevant causes