Measuring Health Systems Measuring Health Systems Performance and NHA: Performance and NHA:

Agenda for Health Services Agenda for Health Services Research and EvaluationResearch and Evaluation

Akiko Maeda, Ph.D.Akiko Maeda, Ph.D.Lead Health SpecialistLead Health Specialist

Europe and Central Asia RegionEurope and Central Asia RegionThe World BankThe World Bank

IntroductionIntroduction

NHA role in measuring health system NHA role in measuring health system performanceperformance

Challenges in measuring efficiency and Challenges in measuring efficiency and quality of health carequality of health care

Need for improved data collection and Need for improved data collection and evaluation toolsevaluation tools

Agenda for health services research & Agenda for health services research & evaluation & NHA in developing countriesevaluation & NHA in developing countries

Health Sector Resource Allocation & Health Sector Resource Allocation & Utilization QuestionsUtilization Questions

How large is the sector?How large is the sector? What are its constituent parts?What are its constituent parts? What items currently dominates in What items currently dominates in

resource allocation?resource allocation? Who pays for what, and what services Who pays for what, and what services

do they get for their payment? do they get for their payment?

Measuring Health Systems Measuring Health Systems PerformancePerformance

Revenues /Inputs

• Redistribution (progressive?)

• Administrative Efficiency

• Risk-pooling/ Insurance

Health Services Throughputs

• Allocative Efficiency

• Microecon. efficiency

• Efficacy/ Effectiveness

Health Outcomes

• Aggregate

• Disease specific

• Socio-economic factors

NHA defines financial flows from NHA defines financial flows from sources to service outputssources to service outputs

NHA Flow of Funds AnalysisNHA Flow of Funds Analysis

Uses of Funds:Uses of Funds:- By Institutions- By Institutions- By Functions- By Functions

Financial IntermediariesFinancial Intermediaries

ICHA-HC

ICHA-HP

ICHA-HF

SHA Coding

Sources of FundsSources of Funds

Why Health Accounts?Why Health Accounts?

Health accounts offerHealth accounts offer : : – Consistent and comprehensive definition of Consistent and comprehensive definition of

health system taxonomy and boundaries, for health system taxonomy and boundaries, for comparabilitycomparability

– Classification of resource Classification of resource inputsinputs and and throughputsthroughputs , by , by functionalfunctional and and organizationalorganizational categories categories

– A method for costing (valuation of) health A method for costing (valuation of) health “transactions” for comparability, normative “transactions” for comparability, normative evaluationevaluation

Measuring Health System Measuring Health System PerformancePerformance

Outcome/InputsOutcome/Inputs = Cost-benefit, cost = Cost-benefit, cost effectiveness analysis, e.g., reduction in disease effectiveness analysis, e.g., reduction in disease incidence rate per $$incidence rate per $$

Outputs/InputsOutputs/Inputs – Efficiency measure based on – Efficiency measure based on intermediate process indicators, e.g., cost per intermediate process indicators, e.g., cost per hospitalization casehospitalization case

Outcome/OutputsOutcome/Outputs = Effectiveness/ efficacy of = Effectiveness/ efficacy of intervention, e.g., immunization rate and intervention, e.g., immunization rate and reduction in disease incidence. reduction in disease incidence.

Challenges for improving the Challenges for improving the relevance of NHArelevance of NHA

NHA offers a consistent framework for NHA offers a consistent framework for measurement, but does not offer normative measurement, but does not offer normative measuresmeasures

Difficult to evaluate performance based only on Difficult to evaluate performance based only on aggregate expenditure dataaggregate expenditure data

Need complementary indicators of efficiency Need complementary indicators of efficiency and quality: appropriateness of care, and quality: appropriateness of care, productivity measures, and other benchmarksproductivity measures, and other benchmarks

Poor quality and incomplete data at micro-level Poor quality and incomplete data at micro-level (provider) limits usefulness of aggregate data(provider) limits usefulness of aggregate data

Measuring “Efficiency” of Measuring “Efficiency” of Health System: ChallengesHealth System: Challenges

Major gap between aggregate macro-level dataMajor gap between aggregate macro-level data

....And microeconomic performance data....And microeconomic performance data

at provider and population levelat provider and population level

GAPGAP

Challenges in Comparing Health Challenges in Comparing Health System PerformanceSystem Performance

Variability in the organization of health care Variability in the organization of health care delivery systemdelivery system

Dynamically evolving technology: e.g., Dynamically evolving technology: e.g., increasing use of day surgeries, decreasing acute increasing use of day surgeries, decreasing acute care hospitalization days care hospitalization days

Internal variability in performance among Internal variability in performance among providers and outcome among population groupsproviders and outcome among population groups

Confounding factors: complex interactions Confounding factors: complex interactions between socio-economic factors and health between socio-economic factors and health outcomesoutcomes

Example: Using NHA to evaluate Example: Using NHA to evaluate Estimating Allocative EfficiencyEstimating Allocative Efficiency

NHA functional categories: e.g., public NHA functional categories: e.g., public health programs, acute vs. chronic health programs, acute vs. chronic inpatient care provides common inpatient care provides common framework for defining products, but framework for defining products, but needs to be adjusted for:needs to be adjusted for:– Intensity, quality of care Intensity, quality of care – Population – demographic profilePopulation – demographic profile– Evidence of clinical efficacy, cost-Evidence of clinical efficacy, cost-

effectiveness effectiveness The World Bank

0%

20%

40%

60%

80%

100%

Czech Rep1999

Denmark1999

Germany1998

France1999

Italy 1997 Croatia2002

Comparison of Health Spending by Functions as % Total

Others (Health Administration, Public Health, etc)Therapeutic appliancesPharmaceuticals & medical consumablesOutpatient, Ancillary & Home HealthInpatient

What can we tell from these aggregate expenditure data...?

Developing data collection and research Developing data collection and research capacity in the following areas:capacity in the following areas:

– Household consumption and expenditure Household consumption and expenditure surveys, with improved designs on medical surveys, with improved designs on medical services purchased directly or through services purchased directly or through insuranceinsurance

– Health care provider utilization surveysHealth care provider utilization surveys

– Improved quality of expenditure data at Improved quality of expenditure data at provider level (case-mix, medical procedures)provider level (case-mix, medical procedures)

Agenda for ResearchAgenda for Research

Review of classification of services Review of classification of services

– Need to review, adapt OECD SHA functional Need to review, adapt OECD SHA functional and provider classifications to suit developing and provider classifications to suit developing country health systemscountry health systems

Health services research agenda:Health services research agenda:– Developing affordable instruments for Developing affordable instruments for

collecting data at provider (private & public) collecting data at provider (private & public) and population groupsand population groups

Agenda for NHA in Developing CountriesAgenda for NHA in Developing Countries

Examples of Priority Topics for Examples of Priority Topics for Health System ResearchHealth System Research

Rational Use of DrugsRational Use of Drugs Hospitalization – tracking changes in acute Hospitalization – tracking changes in acute

and chronic care admission rates, length of and chronic care admission rates, length of staystay

Administrative efficiencyAdministrative efficiency Public health programs – allocation on Public health programs – allocation on

prevention and population-based programs prevention and population-based programs versus personal/ clinical interventionsversus personal/ clinical interventions

Agenda for Health System Agenda for Health System ResearchResearch

Recommendations:Recommendations:– Focus greater attention on capacity building on Focus greater attention on capacity building on

health services research and evaluation health services research and evaluation – Introduce National Health Accounts within the Introduce National Health Accounts within the

context of health services evaluation, not as a context of health services evaluation, not as a stand-alone instrument stand-alone instrument

– With improved quality of data, it should With improved quality of data, it should become more feasible to undertake meaningful become more feasible to undertake meaningful national and international comparisons national and international comparisons

Agenda for Health System Agenda for Health System ResearchResearch

World Bank’s standard economic and sector World Bank’s standard economic and sector work focuses on Public Expenditure work focuses on Public Expenditure Reviews (PER) and Household Surveys Reviews (PER) and Household Surveys (e.g., Living Standards Measurement (e.g., Living Standards Measurement Surveys)Surveys)

Complementary capacity building should Complementary capacity building should focus on supporting provider surveys, focus on supporting provider surveys, operational research on servicesoperational research on services

Measuring Health System Measuring Health System Performance – InstrumentsPerformance – Instruments

Provider Surveys

Household Surveys

Administrative Data

Public Expenditure

Reviews

Health SystemsPerformance

EvaluationQuality & utilization reviews, technology

assessment, demo. & epid. analyses

Health System Performance Health System Performance Evaluation - Evaluation -

Low-income countriesLow-income countries– Administrative data may be of limited valueAdministrative data may be of limited value– Limited data on private sector activitiesLimited data on private sector activities– May need additional surveys to evaluate May need additional surveys to evaluate

performanceperformance Middle Income CountriesMiddle Income Countries

– Improved capacity to collect performance-Improved capacity to collect performance-based data through regular operational and based data through regular operational and regulatory systemsregulatory systems

Health System Evaluation Process Health System Evaluation Process

Utilization and Quality Reviews

Analysis of Outputs, outcomesand Expenditures

Provider levelPopulation

groups

Clinical and epidemiological

outcomes

NHA frameworkon resource flows

Evi

denc

e-B

a se d

Me d

icin

e

Ser

v ic e

ben

c hm

a rks

Co s

t -e f

f ect

iven

e ss

An a

lys i

s

Contact: Akiko MaedaAddress: The World Bank H7-700

1818 H. St. N.W., Washington, D.C. 20433

USATelephone: +1 202 473 3793Email: amaeda@worldbank.org