Evaluation of NSRF Health Care Development Projects Norbert Kiss Hétfa Research Institute – Corvinus University of Budapest Balázs Váradi Budapest Institute

Evaluation of NSRF Health Care Development Projects

Norbert KissHétfa Research Institute – Corvinus University of Budapest

Balázs VáradiBudapest Institute – Eötvös Loránd University

30 April 2013

Evaluation of NSRF Health Care Development Projects 2

Contents

• General overview and evaluation of health care developments

• Geographic targeting of health care development projects

• Evaluation of developments of outpatient care providers

• Modernisation of the central medical equipment register

• Evaluation of lifestyle development programmes

3

Overview of health care development

• Methodology:– Inclusion of health care constructions (from SMIS)– Health care providers as beneficiaries (from SMIS)– Assignment of current funding of the National Health Insurance

Fund (OEP) to levels of health care

• Main conclusions:– Large share of health care constructions in total funding– Smaller amounts from ROPs (accessibility for the disabled,

medical tourism), EEOP, EAOP (e.g. e-health)– EDOP, ROPs, SROP (lifestyle development): private providers

were also eligible (app. HUF 11.5 billion)– Connections between SIOP and ROPs (statistical regions and

„health care planning and management areas”)– Public providers were typically supported from one source only– HR developments (HUF 8,5 billion): only a fraction of the wages

in the health care sector, longer term impacts can only be achived at very specific areas


4

Regular OEP funding (2011) Development programs

Funding (billion HUF)

% Funding (billion HUF)

%

Primary129.1 17.1% 15.0 4.5%

Outpatient143.2 19.0% 93.5 27.5%

Inpatient446.3 59.3% 199.9 59.1%

Ambulance25.3 3.4% 15.3 4.5%

Blood supply8.2 1.1% 3.2 1.0%

Health promotion0.5 0.1% 11.4 3.4%

Total752.7 100.0% 336.3 100.0%

Targeting of the development programs (by progressivity – levels of health care)


5

Evaluation of geographical targeting

• Methodology:– Health care development projects (from SMIS database)

• Assignment to progressivity levels and to the population of microregions (LAU1) (at institutional level)

– Current („regular”) OEP funding• Assignment to progressivity levels • Assignment to microregions (by using OEP data)

– Avoidable mortality by microregions (J. Vitrai & M. Bakacs)

• Assessment of the targeting of regular and development funds in terms of avoidable mortality and the complex developmental indicator, using Lorenz curves– What percentage of the sources were allocated to the

microregions with greatest needs


6

Avoidable mortality by microregion (J. Vitrai & M. Bakacs)


7

Development projects are better targeted than regular funding

0.1

.2.3

.4.5

.6.7

.8.9

1S

our

ces

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level

Regular funding(2011) Programs(2008-12)


8

Primary care projects are the best targeted

0.1

.2.3

.4.5

.6.7

.8.9

1S

our

ces

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level

Primary regular funding(2011) Primary programs(2008-12)


9

However, outpatient care targeting is not bad either

0.1

.2.3

.4.5

.6.7

.8.9

1S

our

ces

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level

Outpatient funding(2011) Outpatient programs(2008-12)


10

Targeting of inpatient care development projects

0.1

.2.3

.4.5

.6.7

.8.9

1S

our

ces

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health level

Inpatient regular funding(2011) Inpatient programs(2008-12)


11

Geographical targeting of development projects: quantitative indicators

Overall funding Outpatient care Inpatient care Primary care

How much was allocated to the least healthy half of the population? (in % of the total)

Development projects 60.1 74.4 56.5 79.2

Regular funding 48.1 43.9 49.9 46.7

Ratio of sources allocated to the worst and best deciles (in terms of health status)

Development projects 0.62 0.38 0.73 0.18

Regular funding 1.04 1.29 0.94 1.14


12

Positioning of the outpatient care development projects

Did the development projects meet the local needs?• Methodology: document analysis, statistical analysis • Key findings:

– Feasibility studies for the tender used different methodologies.

– Short run-up period– Accessibility in terms of time improved:

• App. 310,000 people got access to basic level outpatient care in 20 minutes

• App. 150,000 people got access to advanced level outpatient care in 25 minutes (4 basic specialities + cardiology, traumatology)

• Results are similar when accessibility is defined by either car or bus

• Bus timetables are not in harmonised with office hours

– Low scale of economy, modest capacity utilisationEvaluation of NSRF Health Care Development Projects

13

Improvement in the accessability of basic level outpatient care

No access to basic level care in 20 min

Access to basic level care in 20 min

Place of development (no access to basic level care in 20 min)

Place of development (access to basic level care in 20 min)

Development created access to basic level care in 20 min

Basic level access obtained by other reason


14

Changes in the utilisation of services

How have the projects affected the utilisation of health care services?• Methodology: comparing the number of cases between the

population of the developed microregions (= „treated microregions”) vs. the population of control regions– Matching methods, combined with difference in differences– Panel data analysis with fixed effect estimation

• Key findings (SIOP 2.1.2):– Solid increase (25-30%) of utilisation in almost each

medical specialities regardless of the estimation methodology

– Rise in utilisation is 36-43% among people above 60, while 13-23% among younger people

– Standardized number of cases has increased to the level of similar regions


15

Short term health impacts

Has the improved access to outpatient care improved the care of chronic patients (e.g. identifying chronic conditions, utilisation of necessary tests and consultations)?Have the additional outpatient care capacities contributed to the reduction of sick leave days?

• Methodology: statistical analysis• Key findings:

– Sick leave days and number of cases dropped by 4-6% (SIOP 2.1.2-3)

– Annual eye screening of diabetic patients: 8% increase (no significant effects on other types of screenings and laboratory tests)


16

Economic sustainability

Are the developed outpatient care centers in microregions sustainable on the long term?

• Methodology: case studies, expert interviews• Key findings:

– The original business plans and sustainability calculations became practically outdated at the time of contracting

– No common responsibility of the microregion for the sustainment of the center in several cases

– Problems with additional revenue creating services – The utilisation of the physical infrastructure cannot be

efficient due to the low scale of economy– It is a consequence of the financing system that these

centers cannot break even – or only with great difficulties – Aspects of equity


17

Modernisation of the central medical equipment register: the IT system

• Adequate hardware and software environment for the central system

• Data collection methodology uses up-to-date technology• Automatic uploading + web-based interface• Client side interface development was also initiated, 2

major IT system suppliers implemented it

• Preliminary interviews with data suppliers and users • The content of reporting has been expanded

(compulsory+optional)• Information gathering processes of data suppliers have

not been modelled and analysed• Problems with the nomenclature and master data


18

Modernisation of the medical equipment register: Efficiency and effectiveness

• „Accurate” reporting practices would lead to increase of administrative burden

• Thus few providers thrive to report „accordingly”• No legal consequences, and there is a conflict of interests

• Data migration between the old and new systems• Validation with sampling

• No change in the number of providers with accurate reports (app. 10%)

• Limited number of users of the database

• Recommendation:– Analysing data gathering processes of providers– Connection to the licensing of providers


Lifestyle development programmes in 2009: overview of applications

• App. 250.000 participants (SMIS-monitoring)

Source: SMIS-reporting (10th Jan 2013)

19

Calls for applicationsNo.

applications received

No. projects selected

Grant awarded (HUF)No. of

contractsValue of contracts

(HUF)Payments made (HUF)

SROP - 6.1.2./A-09/1 1406 170 1 554 559 581 HUF 170 1 554 559 581 HUF 1 436 364 923 HUF SROP - 6.1.2./A-09/1-CHR 689 135 1 252 546 799 HUF 135 1 252 546 799 HUF 1 068 866 420 HUF SROP - 6.1.2./LAMR-09/1 50 14 764 442 802 HUF 14 764 442 802 HUF 638 503 871 HUF SROP - 6.1.2./LAMR-09/2 13 1 19 645 825 HUF 1 19 645 825 HUF 12 954 979 HUF

Primary scenes of activitiesSROP -

6.1.2./A-09/1SROP - 6.1.2./A-

09/1-CHR

Micro-region/municipality 12% 15%

Education institutions 35% 24%

Workplaces 34% 46%

Target group specific scenes 12% 6%

Other 7% 9%


Geographic targeting of the lifestyle development projects

20

0.1

.2.3

.4.5

.6.7

.8.9

1C

umu

lativ

e va

lue

of fu

ndi

ng

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1Health status

SROP 6.1.2. Allocation of funds


Adequateness of content and impact of lifestyle development projects

• Motivation for individual and/or community level lifestyle changes were in the focus

• Capacity building, changes in the local environment to promote healthy lifestyle were given lower priority

• Difficulties with identifying and following the real professional content of the projects – assessing whether the content has met the needs is also difficult

• Quality assurance of the professional content was missing from the selection procedure (except for LAMR applications)

• Non-LAMR health plans were often only formally adequate

• Recommendations and resources of the (central) research and methodological development institutions were rarely used

• „Copy-paste” type content was typical in communication materials

• Long lasting impacts are not achievable (according to expert opinions)

• Low value of grants – high administrative burden

21Evaluation of NSRF Health Care Development Projects

Contribution to improvement of local conditions

• Not clearly demostrated how the health plans are connected to the local needs and demands

• Health plans, prepared during the projects, have not become a central part of local health development planning

• Methodological support was provided for LAMRs • Inclusion of partners: mostly from the social sector• Partnership agreements were typically made for the

duration of the projects• Occasionally „simple and inexpensive” solutions were less

preferred („this is the time when we have the money…”)• Equal opportunity appeared as a horizontal objective • The practice of „one size fits all” was strong

• A large part of the experiences have been incorporated into the newer calls in 2011-2012

22Evaluation of NSRF Health Care Development Projects

Thank you for your attention!

23

Documents

Evaluation of NSRF Health Care Development Projects Norbert Kiss Hétfa Research Institute – Corvinus University of Budapest Balázs Váradi Budapest Institute