HPT MODULE 7 V.2:Layout 1A priority is a “status established in order of importance or urgency”.1 Priority setting (also called prioritization) is the establishment of an order

Information Management

A System We Can Count On

The Health Planner’s ToolkitHealth System Intelligence Project – 2008

Priority Setting

MODULE

7

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iii

This Module’s Purpose . . . . . . . . . . . . . . . . . . . . . . . . .iv

Section 1

What is Priority Setting? . . . . . . . . . . . . . . . . . . . . . . . .11.1 Why is Priority Setting Necessary? . . . . . . . . . . . .11.2 Key Elements in Priority Setting . . . . . . . . . . . . . .31.3 The Field of Activity for Priority Setting . . . . . . .41.4 Who is Responsible for What Kind of

Priority Setting? . . . . . . . . . . . . . . . . . . . . . . . . . . . .51.4.1 Macro Level Priority Setting . . . . . . . . . . . . . . . . . .51.4.2 Meso Level Priority Setting . . . . . . . . . . . . . . . . . . .51.4.3 Micro Level Priority Setting . . . . . . . . . . . . . . . . . .61.5 Vertical, Horizontal and Population-Based

Priority Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71.6 Priority Setting: The People Part and the

Numbers Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Section 2

The People Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112.1 Culture, Values and Setting Priorities . . . . . . . . .112.1.1 Some Key Values in Priority Setting . . . . . . . . . .122.1.2 Deserving and Undeserving Recipients . . . . . . .152.1.3 Fair Priority Setting Processes:

Accountability for Reasonableness . . . . . . . . . . .152.1.4 A Framework for Ethical Priority Setting . . . . .172.1.5 Unarticulated Assumptions . . . . . . . . . . . . . . . . .172.2 Group Process in Priority Setting . . . . . . . . . . . .182.2.1 People Who Manage the Priority

Setting Process . . . . . . . . . . . . . . . . . . . . . . . . . . . .182.2.2 People Who Set the Priorities . . . . . . . . . . . . . . .18

Section 3

The Numbers Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213.1 The Steps That Numbers Serve . . . . . . . . . . . . . .213.1.1 Identifying the Set of Items to be Prioritized . . .213.1.2 Identifying Criteria for Priority Setting . . . . . . . .213.1.3 All Criteria are Not Created Equal:

Assigning Weights to the Criteria . . . . . . . . . . . . .253.1.4 Indicators That Provide Information

on Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .263.1.5 Gathering Data on the Indicators . . . . . . . . . . . .263.2 Program Budgeting and Marginal Analysis

(PBMA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27

Section 4

Organizing and Carrying Out a Priority Setting

Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .314.1 Establish a Mandate for Priority Setting . . . . . . .314.2 Establish the Process Management Group . . . . .314.3 Develop the Project Plan . . . . . . . . . . . . . . . . . . .314.4 Develop the Priority Setting Group . . . . . . . . . . .324.5 Establish an Accountability for

Reasonableness Strategy . . . . . . . . . . . . . . . . . . . .324.6 Develop an Appeals Process . . . . . . . . . . . . . . . .324.7 Establish and Implement a

Communications Plan . . . . . . . . . . . . . . . . . . . . . .334.8 Understand the Environments for

Priority Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . .334.9 Identify Criteria, Weights and Indicators . . . . . .344.10 Identify Items for Inclusion in the Set to

be Prioritized . . . . . . . . . . . . . . . . . . . . . . . . . . . . .354.11 Gather Indicator-Based Information on

Each Item . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .354.12 Develop or Adopt a Scoring System . . . . . . . . . .364.13 Apply the Criteria to the Items in the Set

Using the Scoring System . . . . . . . . . . . . . . . . . . .364.14 Collate and Analyze the Scores and Present

the Provisional Results . . . . . . . . . . . . . . . . . . . . .364.15 Discuss, Negotiate and Adjust the List

if Necessary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .364.16 Distribute the Priority List to Stakeholders

and Invite Appeals . . . . . . . . . . . . . . . . . . . . . . . . .374.17 Accept and Review Appeals and Make

Adjustments if Necessary . . . . . . . . . . . . . . . . . . .374.18 Hand Off the List of Priorities to the

Implementers . . . . . . . . . . . . . . . . . . . . . . . . . . . . .384.19 Evaluate the Priority Setting Project . . . . . . . . . .38

Section 5

Challenges and Facilitators for Priority Setting . .39

Section 6

A Few Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Section 7

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Appendix A

Using Matched Pairs to Determine the Weight

of Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47

Page i

Table of Contents

Health System Intelligence Project (HSIP)

The Health Planning Toolkit is produced by the HealthSystem Intelligence Project. HSIP consists of a team ofhealth system experts retained by the Ministry of Healthand Long-Term Care’s Health Results Team forInformation Management (HRT-IM) to provide the LocalHealth Integration Networks (LHINs) with:

• sophisticated data analysis;• interpretation of results;• orientation of new staff to health system data

analysis issues; and • training on new techniques and technologies

pertaining to health system analysis and planning.

The Health Results Team for Information Managementcreated the Health System Intelligence Project tocomplement and augment the existing analytical andplanning capacity within the Ministry of Health andLong-Term Care. The project team is working in concertwith Ministry analysts to ensure that LHINs areprovided with analytic supports they need for their localhealth system planning activities.

Report Authors

Sten Ardal

John Butler (Module 7 Lead Author)

Richard Edwards

Lynne Lawrie

Page ii

About HSIP

Rinalda is an integration consultant on the staff of aLocal Health Integration Network (LHIN) in Ontario.She has been asked to survey the community todetermine its readiness for more disciplined approachesto priority setting for health services.

Rinalda finds a broad range of opinion in thecommunity. Some people believe priority setting isunnecessary or should be left only to physicians. Othersbelieve it should involve vast engagement of citizens todrive all aspects of priority setting. Some people believethat setting priorities should be driven by dispassionateeconomic and epidemiological numbers and experts.Others believe it should only be driven by the hearts,the values and the folk wisdom of the people who livein the community.

Some people believe in prioritizing all health servicesfor the entire community all at once. Others believe it

should involve setting priorities program-by-program oragency-by-agency, yielding a multiplicity of priority lists.

Some believe priority setting should apply only todetermine how new money should be spent. Othersthink it should also be used to determine whatprograms could be cut or reduced.

In short, Rinalda concludes that the community needseducational and discussion sessions to raise awarenessof the concepts, possibilities, drawbacks and variationsof priority setting, backed up by written resources asaids for learning and debate.

This module is meant to be a modest contribution tothese written resources.

Introduction Page iii

Introduction

This module will not turn the reader into an expert inpriority setting. It will provide basic information aboutpriority setting so the reader can grasp its essentialconcepts, activities and challenges. It will help thereader to get the most from priority setting and to knowwhen and how to use it and to balance the people partof priority setting with the numbers part.

Page iv This Module’s Purpose

The Module’s Purpose

A priority is a “status established in order of importanceor urgency”.1 Priority setting (also called prioritization)is the establishment of an order of importance orurgency among a set of items.

Within the health system, the kinds of items that havebeen prioritized can vary greatly. Broad health strategiesfor the entire world (World Health Organizationpriorities for example)3 or for an entire nation(Australia’s seven health priority areas for instance)4

can be prioritized. At the other end of the range,individual treatments for individual clients can beprioritized by health workers or by clients themselves.

Two characteristics of priority setting are paramountand are woven throughout this module:

1. Priority setting involves establishing the priority ofan item in relation to other items. An item may behigher, lower or the same priority as other items –but the very nature of prioritization suggests thatitems must be compared with others so a priority canbe established for each item.

2. Priority setting is one step – but not the final step –in a decision-making process. In and of itself a list ofpriorities is of little value. Its utility lies in the extentto which action is taken based on the list. That actionmost often involves maintenance, enhancement,reduction or removal of resources on the basis of thepositions of items on the prioritized list.

1.1 Why is Priority Setting Necessary?

In a world in which the supply of resources is sufficientto meet health service demand, prioritization isunnecessary. But no such world exists – or at least notany world that bases its health system on the public purseas a major supply source for meeting health demand.

In publicly funded health systems, the first activityundertaken to deal with the demand-supply mismatch isneeds assessment. This activity attempts to separatedemand for which there is insufficient justifiable needfrom demand for which there is sufficient justifiable need.

Needs assessment is itself a form of priority settingbecause it implicitly or explicitly assigns a greaterdegree of importance to demand for which there issufficient need, from demand for which there isinsufficient need. Module 2 in the Health Planner’sToolkit (Assessing Need)i provides extensive materialon needs assessment and should be read in conjunctionwith the current module on setting priorities.

Needs assessment alone however, does not solve theproblem of how to apply finite resources to what may

Section 1: What is Priority Setting? Page 1

Section 1

What is Priority Setting?

“Action expresses priorities.” – Mohandas Gandhi

“The word prioritization means ‘put before’. Thismeans then, a choice between two or morealternatives. In practice prioritization means thatsomething that seems more important is put beforesomething that seems less important.”

– Horizontal Political Prioritization: from words toaction. Sub-report January 2004, County Council of

Östergötland, Sweden2

“Key features of the public model inevitablyconstrain some of the freedom found in the freemarket model: sellers cannot sell everything theywant within the public arena, nor can buyers buyeverything they want. Needs assessment, then, is atool for helping ensure that people are free toprovide and receive the most crucial services, andfor helping the public sector to decide what is mostimportant to fund.”

– Health Planner’s Toolkit Module 2, Assessing Need

i The Health Planner’s Toolkit Module 2 – Assessing Need. [Online]. 2006 [cited 2008 Mar]; Available from: URL: http://www.health.gov.on.ca/transformation/providers/information/resources/health_planner/module_2.pdf

be a group of socially legitimized needs that, if fullymet, would exceed the resource supply. Taken together,competent needs assessment and priority setting canmove a health system from the supply-demandconundrum to a pragmatic position of optimally meetingneeds within available supply.

In short, setting priorities is not primarily aboutseparating good initiatives from bad ones. It is aboutestablishing the degree of importance or urgency ofitems within a set of items, all of which may be good

or desirable.

Page 2 Section 1: What is Priority Setting?

Figure 1: Moving to Pragmatism

LIMITED

SUPPLY

VIRTUALLY

UNLIMITED

DEMAND

needs assessment leads to

SOCIALLY

LEGITIMIZED

NEEDS

priority setting leads to

OPTIMALLY MEETING

NEEDS WITHIN

AVAILABLE SUPPLY

and

and

LIMITED

SUPPLY

LIMITED

SUPPLY

1.2 Key Elements in Priority Setting

Any priority setting process involves ten components:

1. A purpose for setting the priorities. This usuallyexpresses how the resulting priority list will be used.

2. A sponsor/manager for the process. Thesponsor/manager may be a single clinician whodecides to set treatment priorities for a single client,or it may be as large as an entire politicaljurisdiction (a national government for instance).

3. A set of items to be prioritized. These could beindividual treatments, programs, or broad strategiesfor an entire population.

4. A person or set of people who conduct the

prioritiza tion. An individual, a small group or amuch larger population can be involved in settingthe priorities.

5. A set of criteria to be used in setting the priorities.The set can be as small as a single criterion or mayinclude hundreds of criteria.

6. A weight for each criterion which reflects therelative importance of that criterion. All criteria maybe considered equally important, or some may beconsidered more important than others.

7. Data or other forms of information that can beused as evidence to understand how well an itemdoes, related to each criterion.

8. A rating scale to be used to reflect the importanceof each item on each criterion.

9. Application of information, criteria and rating

scale to each item on the list to gauge its priority.

10. A final priority list.

These elements, shown in figure 2, are described ingreater detail in Section Four of this module.


1

2

3

4

5

1 2 3 4 5

1

2

3

4

Ashleigh

Rohan

Olive

Wei-Lin

Carlos

Magda Percival

Morag Indira Josef

Jaswant Ehud

Rhiannon

Janusz

Koffi

Alisha

Ali

Ian

Feodor

set of items

to be

prioritized

people who

will set

priorities

weight

(importance)

of each criterion

sponsor/manager

of the priority

setting process

criteria

to be

used

information/database

for use in applying

each criterion

rating scale

to be used

applying

criteria to

items in the

set

the list of priorities

Rosa

purpose of

setting

priorities

SAMPLE ACTION: As a result of this

process, the fourth

priority may be

unfunded, while the top

three are funded.

PU

RP

OS

E

No

rth

So

uth

LH

IN

Figure 2: Elements in a Priority Setting Process

1.3 The Field of Activity for Priority

Setting

In Ontario and in other jurisdictions, priority settingwas most commonly done in the past when new fundingbecame available for health services – usually as an add-on to the existing pot of money for health care.Ontario’s Ministry of Health might, for instance, makenew money available for mental health and might carryout a formal or informal, explicit or implicit,prioritization to divide the new money among mentalhealth sub-sectors or among geographical parts of theprovince or both. District Health Councils in Ontariomight then have created priority lists for the fundsassigned to their sectors, providing their prioritized liststo the Minister of Health as advice.

What was generally off limits in these priority settingprocesses was the reallocation of money already withinthe system to underwrite growth and serviceimprovement.

But given the likelihood that money for the healthsector is at, or close to, its maximum as a percentage oftotal government expenditure, opportunities forreallocating existing funds to underwrite some of thechange and growth required in the system becomesmore enticing. However, reallocations must be madecarefully and with foresight – just as allocation of newfunds must be made with care.

Sound techniques used in priority setting can be usedboth to allocate new funds and to determine what existingfunds can be reallocated. One technique cited later inthis report – program budgeting and marginal analysis –is predicated specifically on finding defensible ways tosave and ways to apply those savings to high priorities.


Figure 3: Future Streams for Priority Setting and Resource Allocation

growth due to

cost escalation

(same amount

of same things

cost more)

growth due to

population

change

(e.g., more people

or older people)

growth due to

morbidity change

(emerging/

more severe

disorders)

growth due to

increased demand

(same people

want more)

growth in knowledge application

(new/better interventions

and technologies) or health system

scope creep (coverage of existing

procedures not previously covered)

NEW RESOURCES INTRODUCED INTO THE HEALTH SYSTEM

resources already applied

to health interventions

funding that can be reallocated

“Don't ever take a fence down until you know whyit was put up.”

– Robert Frost

1.4 Who is Responsible for What Kind of

Priority Setting?

The literature on prioritization often identifies threelevels of prioritization: macro, meso and micro.

1.4.1 Macro Level Priority Setting

Macro level priority setting occurs at international,national or provincial levels. While international prioritysetting takes place through organizations such as theWorld Health Organization, the effective levels of macrolevel prioritization are the levels of federal andprovincial governments because they hold levers ofpower that seldom exist at the international level.

In Canada the macro level is spread over these twojurisdictions. The federal government's role involvessetting and administering national principles for thehealth care system through the Canada Health Act

(CHA) and helping to fund provincial health careservices through fiscal transfers. The federalgovernment has other constitutional regulatory andpolicy responsibilities through which it influences theprovinces, but in practice the federal government doesnot act as a principal health care decision-maker.Nonetheless, the comprehensive definition of the CHAand the associated funding provisions give Ottawa amajor influence on what health care must be insured byprovincial governments, entrenching physician servicesand hospital services as “medically necessary” serviceswithin Medicare. This means that effective priority at theprovincial level has been given to medical and hospitalservices despite a growing emphasis on community-based care. On the other hand, the existence of theCHA has resulted in priority given to publicly fundedservices, ensuring significant equity in access byprohibiting private insurance, user fees or extra-billingfor these medically necessary services.6, 7, 8, 9

Most macro choices are made at the provincial level. Aprovincial macro choice that garners much attention,for instance, is the proportion of provincial budgetallocated to health. Ontario’s total health spending isestimated at 46 per cent of total provincial programexpense in 2006–07,10 fuelling ongoing concern that thepriority given to health funding overwhelms other

government responsibilities, particularly pressing needsfor social services and education.11 This motivatesinterest in priority setting among all provincialprograms and within the health envelope itself.

1.4.2 Meso Level Priority Setting

Meso level prioritization occurs at the level of regionalhealth authorities created by Canada’s provinces. LHINsare Ontario’s version of regional health authorities,although the term “network” rather than “authority”reflects Ontario’s intention that LHINs will work aspartners with provider agencies to create change.

Some provinces have introduced upper, regional andlocal tiers of devolved governance. Others haveestablished only a single tier at either the district orregional level.12 While certain province-wide servicessuch as cardiology and nephrology may be fundeddirectly by the provinces while being administeredthrough regional health authorities, most health servicesexcluding physicians and out-of-hospital drugs fallwithin regional health authorities’ decision-making. Atthis broad intermediate level, regions generally areresponsible for identifying and meeting the health needsof their populations and have the specific mandate ofsetting priorities and allocating resources accordingly.


“Priority setting exists at all levels - at the broadestlevel the Government makes decisions regardingthe priority to be given to the health service relativeto other public sector services in its annualallocation of funds... At the other extreme,clinicians and other health professionals are makingpriority setting decisions when deciding whichpatients to treat first or whom to exclude fromtreatment… The degree of explicitness of decisionsand the amount of public involvement may varyconsiderably at each of these levels.”

– Konrad Obermann and Keith TolleyThe state of health care priority setting and public

participation, 20075

LHINs are accountable to Ontario’s Ministry of Healthand Long-Term Care for planning, funding andintegrating the local health system and haveresponsibility for service providers such as hospitals,community care access centres and long-term carehomes. Physician, dentistry, podiatry and optometrypractices are specifically excluded from the Act bywhich the LHINs were established, but most powers,duties and functions that previously resided with theMinister of Health for planning local health carepriorities and systems have devolved to LHINs. TheGovernment of Ontario continues to develop province-wide plans for health that set broad priorities and eachLHIN was required to develop an integrated healthservice plan for the local health system. Each LHINplan includes a vision, priorities and strategic directionsfor its local system. In developing these plans, LHINsengaged and involved the local community to helpidentify local needs and priorities.

While the LHIN is responsible for setting performancegoals, objectives, standards and targets for the localhealth system, the direct provision of services, theirquality and standards remain the responsibility of eachhealth service provider. These services are subject tomonitoring and review by the LHIN, throughaccountability agreements and possible compliancedirectives for example, in addition to budget allotments.These instruments provide a means by whichorganizational priorities can be aligned with the LHINplan.13, 14, 15

1.4.3 Micro Level Priority Setting

The organizational level sets the immediate context formicro- or clinical-level priority setting at which healthpractitioners, in conjunction with clients, determinewho will receive access to what interventions. At thislevel, choices are made about treatment prioritiesamong conditions, treatment approaches and the typesof clients who will receive which treatments.16, 17

There is considerable evidence of wide variations inclinical practice patterns that appear to be unrelated tovariations in need, suggesting room for improvement inchoices regarding clinical priorities. The evidence-based

medicine movement and the interest in clinicalguidelines stem from a desire to rationalize clinicaldecision-making, but priority setting at this level can bedifficult because it involves direct contact with peoplewho need service. Macro- and meso-level choices dealwith setting priorities in the abstract but at the micro-level the consequences of setting priorities are morepersonal. Health professionals must translate prioritychoices made at higher levels into the context in whichpeople present themselves. Given the many factors thatcan influence an individual’s need for interventions, thiscan be complex and emotionally draining. The diversityof micro-level individual need, and the continuousdevelopment of new services, are rationales usedagainst expressing health priorities through a packageof core health services. There will most always besomeone for whom a service not deemed to be “core” isin fact medically necessary and a health system that isrestricted to paying for certain core services will resultin inappropriate care as research progresses and astechnology changes.18 On the positive side however,micro-level priority setting allows health professionalsto make decisions on issues such as chronic diseasemanagement that produce demonstrable and observablebenefits for the people they serve.

If the key broad challenge in priority setting is to usescarce resources efficiently and appropriately,international experience suggests that influencing thechoices of clinicians on whether and how to serveindividual clients is the central issue to address.19

The macro, meso and micro levels do not operate inisolation. Each level is affected by dialogue with otherlevels. For instance, priorities at national and provinciallevels are informed by the work of regional heathauthorities and provider agencies, and both the macroand meso levels are informed by the perspectives offront-line health service providers who set prioritiesdaily at the micro level.

As well, the border areas between levels requirecooperation as well as division of responsibility. Forinstance, at the macro level Ontario’s Ministry of Healthand Long-Term Care determines the distribution of the


health budget among the geographical areas/populationscovered by the LHINs, while at the meso level eachLHIN determines the distribution of its health budgetamong its geographical areas/populations.

Even within any one level, dialogue must occur amongits components. For example:

• Macro level negotiation between the federalgovernment and provinces is essential.

• At the meso level in Ontario, dialogue must takeplace between LHINs and their health serviceprovider agencies. This is particularly important inOntario because most health service providersremain autonomous board-governed organizations –they are not departments or branches of the LHIN.

• At the micro level, dialogue among providers (oftenabout priorities for joint service to clients) anddialogue between providers and service users areessential.

The World Health Organization’s World Health Assemblyhas identified seven areas of critical choice in prioritysetting and resource allocation.20 Figure 4 relates thesecritical choices to the macro, meso and micro levelswithin Canadian and Ontario contexts.

1.5 Vertical, Horizontal and Population-

Based Priority Setting

A distinction is often drawn between vertical andhorizontal priority setting:

Vertical priority setting deals with setting prioritiesamong patient groups in the same disease category butwith diverse needs, taking into account individuallyfocused prevention, diagnostics, treatment andrehabilitation.

An example of this kind of priority setting is theNational guidelines for stroke care 2005: Support for

Priority Setting, issued by Sweden’s National Board ofHealth and Welfare.21 These guidelines set priorities forstroke care by prioritizing 102 combinations of disease


Figure 4: Critical Priority Setting Choices, and Levels of Priority Setting

MESO:

LHINs and health service

provider agencies

MICRO:

health service providers

within the mandates

of their agencies

MACRO:

Government of Canada/ Health

Canada, and/or Government of

Ontario/Ministry of Health

& Long-Term Care

pu

blic

, co

nsu

mer a

nd

pro

vid

er in

pu

t

World Health Assembly: critical choices for priority setting

and resource allocation:

1. Forms of treatment that should be available through publicfunding

2. Proportion of public revenues allocated to health

3. Distribution of the health budget between geographicalareas/population aggregations

4. Proportion of available funds spent on capital developmentrelative to operating costs

5. Allocation of resources to different levels of the healthservice and to specific disease control programs

6. Definition of who should be eligible to receive such treatment

7. Definition of the amount to be spent on individual patients

states and procedures used in stroke care. Each itemhas a rating on a scale from 1 to 10. Number 1 indicatesprocedures with highest priority and number 10indicates procedures that are lowest priority becausethey are of very little clinical use, or of little clinical usein relation to the cost of the disease states concerned.The guidelines also contain a “not to be done” list thatincludes:

• procedures that should not be taken at all (becausethey are unacceptable in terms of patient safetycompared with some other measure for the samedisease state, or entail increased risk of undesirableeffects); and

• procedures that should not be taken as a matter ofroutine (procedures for which scientific support isincomplete and for which it cannot be claimed thatthe procedure is being taken in accordance withscientific knowledge and proven experience). Thisheading also includes diagnostic measures thatseldom affect case management and that thereforelack any routine clinical indication.

The guidelines also contain an “R&D” list of proceduresthat should only be carried out within the framework ofclinical trials or as research and development projects.

Responsibility for vertical priority setting has in the pastbeen mainly at the clinical level – although as theSwedish example shows, central authorities at themacro or meso level are increasingly taking a role inconjunction with clinicians in vertical priority setting.

Horizontal priority setting refers to setting prioritiesamong different areas of practice or among differentdisease groups. These choices generally do not focus onindividual service users and deal with the allocation ofresources among services. Responsibility for horizontalpriority setting is mainly at the macro or meso levels.22

An example of horizontal priority setting is the State ofOregon’s experience in prioritizing procedures across alldisease groups to arrive at a set of procedures for whichthe state would pay via Medicare and those for which itwould not pay.

In addition to the traditional division between verticaland horizontal priority setting, a third form has emerged– priority setting on the basis of specific

population groups, defined not by common healthproblems experienced by group members, but byeconomic, social, ethnocultural, age, gender or othercommon characteristics that are determinants of health.

As Figure 5 suggests, setting priorities can take place ina number of ways even within the broad framework ofvertical, horizontal and population based prioritization.For instance, within cancer care a prioritization processcan take place that looks only at options for cancerrehabilitation – or a priority setting process can meldtwo of the approaches – for instance, by settingpriorities for cancer rehabilitation for an elderlypopulation.



VERTICAL

PRIORITY SETTING

e.g., CANCER

SERVICES

e.g., POPULATION

GROUPS

prevention children

and/or and/or

primary care seniors

and/or and/or

secondary care women

and/or and/or

tertiary care ethnocultural groups

and/or and/or

rehabilitation rural groups

and/or and/or

long-term care people with disabilities

and/or and/or

end-of-life care low economic status

cancer

services

mental health

services

diabetes

services

trauma

services

stroke

services

prevention prevention prevention prevention prevention

and/or

primary care primary care primary care primary care primary care

and/or

secondary care secondary care secondary care secondary care secondary care

and/or

tertiary care tertiary care tertiary care tertiary care tertiary care

and/or

rehabilitation rehabilitation rehabilitation rehabilitation rehabilitation

and/or

long-term care long-term care long-term care long-term care long-term care

and/or

end-of-life care end-of-life care end-of-life care end-of-life care end-of-life care

HORIZONTAL

PRIORITY

SETTING

leads to

CANCER CARE

(VERTICAL)

PRIORITIES

leads to CROSS-SECTORAL (HORIZONTAL) PRIORITIES

POPULATION BASED

PRIORITY SETTING

leads to

POPULATION

PRIORITIES

Figure 5: Vertical, Horizontal and Population Based Priority Setting

1.6 Priority Setting: The People Part and

the Numbers Part

The next section of this report will explore the “peoplepart” of priority setting, with an emphasis on the valuesand behaviours that participants and stakeholders bringto priority setting. This is followed by a section thatdescribes the “numbers part” – the techniques andtechnologies (many of them rooted in economics) – thathelp make priority setting a rigorous and competentprocess.


“combining science with community sentiment”

– motto of the priority setting process of the IslandCounty Community Health Advisory Board

(Washington State)

Like planning as a whole, priority setting in a healthcare context is an ethical act, a social act and atechnical act. This section addresses priority setting asan ethical and social act – the cultural or “people part”of prioritization. It also addresses group process as partof priority setting.

2.1 Culture, Values and Setting Priorities

Setting priorities is an ethical act because it involves theapplication of values (things that are considered to begood or bad, without the need of empirical proof thatthey are good or bad).

In an action-directed endeavour like priority setting it isimportant to differentiate between two dimensions ofvalues:

1. espoused values; and2. values in action.

If people involved in setting priorities profess to holdcertain values but cannot turn them to practical uses, orif they act in ways that are at variance with their

espoused values, then the ethical legitimacy of thepriority setting process and its outcomes is at risk.

In addition to values, priority setting may also enter therealm of unarticulated assumptions – things that aregenerally unspoken but that are considered true or false(even if they do not have a moral association). Suchassumptions are “givens” – so embedded in our beliefsystem that it generally does not occur to us toarticulate them. Many unarticulated assumptions areneither helpful nor harmful. For instance, we assumethe sun will rise tomorrow. This assumption is soembedded in our belief system that we feel no need totalk about it, and it neither harms nor hinders us.

However, unarticulated assumptions can be dangerous.Once we realize they are flawed they can migrate fromthe status of simple beliefs to the status of prejudices,and to continue to hold these beliefs can cross the linefrom error into ethical failings.

Some largely discarded unarticulated assumptions – thebelief that malaria is caused by bad air for example –may not hold enough moral weight to become ethicalfailings if they are still held, but they may still bedangerous if they shape major societal decisionsthrough society’s priority setting processes.

It is worth keeping in mind that harm as well as goodcan be done through priority setting, and that valuesrequire careful consideration in setting priorities.

Section 2: The People Part Page 11

“Organizational culture can be defined as theattitudes, assumptions and values that are often notarticulated but influence and determine thebehaviors of an organization.”

– Edward Kim, The art of making change happen(managing change), 200323

“Differentiating between what might be called ‘espoused values’ (those formally described in missionstatements, for example) and ‘values in action’ (those that actually guide day-to-day behavior) is important. Anorganization that allows too great a gap to grow between the two kinds of values is likely to find itselfgenerating confusion – which will affect its culture in negative ways.”

– Diane L. Dixon, Leadership and Culture Alignment, 200024

Section 2

The People Part

2.1.1 Some Key Values in Priority Setting

Eight key values are common in priority setting:

1. serving the most-in-need;2. minimizing harm;3. health service effectiveness;4. broader social effectiveness;5. health service efficiency;6. health service equity (fair outcomes), which in turn

involves equity of access and equity of outcomes;7. deserving and undeserving recipients; and8. fair allocation processes.

This is not an exhaustive list but it illustrates commonlyfound, or particularly knotty, ethical issues.

Serving the most-in-need is a commonly held ethicaldistributive principle in health care. Defining andoperationalizing most-in-need is, however, a complex

but necessary process. Does it mean meeting the mosturgent needs first – dealing, for instance, with likelyimmediate death before dealing with likely death inthree months? Does it mean serving those at risk ofdeath before serving those at risk of misery but notdeath? Does it mean allocating everything (i.e., givingabsolute priority) to the most-in-need or does it meangiving priority to the most-in-need but also providing atleast basic service to those less in need? What relativeimportance should be given to various kinds of “worstpossible” subjective anguish on the part of people inneed – for instance, is worst possible physical pain moreimportant than worst possible depression? How canmost-in-need be reconciled with most-likely-to benefit?

Minimizing harm is an ethical issue for two reasons.First of all, most health services may cause harm aswell as good. An amputation, for instance, may save apatient’s life but it causes the harm of restrictingmobility for the remainder of the patient’s life. In thiscase the good likely exceeds the harm. Secondly, theopportunity cost of choosing one option over anothermeans that people who would have benefited from theoption not chosen are “harmed” in the sense that anopportunity for their health improvement is forgone.

The ethical issue is not avoiding harm at all costs. It liesin ensuring that harms are identified and evaluated, andthat options that do more harm than good are not chosen.

Health service effectiveness may not seem like avalue or ethical principle, but its roots lie in ethics andit wears several ethical faces – for example:

• It is unethical to provide a service known to beineffective if it leads people to false hopes that theservice will improve their lives.

• It is unethical to provide a service known to beineffective or of low effectiveness if the resourcesused for the service could produce greater socialgood (effectiveness) if applied to some other service.This involves the concept of opportunity cost,discussed later in this module.

• It is unethical to provide a service if its known harmexceeds its known effectiveness.

Page 12 Section 2: The People Part

“Resources should be allocated efficiently so as tomaximize the health benefits they produce. This isan ethical consideration, not merely an economicconcern, because promoting people's health andwell-being through efficient allocation is an ethicalgoal and allocating inefficiently results in theavoidable sacrifice of some patients' well-being.Second, the health benefits derived from limitedhealth care resources should be distributed fairly orequitably; we are not and should not be indifferentto how resources are distributed to distinctindividuals or groups... But while there is at leastgeneral agreement among health economists andpolicymakers about how to evaluate the efficiencyof different interventions in producing healthbenefits, there is no analogous agreement aboutwhat is a fair or equitable distribution of thosebenefits. Controversies about what count asequitable distributions in the health sector are afundamental source of the difficulty of prioritizationand rationing choices.”

– Dan W. Brock, Health care resource prioritizationand rationing: why is it so difficult? 200725

Broader social effectiveness has much to do withwhether direct health outcomes alone, or broader socialoutcomes as well, should be considered in prioritysetting. Some argue that direct health outcomes areparamount and broader social outcomes should not beconsidered or should be deemed second-order benefits,

worth much less weight – particularly if they benefitpeople who are not direct recipients of service. Othersargue that health outcomes can only be seen in thecontext of broader social outcomes, and that in someinstances indirect broader social benefits shouldoutweigh direct health benefits.


“The third component of the issue of priority to the worst-off in health care resource prioritization is how muchpriority they should receive, which will be determined in part by why they should get priority. Even assuming acompelling moral reason for such priority, it would be implausible to give the worst-off absolute priority. Doingso would raise what has been called the “bottomless pit” problem – for persons with very serious disabilitieswhom we can only make slightly better off, but at enormous cost in resource use, assigning their needs absolutepriority would excessively drain off resources for very little gain that could be used to produce much greaterbenefits for others less badly off. Here again, priority to the worst-off or disabled must be balanced against othermoral considerations, including treatment effectiveness or benefits.”

– Dan W. Brock, Health Care Resource Prioritization and Discrimination Against Persons With Disabilities26

Examples of Health Outcomes and Broader Social Outcomes in Priority Setting

Three programs geared to addiction to prescription medications must be prioritized. One program will servewomen aged 18 to 45 who have children. The second will help seniors with multiple health disorders. The thirdwill help socially isolated people.

The women’s program and senior’s program can achieve the same average addiction outcome per client at thesame cost. The isolates’ program looks as though it can achieve greater average addiction outcome per unit ofcost than the other two programs.

Proponents of the women’s program say it should be high priority since it will benefit women directly in additionto indirectly benefitting their children, since mothers will be able to more consistently nurture their children ifaddiction is under control.

Proponents of the seniors’ program say it should be high priority because it will indirectly benefit familymembers and friends who are caregivers. Seniors who control their addiction will need less care from thecaregivers, reducing caregiver stress.

Proponents of both these programs maintain that the isolates’ program should be given lowest priority, sinceisolated individuals are not looking after children and are estranged from family members or friends who mighthave been caregivers – and that there are therefore no groups who can indirectly benefit from the program.

Proponents of the isolates’ program, however, say it produces the most direct health gain per unit of cost andproduces the indirect benefit of helping isolated people to reconnect with families – a social benefit to theisolated individuals and their families.

To what extent should broader social benefits be factored into deciding how the three programs will be prioritized?

Efficiency may also seem like mere bloodlesscalculation rather than an ethical principle. But, if aservice can be provided at less cost with the same levelof outcome (i.e., it can be more efficient), theninefficiencies are a waste of resources that could be putto better use. Conscious waste is an ethical issuebecause of its denial of resources to other social goods.This can be seen as unfair to either the taxpayer (whomay have to pick up the cost of meeting some otherneed anyway) or to populations whose needs remainunmet because of inefficiencies.

This does not mean that every person or jurisdictionthat is inefficient is unethical. But when an inefficiencyis known to exist, when it is known that it can beremoved without harm, when the option exists to findout how to remove it and when it is known that savingscan be applied to other social goods, inefficiency entersthe realm of ethics.

Equity, or fairness of the health service outcomes ofpriority setting across a population, is anotherimportant value consideration. Equity is increasinglyseen as an ethical issue with two dimensions:

1. equity of access to health services; and

2. equity of health outcome as a result of health services.

The equity-of-access proponent would argue that allpeople should have equal access to a service, even ifsome people in the population have characteristics orface conditions that make it unlikely they will benefitfrom the service to the same degree as other people.

The equity-of-outcome proponent would argue that allpeople who access a service should achieve the samelevel of outcome if at all possible, even if it costs morefor some people to achieve that level of outcome thanothers.


An Example of an Equity Issue in Priority Setting

Two identically priced proposals ($100,000) have been put forward for prioritization.

One proposal would augment trauma services to a middle class population of 2,000 people. It is calculated that thecost to restore a person within that population to “normal” (i.e., pre-trauma) functioning would be $2,000 per person.

The second proposal would augment trauma services to an inner city population of 2,000, many of whom arehomeless. The prevalence of joblessness, severe mental illness and addiction is higher in the homelesspopulation. Based on these compounding problems, restoring a person to the pre-trauma level of functioning iscalculated at $4,000 per person.

Proponents of the “middle class” proposal argue that it should be higher priority because it incurs less cost perunit of improvement, the population makes more money and its members are therefore more socially useful thanhomeless people, and failure to award the program to the middle class population would punish its members forbeing successful.

Proponents of the inner city proposal argue that it should be higher priority because inner city residents are athigher risk of trauma, they have fewer coping resources when dealing with trauma, their chances of overcomingother problems are made less likely if they cannot be restored to at least their pre-trauma level of functioning,and failure to award the program to the inner city population would punish them for being poor and for havingenvironmental, health and social deficits.

Based on numbers, 50 middle class people could be restored to full functioning for the investment, while only 25inner city residents could be restored.

Who should get what – and why?

These two kinds of equity are not innately at odds witheach other. Most people would argue that both kinds ofequity are desirable, but these two positions maycompete with each other for priority. For example, ifnew money is available to a program, an argument maytake place about whether the money should be used toimprove equity of access to the program fordisadvantaged people or to help ensure that people whocan already access the program receive equitableoutcomes from the program.

In short, equity of access does not guarantee equity ofoutcome; but not even an inequitable level of outcomecan be achieved by a person who cannot access theservice in the first place.

2.1.2 Deserving and Undeserving Recipients

Issues sometimes arise in priority setting because of thebelief that there is a difference between servicerecipients who fully deserve service (and therefore canlay claim to high priority) and those who are lessdeserving (and therefore warrant lower priority).

This can arise in relation to health conditions that aredeemed self-inflicted and lifestyle-driven (a healthcondition related to smoking, substance abuse orinappropriate eating practices for instance).Counterbalancing this is the theory that many suchsupposedly self-inflicted problems are the results ofdeterminants of health beyond the control of theindividual, and that giving lower priority to service forthese populations is to punish them twice viaperpetuation of socioeconomic conditions that harmthem in the first place and via denial of service whenthey develop health problems.

The deserving/undeserving paradigm is also found in thearea of the non-compliant service recipient. Cardiaccare patients whose cardiac problems result largelyfrom smoking for instance, and who do not give upsmoking before or during treatment, may end up aslower priorities for continued or repeated (and oftenexpensive) treatment, on the grounds that suchtreatment will be relatively less effective than it will befor people who quit smoking (i.e., “people who deserve

it more”). In fairness, such decisions are often not basedon moral judgment of the client; they can just as easilybe based on the principle of giving service to those whocan most benefit from it, and may be justified.

At times, whole service sectors are vulnerable in termsof the deserving/undeserving paradigm. In relation tothe addictions field, for instance, where an addiction issometimes seen as a chronic disorder with relapses asthe norm rather than the exception, there are nonethelessconcerns from time to time that relapse is a moral failure,rendering its victim less deserving of future service.

2.1.3 Fair Priority Setting Processes:

Accountability for Reasonableness

While fairness in the outcomes of priority setting ishighly desirable, the fairness of the process itself is alsoimportant. This is an ethical issue (since fair process isa form of procedural honesty) and a pragmatic issuebecause a fair process is more likely to produce fairoutcomes and to be accepted by the public.

One recently developed approach to development of afair priority setting process is the Accountability forReasonableness model (sometimes called “A4R”). Asoriginally developed, this model comprised four


“Bethwaite and Cangialose (2000) argue thatwithout a transparent mechanism for reconcilingprinciples the prioritization process will not havethe public’s confidence. Moreover they argue thatbecause it may be impossible to achieve fairness inthe outcomes of any systematic process, the bestthat can be hoped for is that the process itself isfair. They recommend a system that incorporatesthe due process characteristic of the courts, e.g.,community juries. Further, they recommend thatdecisions be made by people chosen for their abilityand credibility in making wise judgments accordingto defined principles.”

– Draft Prioritization Policy: Background Paper,Canterbury District Health Board (New Zealand),

200127

conditions or components,28 but Gibson, Martin andSinger have suggested that a fifth condition –empowerment – be added.29

The five conditions are:

1. Relevance

Decisions should be based on reasons (i.e., evidence,principles, values or arguments) that fair-minded peopleagree are relevant under the circumstances. Fair-mindedpeople are those who seek in principle to cooperatewith others to find mutually justifiable solutions topriority-setting problems.

2. Publicity

Decisions and their rationales and the processes thatled to the decisions should be made publicly andconveniently accessible.

3. Revision

There should be accessible opportunities and processesto revise decisions based on further evidence orarguments and there should be a mechanism forchallenges, appeals and dispute resolution.

4. Enforcement

There should be either voluntary or public regulation ofthe process to ensure that the other four conditions aremet.

5. Empowerment

There should be efforts to optimize effective opportunitiesfor participation in priority setting and to minimizepower differences in the decision-making context.29

Gibson, Martin and Singer also identify a number of“lessons learned” about what can be done to achievethese five conditions:29

Relevance

• develop a rationale for each priority-setting decision;

• use decision criteria based on your mission, visionand values;

• collect data/information related to each criterion;

• consult with internal/external stakeholders to ensurerelevance of decision criteria and to collect relevantinformation; and

• make decisions using a multidisciplinary group ofpeople.

Publicity

• communicate the decision and its rationale; and

• use an effective communication strategy to engageinternal/external stakeholders around priority settinggoals, criteria, processes and decisions.

Revision

• incorporate opportunities for iterative decisionreview; and

• develop a formal decision-review process based onexplicit decision-review criteria.

Enforcement

• lead by example (i.e., ethical leadership); and

• evaluate and improve the priority-setting process.

Empowerment

• support people with leadership development andchange management strategies.


“Accountability for reasonableness identifies four conditions. Together, these conditions describe an open andtransparent priority-setting process that engages stakeholders constructively, ensures publicly defensibledecisions and supports decision-makers’ accountability for managing limited resources. The biggest challengefor decision-makers is how to implement these conditions in real-time priority setting – that is, how to putethical priority setting into practice.”

– Jennifer L. Gibson, Douglas K. Martin and Peter A. Singer, Evidence, Economics and Ethics: Resource Allocation in Health Services Organizations, 200529

2.1.4 A Framework for Ethical Priority Setting

There is no widely accepted tool or universalframework that can let priority setters know if theirwork is ethical, but several attempts to develop such aframework have been tried. One such attempt identifiesfive approaches to allocative ethics:30

1. the Virtue Approach, based on the principle that theethical action is the one that embodies the habits andvalues of humans at their best;

2. the Utilitarian Approach, based on the principlethat the ethical action is one that will produce thegreatest balance of benefits over harms;

3. the Rights Approach, based on the principle thatthe ethical action is the one that most dutifullyrespects the rights of all affected;

4. the Fairness or Justice Approach, based on theprinciple that the ethical action is the one that treatspeople equally, or if unequally, that treats peopleproportionately and fairly; and

5. the Common Good Approach, based on theprinciple that the ethical action is the one thatcontributes most to the achievement of a quality ofcommon life together.

2.1.5 Unarticulated Assumptions

This section has defined unarticulated assumptions asthings that are generally unspoken but that are

considered true or false (even if they do not have a

moral association).

Unarticulated assumptions in any field are difficult touncover and address even though – as the deepest levelof organizational or system culture – they are ultimatelythe culture’s bedrock.

While a priority setting process should not become anendless exercise in identifying unarticulatedassumptions, it is important to at least identify andconsider some of the most crucial assumptions, todetermine if they are valid and if they are shared. Forinstance:

• Some participants in priority setting may assume thatif people agree to deliberate dispassionately on theitems they are prioritizing, they are able to do sowithout any guidance on how to deliberate.

• Some participants may assume that certainprofessions are more (or less) able to contribute toprioritization.


Figure 6: A Sample Ethical Framework

Does this action embody

the habits and values of

humans at their best?

Does this action produce

the greatest balance of

benefits over harms?

Does this action most

dutifully respect the rights

of all affected?

Does this action treat

people equally, or if

unequally, does it treat

people proportionately

and fairly?

Does this action contribute

most to the achievement of a

quality of common life

together?

VIRTUE

UTILITARIAN

RIGHTS

FAIRNESS

OR

JUSTICE

COMMON

GOOD

• Some participants may assume that controversy andfrank debate are signs of a process’s weaknesses.Others may assume it is a sign of strength.

• Some participants may assume that compromisewhen setting priorities is a betrayal of objective truthand evidence. Others may assume compromise isnecessary to achieve consensus on priorities.

If done well, consideration of values and unarticulatedassumptions can form the basis of working principlesfor a priority setting process.

2.2 Group Process in Priority Setting

A priority setting process can be carried out by a singleperson with a calculator and a conscience – but that isnot how it is normally done. Two groups of people arenecessary for any but the simplest of priority settingendeavours, and these two groups must work together.

1. People who manage the prioritization process. Thesemay be administrators or they may be specialists ininformation retrieval, project design andprioritization methodologies.

2. People who actually set the priorities, within thefacilitating framework created by the first group.

2.2.1 People Who Manage the Priority Setting

Process

As with any other projects, a priority setting processbenefits from having somebody designated to managethe overall process. In addition, the team should includepeople with skills sufficient to design a sound processand to advise the people who actually set the priorities.Such advisors may include:

• epidemiologists;

• people who have conducted research into the priority setting phenomenon;

• statisticians;

• economists;

• planners; and

• people with expertise in health care ethics.

2.2.2 People Who Set the Priorities

The group of people who actually set the priorities can beas small as a few experts or as large as a whole nation.

The Experts

A priority setting project benefits from having peopleinvolved in the process who are experts in the healthfield in which the items to be prioritized are situated. In setting priorities related to mental health, forinstance, it makes sense to have mental health expertsaround the table. When this is done, however, careshould be taken to avoid conflict of interest or sub-field


“At the… Conference on Priorities in Health Care…one resonating theme was that for priority setting tobe effective, it has to include clinicians in bothdecision making and the enforcement of thosedecisions. There was, however, a disturbingundertone to this theme, namely that doctors, inparticular, were unjustifiably thwarting goodsystems of prioritising scarce healthcare resources.This undertone seems unfair precisely becausedoctors may, and in some cases do, feel obligatedby their professional ethics to remain uninvolvedeither in deciding priorities and in some cases inenforcing them… The professional role of a doctorought not be considered inconsistent with the roleof a priority setter or enforcer, as long as onecrucial element is in place, a rationally coherent andbroadly justifiable regime for prioritising healthcare.Given this I conclude both that prioritisation anddoctoring are not incompatible under certainconditions, and that the education of healthcareprofessionals ought to include material ondistributive justice in healthcare.”

– David Hunter, Am I my brother’s gatekeeper?Professional ethics and the prioritization of

healthcare, 200731

bias (a psychogeriatrician involved in priority settingmay, for instance, give greater priority to mental healthservices for the elderly because this is her sub-field).And while epidemiologists, economists and ethicistsmay be part of the management group, involvement ofpeople from these backgrounds in the actual setting ofpriorities may also be warranted. Consumers and familymembers are also experts with unique perspectives onservice delivery and should be involved not merely as acourtesy, but because their expertise is essential.

The involvement of a wide cross-section of interestedparties increases the likelihood that the decisions madeby priority setting bodies will be accepted asreasonable. While this will partly be due to the ability ofstakeholders to affect the decision-making process,involvement in decision-making can also lead to agreater appreciation of the issues involved in prioritysetting and a recognition of the need for compromisesand making of difficult decisions.32

The Public

As part of the wide cross section of interested parties,some priority setting processes engage the public, togreater or lesser degrees. Ontario’s LHINs have a clearinterest in public involvement in priority setting, in partbecause the legislation that established LHINs requiressuch participation:

A local health integration network shall engagethe community of diverse persons and entitiesinvolved with the local health system about thatsystem on an ongoing basis, including about theintegrated health service plan and while settingpriorities.33

Despite the enthusiasm shown by decision-makers forpublic involvement, the consequences of involving thepublic in priority setting has been the subject ofconsiderable debate. Some critics of public involvementwarn of the risk of establishing a “dictatorship of theuninformed” as a result of public engagement.15

According to this view, public participation is inherentlyun-egalitarian and carries the danger of attaching a lowpriority to the needs of people with mental illness,

people with developmental disabilities, elderly peopleand those who are poor and inarticulate.

Writing from a Canadian perspective, Lomas argues thatthe limited interest and skills of the general public inrelation to specific health services means that theirinput should be restricted to advice on broad servicecategories, and that they should not be able to actuallydetermine priorities.34 This raises the possibility that thepublic might be involved in some steps or kinds – butnot all steps or kinds – of prioritization. For instance,citizens might be involved in determining that initiativesgeared to seriously mentally ill people ought to be givenhigher priority than services for people with less severemental illness, without getting into a prioritization ofproposals for serving people living with mental illness.

In the UK the National Institute for Health and ClinicalExcellence (NICE) has used a citizen’s council (madeup of 30 ordinary members of the public, reflecting theage, gender, socioeconomic status and ethnicity of thepeople of England and Wales) to produce rules forpriority setting – a concrete example of using a citizen-based tool to engage the public in part, but not all, ofthe priority setting process.47


“The Citizens Council makes an importantcontribution by enabling NICE to take the views ofthe general public into account when undertakingits work. Since its inception, the Citizens Councilhas provided valuable input on a number ofcontentious subjects, including social valuejudgements, and the ‘rule of rescue’… Now we areasking the Council to consider health inequalities.…We will ask the council to consider whether it isappropriate for NICE guidance to concentrateresources on trying to narrow the gap between theleast and most disadvantaged members of oursociety even if this has only a modest impact on thehealth of the population as a whole.”

– Michael Rawlins, Chair of the UK NationalInstitute for Health and Clinical Excellence, 200647

The State of Oregon’s groundbreaking effort atestablishing health priorities provides another exampleof how the public can be involved in setting priorities.35

While the state established a Health ServicesCommission with the responsibility for developing a listof health services prioritized from most important toleast important, the Commission sought information onpublic values concerning health care using three tools:

1. twelve public hearings that received input fromOregon residents concerning their health careexperiences and preferences;

2. approximately 50 facilitated focus groups acrossOregon that identified health values on which therewas some consensus; and

3. a survey of 1,000 Oregon residents to identify theimpact on overall health resulting from a broad rangeof hundreds of health conditions such as shortness ofbreath, limited range of motion, social dysfunctionand hearing loss.

Recognizing the problems arising from the public’slimited expertise in complex, technical areas, otherresearchers have pointed out that representativedemocracy is the traditional method for dealing withsuch problems and argue that direct public participationin priority setting decisions represents a break with thistradition. Set against these views, advocates ofmechanisms such as citizens’ juries regard publicparticipation as a valuable supplement to thedemocratic process.

The outcome of a citizen engagement process is not ourbest hypothesis about what is right. It is our besthypothesis about what the community thinks is right,after deliberation, clarification and careful reflection. Ifthe process that incorporates the community’s view isinclusive, uncoercive and transparent, there are groundsfor accepting the outcomes as legitimate and the basisfor public policy, even if there remain disagreementsabout whether it is just.36


This section outlines steps found in many priority settingprocesses. It then identifies key economic concepts thatare applied in these steps. Next, it discusses a specifictechnical process – program budgeting and marginalanalysis – that illustrates a way to set priorities basedon what the numbers say and what the people say.

3.1 The Steps That Numbers Serve

3.1.1 Identifying the Set of Items to be

Prioritized

The first step in priority setting involves identifying theset of items to be prioritized. This may seem simpleenough, but it is sometimes useful to identify threecomponents in the project’s initial decision makingprocess:

1. a base definition of the set (for instance, programsmeant to rehabilitate people with an acquired braininjury);

2. a statement of what is to be included in the set; and

3. a statement of what is not to be included in the set.

Approaching it in this three-dimensional way helps withproject clarity and communications.

3.1.2 Identifying Criteria for Priority Setting

This process should identify the specific characteristicsof the items in the set that will be compared with eachother. In general, seven kinds of criteria can beidentified:

1. problem severity criteria (How severe is theproblem that will be addressed by this item?);

2. problem prevalence criteria (How many peopleexperience this problem at each level of severity?);

3. benefit criteria (How much good will the item do?);

4. cost criteria (How much will the item cost?);

5. ethical/values criteria (What values should theitem serve or not serve?);

6. feasibility/competence criteria (Can the item dowhat it proposes to do, and does it have a strategyfor doing it?); and

7. risk/harm criteria (Would the operation of the itementail risk or harm, at acceptable or unacceptablelevels?).

Section 3: The Numbers Part Page 21

Section 3

The Numbers Part

Figure 7: What’s In and What’s Out

WHAT IS TO BE INCLUDED

IN THE SET

e.g.,

new proposals for service

or

proposals for expansion of service

or

programs in which cost savings

might be made

or

programs that might be eliminated

or

programs that should be better

integrated with other programs

WHAT IS NOT TO BE INCLUDED

IN THE SET

e.g.,

services that address brain injury as a

secondary role

or

proposals from agencies with no track

record in brain injury service

or

services that have not undertaken

ongoing evaluation of their work

BASE DEFINITION OF THE SET

e.g.,

programs meant to rehabilitate people

with an acquired brain injury

Severity and prevalence criteria:

These are two important criteria because they are keycriteria for determining the negative health impact of acondition, and therefore can influence the priority to begiven to a solution to the condition when compared tosolutions to other conditions.

Severity can be measured in a number of ways, fromsimple scales to more complex approaches thatrecognize that as severity increases, complexity mayincrease.

A number of value issues arise in looking at complexity,essentially because different people attach differentlevels of importance to different kinds of severity.

Prevalence is about the number of people in apopulation who experience a health problem. But theraw number of people who have a disorder says littleuntil it is coupled with an understanding of severity. Forinstance, a comparison of two conditions only on thebasis of prevalence may show that within a population,100 people per 100,000 population experience the firsthealth condition. Only 60 people per 100,000 experiencethe second condition. Based on prevalence alone, thefirst condition seems more important because it affects

more people. However, the percentage of the populationwith the first condition who experience it in a formsevere enough to make it impossible for them to walk is10%, or 5 people per 100,000 population – but thepopulation who experience the second condition in aform severe enough to make it impossible for them towalk is 50%, or 30 people per 100,000. Therefore whenboth prevalence and severity are taken into account, thesecond disorder may emerge as a higher priority eventhough it is less prevalent.

Usually when a disorder is low prevalence and lowincidence, or high prevalence and high severity, how itfits broadly in a priority setting exercise is fairly clear.However, comparing a high prevalence/low severitydisorder with a low prevalence/high severity disordercan be tricky.

In the State of Oregon’s initial priority setting process,for instance, pulp capping for dental pulp exposure (todeal with a high prevalence/low severity problem) wasgiven higher priority than surgery for ectopic pregnancy(to deal with a low prevalence/high severity problem).37

This priority was later reversed by Oregon’s HealthCommission.

Page 22 Section 3: The Numbers Part

An example of severity complexity

A clinician trained in rehabilitation may look at a debilitating and painful health condition and place greatimportance on the severity of life restrictions that are placed on people with the condition. Accordingly he giveshighest priority to restoring mobility, and gives lower priority to the severity of pain experienced by people withthe condition.

A person who suffers from the condition may however, be willing to live with less restoration of dailyfunctioning in return for greater attention to pain alleviation. As she puts it, “I would sooner use a wheelchair if

it means I can live my daily life with a more tolerable level of pain.”

In short, a clinician and a client may have different views of which kind of severity reduction should havegreatest priority.

Benefit and cost criteria:

Two kinds of criteria – benefit and cost – are oftencombined in a formula. For example:38

• Cost-benefit analysis estimates the overall value,or net social benefit of an item. The net social benefitis the difference between total benefit (expressed inmonetary terms) and total cost. • Cost-effectiveness analysis shows the cost of

achieving a given output. It measures outputs inphysical units rather than in monetary terms. Costsare still measured in dollars.


Figure 8: The Tricky Ones In Terms of Prevalence and Severity

high prevalence, low

severity

high prevalence, high severity

(usually high priority)

)

low prevalence, low severity

(usually low priority)

low prevalence, high

severity

SEVERITY

PR

EV

AL

EN

CE +

+ +

THESE ARE THE TRICKY ONES

WHEN SETTING PRIORITIES.

“Decide what you want, decide what you are willingto exchange for it. Establish your priorities and goto work.”

– H.L. Hunt

Examples of cost-effectiveness analysis

statements:

• Cost of $100,000 produces 500 counsellingsessions

• One counselling session costs $100,000 dividedby 500 sessions = $200

Examples of cost-benefit analysis

statements:

• At a cost of $100,000, a program will producebenefits worth $180,000

• Net social benefit = $180,000 minus $100,000 =$80,000

• Benefit-cost ratio: benefit of $180,000 dividedby cost of $100,000 = benefit-cost ratio of 1.8 (aratio greater than 1 is a net social benefit)

• Cost-utility analysis is a compromise betweencost-benefit analysis and cost-effectiveness analysis.It attempts to overcome the limitations of a singleoutput measure by valuing different health outcomesin utility terms, using a quality of life (QoL) index.

If a priority setting process decides that both costs andbenefits are important and wants to look at therelationship between them, the choice of any of themethods cited above is complex. Each method hasadvantages and disadvantages. For instance:

• cost-benefit analysis expresses both costs andbenefits in comparable dollar terms, but cannot beused if benefits cannot be quantified in dollar terms;

• cost-effectiveness analysis allows two items to becompared to determine if one item produces its unitof outcome at lower cost, but this only works whenthe outcome units of the items are the same; and

• cost-utility analysis tries to measure benefits in unitsthat reflect actual improvements in health, but commonutility indices such as quality-adjusted life years ordisability-adjusted life years have been criticized asimprecise or biased against some populations (e.g.,the elderly, or people with disabilities).


Examples of cost-utility analysis

statements:

• Without a one-time intervention, 10 people wouldeach have a quality of life of 0.4 for the next fiveyears (where 1 = complete health and 0 = animpaired state of health judged to be equivalentto death).

• With intervention, these 10 people would eachhave a quality of life of 0.6 for the next five years.

• Gain in quality of life for these 10 people = 10people x gain of 0.2 (0.6 minus 0.4) x five years =10 quality adjusted life years.

• Cost of one-time intervention for these tenpeople = $100,000. Therefore, cost of each qualityadjusted life year = $100,000 divided by 10 qualityadjusted life years = $10,000.

“Cost-effectiveness: Made popular by the defenseindustry when it was used to find the cheapest waysto kill the largest number of people, cost-effectiveness is now used by health specialists tomaximize the positive health outcomes with thelimited available resources. While it is basically aneconomic evaluation tool, public health specialists,much more so than economists, swear by it as aprimary prioritization tool. A cousin of cost-effectiveness, cost-benefit, has been shunned by thepublic health community for shamelessly assigningmonetary value to human life (something personalinjury lawyers do every day).”

– Abdo S. Yazbeck, An Idiot’s Guide to Prioritizationin the Health Sector, World Bank, 200239

Ethics and values criteria:

While these criteria are not generally quantifiable asdollars or units of output or outcome, scales can be usedto gauge participants’ assessments of ethics and values.

Feasibility/competence criteria:

Some priority setting processes may take into accountthe likelihood that a program will be able to achieve itsbenefits or is competent enough to achieve them.Asking questions about feasibility – and particularlycompetence – must be handled with great care toensure that innuendos do not creep into the use of thecriteria. Sometimes, however, it is possible to examinefeasibility or competence by looking at the results ofcompliance checks, accreditation activities or thirdparty evaluations.

Risk/harm criteria:

A hallowed precept within health care – and medicine inparticular – is “primum non nocere” – first, do no harm.But in health care a degree of harm or risk of harm issometimes necessary because the benefits outweigh theharm or risk, calling into play the precept “primumsuccerrere” – first, hasten to help.

A simplistic example of applying the risk/harm criteriamight be priority setting involving two programs meantto serve the same number of people with the same health

problem at the same cost, but each program using adifferent set of interventions. Both programs achievecomparable success rates of cure in 20% of those served.However, the interventions in the first program lead todrug toxicity deaths of 10 people per 100,000 treated,while the second program leads to drug toxicity deathsof two people per 100,000 treated. All else being equal,the second program should be the highest priority.

3.1.3 All Criteria are Not Created Equal:

Assigning Weights to the Criteria

While it is possible to treat all criteria as equal, in manypriority setting projects it will be important to assignweights to the criteria that reflect their different degreesof importance. For instance, some people might consider“severity” as twice as important as “prevalence” whenprioritizing proposed interventions for a health condition.

If a group is setting priorities, some degree of consensuson weights among the priority setters is advisable. Whilethis can be achieved in part through the numericalprocess by which weights are established (averagingparticipants’ weights for each criterion for example), adegree of discussion and negotiation may be necessary tomake the weights credible in the eyes of priority setters.

Appendix A provides an example of establishingweights based on the matched pairs method.


“Does this program respect the freedom of its clients to exercise choice over the care they receive?”

0 1 2 3 4

never

“This program attempts to meet the unique needs of people with disabilities.”

0 1 2 3 4

completely

disagree

agree

somewhat

completely

agree

An Example of a Yes/No Value Criterion

“Does this program have a charter of client rights and responsibilities?”

yes

sometimes always

no

Figure 9: Examples of Value Measurements

3.1.4 Indicators That Provide Information on

Criteria

Having a set of criteria to be used in priority setting isimportant, but setting the priorities cannot proceeduntil the project identifies indicators that allow thepriority setters to determine how well each item in theset stacks up in relation to each criterion. Indicators aremeasures constructed to be comparable over time andacross jurisdictions.

There are different types of indicators used in the healthdomain:40

• Health indicators are measures that reflect, orindicate, the state of health of persons in a definedpopulation.

• Health system or health care indicators reflectactivities that promote health or respond to diseasesuch as disease screening rates or average lengths ofstay.

• Performance and quality indicators are healthsystem indicators that are designed to track specificdimensions of the health services system.

• Structural indicators provide descriptiveinformation such as the number of beds in a facility.

• Process indicators are commonly used to improvemanagement and quality. Wait times may beconsidered a process indicator.

• Output indicators are among the most often citedmeasures. These are the amounts of activity recorded,such as surgeries performed or meals delivered.

• Outcome indicators may be difficult to relatedirectly to the actions of the health care system butare nonetheless measures of the system’s goals.Mortality rates and measures of population health areoutcome indicators.

In some instances there may simply be no existingindicator that would allow a criterion to be understoodand used, even after the priority setters have scoured

the literature searching for an indicator. In such casesthe priority setters may need to develop an indicatorand to assure themselves that information exists, or canbe gathered, to allow the indicator to be applied. In afew cases it may be impossible to develop a reliableindicator, or it may be impossible or unfeasible to findor gather the information needed for items in the setthat will allow the indicator to be applied. When thishappens the criterion may need to be abandoned.

An indicator is not the same as data or otherinformation that allows the indicator to be applied. Forinstance:

• A criterion for priority setting may be “severity of thedisorder a program will address”.

• For this criterion, indicators might be:

• deaths per 100,000 resulting from this disorder;and/or

• decrease in quality adjusted life years per 100,000people with this disorder.

3.1.5 Gathering Data on the Indicators

Data may be readily at hand to allow criteria to beapplied to prioritize items in a set. For instance, thereporting processes of programs or agencies maycontain data of use to priority setters. However, in somecases the prioritization process will need to go furtherafield to find data. If, for instance, a priority settingprocess needs information on the cost-effectiveness of aprocedure such as cochlear implants, it may want toconnect with the web sites of organizations such as theCochrane Collaboration or the UK’s National Institutefor Health and Clinical Excellence (NICE):

• The Cochrane Collaboration was formed in responseto the need for the best evidence to influence healthcare practice. Its aim is to prepare and maintainsystematic reviews of the effects of healthinterventions and to make this information availableto practitioners, policy makers and consumers. TheCochrane Collaboration’s web resources can beaccessed at http://www.cochrane.org.


• The UK’s NICE develops guidance with the aim ofensuring that the promotion of good health andpatient care in local health communities is in linewith the best available evidence of effectiveness andcost effectiveness. NICE web resources can beaccessed at http://www.nice.org.uk.

Module 3 in the Health Planner’s Toolkit (Evidence-

Based Planning) provides more information onindicators, evidence and places to find evidence.

3.2 Program Budgeting and Marginal

Analysis (PBMA)

An approach to priority setting that has gainedascendancy in the last few years is Program Budgetingand Marginal Analysis (PBMA), used extensively inWestern Canadian health authorities and in Australia,New Zealand and Britain. It has merit for four reasons:

1. It is not only about what priorities to fund. It is alsoabout what priorities for cost savings can be found toprovide the fuel for service growth and expansion.

2. It uses the same language on both sides of thesave/spend ledger – including the language of costeffectiveness.

3. It does not try to reform the entire system all at oncethrough a single grand priority setting project. It canbe applied within a single program, or between twoor more programs without breaking the link betweenwhat to save and what to spend. It is an incrementalchange strategy – on the premise that incrementalchange eventually produces system change. In short,it is pragmatically flexible.

4. It is particularly useful when the supply of new money,and the supply of ways to create efficiencies withoutcreating service reductions, has been exhausted.


“The [PBMA] process can then be repeated over aperiod of time, perhaps a number of years, so thatmore and more services which are difficult toevaluate are progressively assessed. By sequentiallyrepeating the process, the emphasis of PBMA is togradually move towards the allocative efficiencygoal of maximising health gains for a given level ofresources.”

– Diana Edwards, Stuart Peacock and Rob Carter,Setting Priorities in South Australian Community

Health III: Regional Applications of ProgramBudgeting and Marginal Analysis41

WHAT, WHERE AND

HOW TO SAVE:

One measure is cost

effectiveness

WHAT, WHERE AND

HOW TO SPEND:

One measure is cost

effectiveness

+

=scalable fundable change

Where the $$ go

Figure 10: PBMA’s Virtues

The PBMA approach involves a number of stages.42 A key step in the PBMA process is the development of“candidate lists” – candidates both for creation/growthand for elimination/reduction (see steps 5a and 5cabove). Both lists are analyzed using the same criteria,to arrive at a priority status for each item on each list,with comparable scores that allow the two lists to beintegrated.

Assume, for instance, that the PBMA project hascreated a list of four programs that could be created orcould grow (A, B, C and D) and a list of four programsthat might be reduced or eliminated (E, F, G and H). Alleight programs are then evaluated based on a set ofweighted criteria, to produce the following scores andranking in each list.

Figure 11: The Candidate Lists

Since the same criteria were used for all items on eachlist, the lists can be integrated – and the lowest rated

program that could be reduced or eliminated isconsidered as the source of funding for the highest

rated program that could be created or expanded.Similarly, the second lowest rated program that couldbe reduced or eliminated is considered as the source offunding for the second highest rated program that couldbe created or expanded.


Stages in a PBMA priority setting process

1. Determine the aim and scope of the prioritysetting exercise

2. Compile a program budget (i.e., map of currentactivity and expenditure)

3. Form marginal analysis advisory panel

4. Determine locally relevant decision makingcriteria

a) decision maker inputb) board of Director inputc) public input

5. Advisory panel to identify options in terms of:

a) areas for service growthb) areas for resource release through producing

same level of output (or outcomes) but withless resources

c) areas for resource release through scalingback or stopping some services

6. Advisory panel to make recommendations interms of:

a) funding growth areas with new resourcesb) decisions to move resources from (5b) into (5a)c) trade-off decisions to move resources from

(5c) to (5a) if relative value in (5c) is deemedgreater than that in (5a)

7. Validity checks with additional stakeholders andfinal decisions to inform budget planning process

– Mitton and Donaldson, Health care prioritysetting: principles, practice and challenges42

A = 97

B = 81

D = 69

The candidate list for

creation or growth

C = 55

The candidate list for

reduction or elimination

G = 76

F = 60

H = 49

E = 41

Program G was originally considered a program thatmight be reduced or eliminated – but it scored fairlyhighly and is therefore no longer considered an optionfor reduction or elimination.

When the priority setters get to program C (a candidatefor growth or expansion), there is no lower ratedprogram that could be reduced or eliminated.


A = 97

B = 81

G = 76

D = 69

F = 60

C = 55

H = 49

E = 41

Fund “A” by taking money from “E”

Fund “B” by taking money from “H”

Figure 12: Interaction Between the Candidate List

What this PBMA process has produced so far is not afinal decision on what will grow and what will not.Instead, it is the basis for negotiation. For instance – tofund program A, should program E be eliminatedaltogether, or should it be reduced but not eliminated?

Section 4: Organizing and Carrying Out a Priority Setting Process Page 31

Organizing and carrying out a priority setting processshow many features common to other projects. Theyalso involve specific unique steps related both to thepeople part and the numbers part of priority setting.The following framework incorporates the why, who,what and how questions relevant to setting priorities.

4.1 Establish a Mandate for Priority

Setting

Somebody has got to start a priority setting process. Itmay be a provincial ministry of health, a regional healthauthority, a network of agencies serving similarpopulations, an agency that wants to set prioritieswithin its programs, or a single program that wants toprioritize program activities. In addition to the initialperson, group or organization that gets it started, theremay be community partners whose help is essential incarrying out the process. Establishing a mandate shouldinvolve identifying (even provisionally) the partnergrouping that is necessary to allow prioritization tohappen. Within this grouping, a lead entity forcoordinating the process should be identified. Usually(but not always) the lead turns out to be the entity thatthought up the idea of the priority setting process in thefirst place, or an entity that was instructed or requestedto do so by another authority (a ministry, for instance,might require a regional health authority to take a leadin a prioritization project).

Establishing the mandate should also involve:

• the development of a clear statement of the purposeof the priority setting process;

• a statement of the set of items to be prioritized,although the definition of the set might be revised(either broadened or narrowed) as the project gains

greater understanding of the issues it faces. Forinstance, a project may initially define the set as“proposed new programs/program expansions forpost-stroke rehabilitation”. It may later broaden theset by adding rehabilitation programming for peoplewith acquired brain injuries; and

• at least a preliminary start at dealing with one of thefive conditions for Accountability for Reasonablenessdescribed earlier in this module – the “enforcement”condition (voluntary or public regulation of theprocess to ensure that the other four conditions aremet). Stakeholders may take a priority setting processmore seriously from the very beginning if they knowits processes and outcomes will be enforced.

At this point the items that will eventually fit within theset may not have been identified. This can take placelater in the process.

4.2 Establish the Process Management

Group

This is comprised of the people who will manage andprovide technical support to the project. Generally theyare not members of the priority setting group (describedbelow). In other words, they do not “cast a vote” whenprioritization takes place. They fill the important role ofsupporting and informing, but not carrying out, theprocess.

4.3 Develop the Project Plan

This is an initial plan meant to identify and shapefurther steps in the process. It may be guided by a logicmodel developed for the priority setting project. Module6 (Evaluation)i in the Health Planner’s Toolkit seriesprovides information on developing a logic model.

Section 4

Organizing and Carrying Out a PrioritySetting Process

i The Health Planner’s Toolkit Module 6 – Evaluation. [Online]. 2008 [cited 2008 Mar]; Available from: URL:http://www.health.gov.on.ca/transformation/providers/information/resources/health_planner/module_6.pdf

4.4 Develop the Priority Setting Group

Members of this group will be responsible for carryingout the major priority setting activities such asdeveloping criteria, weighting the criteria, rating itemsin the set based on these criteria, and adjusting the finalpriority list.

While each of these activities could be carried out by aseparate group, there should at least be a core groupwhose members bring consistency to the process byhaving at least an oversight role (if not directparticipation) in all core priority setting activities.

This group may be multidimensional, including expertsin the subject matter of the set to be prioritized(pediatricians and parents, for example, if child healthservices are being prioritized). It may also includepeople knowledgeable about economic analysis, ethicsand health service administration. As well, seats on thegroup may be reserved for members of the public(people who can arrive at reasonable and ethicalpriority setting decisions in light of clear, relevant andunbiased supporting information).

A priority setting project should steer clear of people onthe priority setting group who can only behaverepresentationally – people who will favour a particularitem or constituency no matter what the evidence baseshows. In other words, priority setting is most credibleand competent when it is an adjudicational processrather than a representational one.

At various points in the priority setting process, othergroups may be identified to help at specific project stages.

For instance, a priority setting group may choose tocreate an advisory group of ethicists to assist with thedevelopment of values-based criteria for prioritization orit may decide to use public polling techniques or publicengagement activities to involve a representative sampleof the population in assigning weights to criteria.

The priority setting group and the process managementgroup will need to negotiate their relationship with eachother early in the project to avoid conflict and assurecollaboration between the two groups, and to ensuretransparency when the broader community inspects thepriority setting project’s processes and outcomes.

4.5 Establish an Accountability for

Reasonableness Strategy

Section 2.1.3 of this module described theAccountability for Reasonableness framework as a wayto ensure fair process. This framework has fivecomponents or conditions: relevance, publicity,revision, enforcement and empowerment.

Given the importance of fair process, the activities thatmust be employed to ensure that all five of theseconditions are met should be decided early in the life ofthe project.

In many ways the project’s Accountability forReasonableness strategy is a prospective formativeevaluation of the project, making it easier for aretrospective summative evaluation to take place at theproject’s end.

4.6 Develop an Appeals Process

Establishing an appeals process early in the life of theproject is desirable even though the process may notcome into play immediately. Such a process is requiredunder the provisions of the Revision condition of theAccountability for Reasonableness framework: “… there

should be a mechanism for challenges, appeals and

dispute resolution”. The existence of the process andthe dissemination of information about it helps forestallstakeholder concerns that the priority setting process isa “done deal”.

Page 32 Section 4: Organizing and Carrying Out a Priority Setting Process

“The best tool available to economists in aprioritization exercise is that nobody understandsthem. They employ three phrases to constantlybaffle other advocates and spend a considerableamount of everybody’s time trying to explain them.”

– Abdo S. Yazbeck, An Idiot’s Guide to Prioritizationin the Health Sector, World Bank, 200239

(with an apology to economists)

The appeals process may contain provision for appealsto guidelines for the inclusion of items within the set, aswell as appeals regarding the composition of thepriority setting group, the criteria, the weights assignedto them and the final prioritized list.

4.7 Establish and Implement a

Communications Plan

It is crucial to minimize community and stakeholderanxiety and resistance during the project. Prioritysetting is competitive, with winners and losers – thelosers being, at the very least, people whose personal ororganizational aspirations may be given low priority,and at worst, people whose cherished programs arereduced or eliminated. A communications plan does notremove anxiety and resistance, but it can minimize themthrough timely, frank, clear communications – startingwith dissemination of the project’s Accountability forReasonableness strategy. The communications plan isone key way to achieve the second condition of theAccountability for Reasonableness framework(“Publicity: Decisions and their rationales and the

processes that led to the decisions should be made

publicly and conveniently accessible”).

4.8 Understand the Environments for

Priority Setting

Agencies and sectors within a health system are oftenintimately interconnected in the sense that a changemade in one program or sector will affect otherprograms and sectors – and most importantly, mayaffect clients in each agency or sector. This suggeststhat before proceeding to other priority setting steps,the project should develop a greater understanding ofthe agency or sector within which priority setting willtake place, and the connections among agencies andsectors. This can include, but is not limited to, servicemapping. It may also include an examination ofdifferences in values and organizational culture amongthe agencies and sectors likely to be affected by thepriority setting process.

This activity may lead to a broadened membership ofthe project’s process management group and thepriority setting group. For instance, a priority settingprocess related to addiction services may initially haveleft out people whose primary knowledge base or“stake” is mental health. Understanding the connectionsbetween addictions and mental health may lead torecruitment of people with mental health perspectivesto serve on the project’s process management group andpriority setting group.

An understanding of the political environment forpriority setting is also important. While many peopledecry the “politicization” of health care, this field ofhuman endeavour is so crucial to a society’s well-beingthat it is inherently political, particularly in terms of thevalues that shape the sector. These values are oftenoperationalized through political processes andstructures. Accordingly, some of the values that drive apriority setting process and that are built into its set ofcriteria may come from understanding the valuescarried by the political process that ultimately fundsand regulates the system. Pragmatically, there arepolitically wise and unwise issues of timing and processwithin most priority setting projects in health care.Sensitivity to political issues involves understanding,respecting and working with the basic social worth andinfluence of the political process.


Three crucial environments…

• The service environment (what is connectedto what – and how, and why?)

• The political environment (how can thepolitical, ethical and technical perspectives worktogether?)

• The body of knowledge about priority

setting (have priorities been set in other placesor in other sectors or at other times, in ways thatthat can provide lessons for the current project?)

• The environment of values and assumptions

(what are the values and assumptions we holdabout priority setting or about the set of items tobe prioritized, and should we use or modify thesevalues and assumptions?)

A third environment that should be understood is thebody of knowledge about prioritization that can shapethe current project. Have similar sets of items beenprioritized in other jurisdictions and if so, what lessonscan be learned from these examples? What state-of-the-art tools have been developed elsewhere? Within thecommunity in which the current priority setting processis taking place, are there other past or current projectedpriority setting processes that can affect the currentprocess? Are adjacent jurisdictions also carrying outsimilar priority setting processes, and should theseneighbours’ processes mesh with the local process?

The fourth environment is the environment of valuesand assumptions (articulated and unarticulated) aboutpriority setting itself or about the set of items to beprioritized. We seldom think of values and assumptionsas an environment but they are much like the lake inwhich fish swim, surrounding us at all times. A frankdiscussion of values and assumptions early in theprocess can prevent misunderstandings later.

4.9 Identify Criteria, Weights and

Indicators

Earlier in this module seven kinds of criteria wereidentified:

1. problem severity criteria (How severe is theproblem to be addressed by this item?);

2. problem prevalence criteria (How many peopleexperience this problem at each level of severity?);

3. benefit criteria (How much good will the item do?);

4. cost criteria (How much will the item cost?);

5. ethical/values criteria (What values should theitem serve or not serve?);

6. feasibility/competence criteria (Can the item dowhat it proposes to do?); and

7. risk/harm criteria (Would the operation of the itementail risk or harm, at acceptable or unacceptablelevels?).

The project may also weight the criteria at this stage(weighting is described earlier in this module).

The criteria themselves and the weight given to each ofthem will be shaped partly by values and partly bytechnical considerations about what can be achieved byitems within the proposed set to be prioritized. Forinstance, a priority setting exercise that has identified“methamphetamine addiction programs” as the set to beprioritized would not be wise to identify “potential toincrease fine motor skills” as a criterion, simply becausemeth treatment programs do not address fine motor skills.

The criteria and their weights may also be in partdetermined by whether indicators and databases existor can be developed that allow actual measurement totake place in regard to an item in the set. It thereforemakes sense at this stage to find or develop indicatorsto be used across the set to show how well each itemfares on each criterion. Indicator development isdescribed earlier in this module and is explored morethoroughly in Module 3 in the Health Planner’s Toolkit(Evidence-Based Planning).

While the priority setting group has a dominant role inidentifying and weighting criteria, the processmanagement group takes on the lion’s share of the worknecessary to find or develop indicators related to thecriteria.

Identification of criteria and weights may be the firstproject stage at which the project widens the net by


“Setting healthcare priorities and rationing is anunavoidably messy, conflict ridden, ultimately tragicsocial process. Different societies will conduct theprocess in accord with their own political culture.But whatever approach a society chooses, it is notlikely to succeed without some form of deliberationamong the concerned stakeholders.”

– James Sabin, Fairness as a problem of love andthe heart: a clinician’s perspective on priority

setting43

seeking broader community input into what the criteriashould be and how much each criterion is worth, in partto address the relevance and empowerment conditionsof the Accountability for Reasonableness strategydescribed earlier. Such broader input may beparticularly useful in terms of values-based criteria –most notably about the inclusion of one or more equitycriteria within the priority setting process.

4.10 Identify Items for Inclusion in the

Set to be Prioritized

Earlier, during the mandate development phase, the setto be prioritized was determined; post-stroke rehabservices for instance. Aided by criteria developed in theprevious phase, it is now time for the project to identifythe items that will be included in the set and the itemsto be excluded from the set. For instance, the projectmay decide to exclude acute stroke treatment programsfrom the set, even though an argument can be madethat the quality of acute treatment helps determine theultimate feasibility of rehabilitation.

Traditional priority setting that is fuelled by newfunding generally determines what items will be in theset by issuing a call for proposals, often informed byprevious planning that has identified gaps and systemcharacteristics that have already helped define the setand helped define criteria for evaluating items in theset. For example, a children’s services planning projectmay have identified health services for children withdevelopmental disabilities as a priority, and may haveidentified inter-service coordination as a systemcharacteristic in need of improvement. The “set” forpriority setting could therefore be “health services for

children with developmental disabilities”, and evidenceof a proponent’s knowledge of and commitment toservice integration may be a requirement for anyproposal submitted when a call for proposals is issued.

Usually a call for proposals includes a statement of thecriteria that priority setters will use when they evaluateproposals, as well as weights assigned to each criterion.

When the mandate of a priority setting process includespriorities for service reduction or elimination, thetraditional call for proposals process is inadequate; fewproviders beat down the doors to get their programsreduced or eliminated.

The problem of identifying options for savings hauntsmany priority setting processes. However, identifyingitems that might lead to savings becomes easier if theentity that identifies these opportunities is also theentity that benefits from reinvestment of any savings.This means a save-and-invest priority setting processsuch as PBMA within a single agency is often morefeasible, even though it does not solve the problem ofinter-agency and inter-sector save-and-invest prioritysetting processes.

However, when it looks like inter-agency or inter-sectorcost savings are necessary, a priority setting process canuse the results of previous evaluation studies andprioritization processes to identify candidates for costsavings – and one should not discount the potential ofcarefully and diplomatically followed “hunches” as away to identify candidates.

4.11 Gather Indicator-Based Information

on Each Item

This is the grunt work associated with priority settingand is best left to the process management groupmembers whose skills allow them to gather andevaluate the quality of the information. They also have arole in telling the priority setting group about anylimitations to the information.


An alternative to detailed priority setting

as a way to find savings:

“One potato, two potato, three potato, four, five potato, six potato, seven potato more. Icha bacha, soda cracker, Icha bacha boo. Icha bacha, soda cracker, out goes you!”

4.12 Develop or Adopt a Scoring System

A scoring system should allow comparison of how twoor more items are valued for a given criterion, and howthe scores on one criterion can be compared to thescores on other criteria. They should also allow theweights assigned to each criterion to be factored intothe scores.

Scoring systems are often based on scales – the classic0 to 5 scale for instance – but scales have also beendevised that allow both positive and negative scores tobe recorded. A negative score, for instance, mightreflect an assessment that an item has lower thanaverage worth on a criterion, and a positive scoreindicates it has a higher than average worth.

Because inappropriate or simplistic scoring systems canbe traps rather than useful tools, the priority settingproject should incorporate statistical expertise withinits process management group, to ensure the mostappropriate scale is chosen and to ensure it is usedproperly and its results are interpreted accurately.

4.13 Apply the Criteria to the Items in the

Set Using the Scoring System

It helps if all members of the priority setting group givea final sign-off on the criteria and weights for eachcriterion, even though these were agreed to by thegroup earlier. Some members of the group may havehad second thoughts after their initial commitment tothe criteria and weights, and if they do not have achance to air any second thoughts and suggest ornegotiate changes, the outcomes of the process will besuspect. Such discussion prior to final sign-off isconsistent with the Accountability for Reasonablenessframework’s requirement that “There should be

accessible opportunities and processes to revise

decisions based on further evidence or arguments”.

This step should also allow a final run-through of thepreviously agreed-upon meaning of each criterion.

It is now time for the members of the priority settinggroup (the raters) to assign a value for each criterionfor each item in the set. This can be daunting if there isa large number of criteria and a large number of itemsin the set. A set of ten items to be judged on ten criteriaobviously requires 100 carefully considered decisions onthe part of each rater. This suggests two essentials:

1. Enough time for raters to make their ratingscarefully; and

2. Discussion among raters as the process proceeds sothey can benefit from each other’s perspectives.While raters need not be in the same place when theymake their ratings, there are advantages to the give-and-take dialogue that can take place when ratershave the chance to debate issues face-to-face duringthe rating process. In this sense, the raters’ work isless like a traditional voting process and more likethe deliberations of a jury.

4.14 Collate and Analyze the Scores and

Present the Provisional Results

The results of the process so far should be a prioritylist. This list, and the calculations that created it, shouldbe presented first of all to the priority setting group soits members can discuss whether the priority list makessense and is consistent with values.

4.15 Discuss, Negotiate and Adjust the

List if Necessary

At this stage it is important to remember that thestatistical tools and methods used to help create the listwere meant to be servants rather than masters of theprocess. If something on the priority list doesn’t lookright, the priority setting group should try to define whyit doesn’t look right and identify what must be done tomake it satisfactory.

Since any adjustments made to the list may be open tothe criticism that they introduced inconsistency in theprocess, the reasons for the adjustment must be clearlyspelled out.



4.16 Distribute the Priority List to

Stakeholders and Invite Appeals

This part of the process can help fulfil the thirdcondition of the Accountability for Reasonablenessframework – Revision: “There should be accessible

opportunities and processes to revise decisions based

on further evidence or arguments and there should be

a mechanism for challenges, appeals and dispute

resolution”. However, it also helps fulfil the Publicitycondition (“Decisions and their rationales and the

processes that led to the decisions should be made

publicly and conveniently accessible”) and theEmpowerment condition (“There should be efforts to

optimize effective opportunities for participation in

priority setting and to minimize power differences in

the decision-making context”).

4.17 Accept and Review Appeals and Make

Adjustments if Necessary

The project should be willing to accept both written andverbal appeals and should be willing to engage inrespectful dialogue with the appellants. It also helps ifthe priority setting team as well as the processmanagement team is front-and-centre in meeting withappellants. It may be wise at the very beginning of theproject to ensure that any people recruited to serve onthe priority setting team agree to participate in and actrespectfully during the appeals process, and also agreeto participate in the next stage – the hand-off to theimplementers.

In summary….

1. Establish a mandate for priority setting2. Establish the process management group3. Develop the project plan4. Develop the priority setting group5. Establish an Accountability for Reasonableness strategy6. Develop an appeals process7. Establish and implement a communications plan8. Understand the environments for priority setting

9. Identify criteria, weights and indicators 10. Identify items for inclusion in the set to be prioritized11. Gather indicator-based information on each item 12. Develop or adopt a scoring system13. Apply the criteria to the items in the set using the scoring system14. Collate and analyze the scores and present the provisional results15. Discuss, negotiate and adjust the list if necessary

16. Distribute the priority list to stakeholders and invite appeals17. Accept and review appeals and make adjustments if necessary18. Hand off the list of priorities to the implementers19. Evaluate the priority setting project

what to do

with what

you’ve done

getting

it done

getting

ready

4.18 Hand Off the List of Priorities to the

Implementers

This module has pointed out that priority setting is onestep – but not the final step – in a decision-makingprocess. Its utility lies in the extent to which action istaken based on the list.

Project personnel should therefore accept aresponsibility to communicate the priorities to decision-makers – a LHIN CEO or board for instance – and tooffer to explain or clarify the process and the outcomesof the priority setting process. These actions will notguarantee that the priorities enter the realm of decision-making and implementation, but they will help.

4.19 Evaluate the Priority Setting Project

Evaluation of the current project can help improvefuture priority setting projects. The evaluation can lookat the inputs and activities of the project to determine if

they unfolded as planned (this would constitute aretrospective formative evaluation component). It couldalso examine short- and long-term outputs such as theuptake of the priorities into the health care decisionmaking process and the flow of funds from lowpriorities to high priorities. Lastly, it could examineshort-term and long-term outcomes (in other words itwould be a retrospective summative evaluation), tellingus whether the priority setting process resulted inimproved levels of health for the beneficiaries of theprocess and more efficient and effective use of dollarsfor health care.

The steps in a priority setting process describedhere are not an inflexible prescription. Theyconstitute a checklist of things to be considered.Any given process may re-order them, omit someand add others or give greater emphasis to some ofthe steps.


Section 5: Challenges and Facilitators for Priority Setting Page 39

Because priority setting is a mix of sometimes overlywarm human aspirations and sometimes overly coolutilitarian numbers, it is fraught with challenges. Andlike all kinds of planning, it is a prelude to action ratherthan action itself. It can be rendered irrelevant if it doesnot serve decision makers or if decision makers do notlive up to the opportunities presented by defensiblepriorities.

Challenges also exist because the methodologies andinformation bases used in priority setting are often intheir infancy. The tools and attitudes necessary forpriority setting have not yet caught up to the need forpriority setting.

Mitton and Donaldson have identified a number ofbarriers to priority setting and have also identifiedfacilitators that help overcome these challenges.45

Section 5

Challenges and Facilitators for Priority Setting

“Priority setting in health is a complex process, atboth the macro and micro levels. There is no provenstandard approach, nor is there likely to be, giventhe tensions between individual and populationneeds, between primary and tertiary care andbetween equity and efficiency. It is widelyacknowledged that there is a tension betweenstrictly rational approaches to decision making andthe need to incorporate a range of socialperspectives.”

– Priorities for Action in Cancer Control 2001–2003,Government of Australia44

BARRIERS

TO EXPLICIT PRIORITY SETTING

lack of trust between stakeholders

service providers not on board

advisory panel lacking health economic knowledge and/or

allocation experience

decision maker/funder constraints preventing program

evaluation

discontinuity of personnel

too many administrative demands leaving priority setting as a

low priority activity

FACILITATORS

FOR EXPLICIT PRIORITY SETTING

senior level managerial and clinical champions

strong leadership

culture to learn and change

integrated management of budgets

resources earmarked for process itself and follow-up on

recommendations

built-in incentives for appropriate and efficient spending

Table 1: Barriers and Facilitators for Explicit Priority Setting (adapted from Mitton and Donaldson)

In addition to the pervasive barriers and promisingfacilitators identified by Mitton and Donaldson, severalmacro level elements have been suggested as essentialin establishing health care priorities in a Canadiancontext:

Page 40 Section 5: Challenges and Facilitators for Priority Setting

“We suggest four essential elements to any

successful effort to establish priorities.

The first is a Canadian 'constitution' for healthservices.

The second requires strengthening and broadeningthe evidence base for decision-making by guidingresearch efforts into evaluation of the costs andoutcomes of marginal changes in health serviceprovision.

Third, we need a decision-making framework suchas that provided by program budgeting and marginalanalysis to translate principles into practice.

Fourth, there needs to be appropriate incentive andinstitutional structures to ensure that recommendedchanges in the system are implemented.”

– Alan Shiell and Gavin Mooney, Framework forDetermining the Extent of Public Financing of

Programs and Services, prepared for theCommission on the Future of Health Care in

Canada, 200246

The body of knowledge about priority setting in healthcare suggests a number of tips that can help people whoare getting involved in priority setting.

1. Don’t expect perfection. The databases and toolsto support priority setting in the health system arestill “under construction”. But setting prioritiesimperfectly is better than not setting priorities at all,or setting them only as crisis responses.

2. Don’t rush the process. Take the time to do it well.A priority setting process must not only becompetent and open – it must be seen to becompetent and open, and a rushed process is often aless competent and more closed process.

3. Include people. Many engagement techniques canbe used to engage a broad pool of people at keystages in the process. Module 5 in the HealthPlanner’s Toolkit (Community Engagement andCommunication)i describes many of thesetechniques. Engaging people helps achieveempowerment, the fifth condition in theAccountability for Reasonableness frameworkdescribed earlier in this module.

4. Act like a jury. Priority setting processes fail if thepeople who set the priorities behave in arepresentational way, defending a particular turf.Expect the people who set the priorities todeliberate completely and dispassionately on bodiesof evidence that are balanced, comprehensive andaccurate, to produce verdicts that are realistic andintelligible and that promote defined communityvalues and goals.

5. Beg and borrow expertise and experience from

elsewhere. Unless you’re already an expert, spareyourself from unnecessary mistakes by finding outhow other people did it.

6. Console the losers. Priority setting is inherentlycompetitive - some people will lose. Take the time tohelp those whose ideas and programs are lowerpriorities, because your interest and advice will helpthem develop ideas and programs that may becomehigher priorities. And nobody needs an unnecessaryenemy.

7. Start modestly and build on success. While itmay be tempting to try to create the Great CanadianPriority List of All Things Forever, you are better offstarting with what is manageable and feasible, andusing that as the basis for more ambitious prioritysetting endeavours.

8. Evaluate what you’ve done. This adds to the know -ledge base about priority setting and this knowledgecan strengthen your next foray into prioritization.

9. Encourage evaluators to make use of your work

in their work. Much of the material you gather in apriority setting project can provide baseline datathat evaluators can use when they develop a plan toevaluate programs that were created or expandedbecause your project gave them high priority.

10. Be prepared to listen and negotiate. Rigidity isnot a virtue in a priority setting project. Even in juryrooms, much time is spent listening to differentperspectives and arriving at a verdict that everyonecan support.

Section 6: A Few Tips Page 41

Section 6

A Few Tips

“It is not so important to be serious as it is to beserious about the important things. The monkeywears an expression of seriousness which would docredit to any college student, but the monkey isserious because he itches.”

– Robert Maynard Hutchins

i The Health Planner’s Toolkit Module 5 – Community Engagement and Communication. [Online]. 2006 [cited 2008 Mar]; Available from: URL: http://www.health.gov.on.ca/transformation/providers/information/resources/health_planner/module_5.pdf

Section 7: Summary Page 43

Setting priorities is a necessary process if a healthsystem wants to make deliberative choice its preferredway of shaping its future. Faced with a host of growthdrivers, health systems must find ways to createpriorities for growth as well as priorities for cost orservice reduction to provide some of the fuel forgrowth.

But numbers do not set priorities. People set priorities,aided by their values, their communities’ values and bynumerical tools.

Getting the right people, and getting the right numbers,are the two key achievements that make a prioritysetting process a success.

Section 7

Summary

1. WordNet, Princeton University. Accessed April 11,2008, athttp://wordnet.princeton.edu/perl/webwn?s=priority.

2. Horizontal political prioritization: from words toaction. County Council of Östergötland Sweden.January 2004 Accessed April 11, 2008, athttp://e.lio.se/prioriteringscentrum/Engelskversion/Delrapport%20II%20Eng.pdf.

3. WHO priorities for the next five years. World HealthOrganization Director-General. Speech, LondonSchool of Hygiene and Tropical Medicine, October 7,2003. Accessed April 11, 2008, athttp://www.who.int/dg/lee/speeches/2003/londonschool_whopriorities/en/index.html

4. National Health Priority Areas. Australian Instituteof Health and Welfare, Government of Australia.Accessed April 11, 2008, athttp://www.aihw.gov.au/nhpa

5. Obermann K, Tolley K The state of health carepriority setting and public participation. Universityof York Centre For Health Economics, DiscussionPaper 154, June 1997. Accessed April 11, 2008, athttp://www.york.ac.uk/inst/che/pdf/DP154.pdf

6. Deber RB. Health Care Reform: Lessons FromCanada. Am J Public Health.2003;93:20–24;

7. C. David Naylor CD. Health Care In Canada:Incrementalism Under Fiscal Duress. Health Affairs1999; May/June: 9-26

8. Tuohy CH. The Costs of Constraint and ProspectsFor Health Care Reform In Canada. Health Affairs,2002; Volume 21, Number 3: 32-46

9. Flood CM, Choudhry S. Strengthening theFoundations: Modernizing the Canada Health Act.Discussion Paper #13, Commission on the Future ofHealth Care in Canada, August 2002.

10. 2006 Ontario Budget: Budget Papers. Toronto:Ontario Ministry of Finance. Accessed April 11, 2008, athttp://www.ontariobudget.ca/english/pdf/papers_all.pdf

11. Maioni A. Roles and Responsibilities in Health CarePolicy. Discussion Paper No. 34, Commission on theFuture of Health Care In Canada. November 2002.

12. Lomas J, Woods J, Veenstra G. Devolving authorityfor health care in Canada's provinces: 1. Anintroduction to the issues. CMAJ 1997;156(3):371-7.

13. Frelick K. Overview of The Proposed Local HealthSystem Integration Act, 2005 Miller Thomson LLPHealth Industry Practice Group, November 25, 2005;

14. Frelick K, White J, Liswood J. Local Health SystemIntegration Act, 2005 – Update and Issues. MillerThomson LLP's Health Industry Practice Group,January 10, 2006. Accessed April 11, 2008, athttp://www.millerthomson.com/mtweb.nsf/wnh?readform&PageID=mtte6a9bk6

15. Dewhirst K. Bill 36: Local Health System IntegrationAct, 2005: Preliminary Analysis. Centre forAddiction and Mental Health, December 2005.Accessed April 11, 2008, athttp://www.camh.net/Public_policy/Public_policy_papers/lhinleg_briefing.html

16. Prioritizing Health Services: A background paper forthe National Health Committee Wellington, NewZealand, October 2004. Accessed April 11, 2008, athttp://www.nhc.govt.nz/publications/prioritisation-backgroundpapers.pdf.

17. Mechanic D. Muddling Through Elegantly: FindingThe Proper Balance In Rationing. Health AffairsVolume 16 Number 5, September/October 1997.

18. Rachlis M. Defining Basic Services and De-InsuringThe Rest: The Wrong Diagnosis and The WrongPrescription. Canadian Medical Association Journal,May 1, 1995; 152(9): 1401-1404.

19. Ham C. Priority setting in health care; learning frominternational experience. Health Policy 42: 49-66;1997.

Page 44 References

References

20. WHA: Priority setting in the health sector.Ministerial Round Table at the 1999 WHA. AccessedApril 11, 2008, athttp://www.medicusmundi.ch/mms/services/bulletin/bulletin200304/kap02/10_WHA.html

21. The Swedish National Board of Health and Welfare.National guidelines for stroke care 2005: Support forPriority Setting. 2005. Accessed April 11, 2008, athttp://www.socialstyrelsen.se/NR/rdonlyres/76CD5474-58F3-4984-804E-24FB266B0DDC/7484/200710210.pdf

22. The Swedish National Board of Health and Welfare.Guidelines for Cardiac Care 2004. Stockholm:Socialstyrelsen 2004. Accessed April 11, 2008, athttp://www.sos.se/FULLTEXT/102/2004-102-4/2004-102-4.pdf

23. Kim E. The art of making change happen (ManagingChange). Physician Executive November 2 2003.Accessed April 11, 2008, athttp://goliath.ecnext.com/coms2/gi_0199-1204617/The-art-of-making-change.html

24. Dixon DL. Leadership and Culture Alignment. HealthProgress November-December 2000 Volume 81,Number 6. Catholic Health Association, U.S.A.

25. Brock DW. Health care resource prioritization andrationing: why is it so difficult? Social Research.3/22/2007. Accessed April 11, 2008, athttp://www.encyclopedia.com/doc/1G1-163198159.html

26. Brock DW. Health Care Resource Prioritization andDiscrimination Against Persons With Disabilities1(undated). New York University. Accessed April 11,2008, athttp://www.nyu.edu/gsas/dept/philo/courses/bioethics/Papers/DisabilitiesPaper.PDF

27. Draft Prioritization Policy: Background Paper.Canterbury District Health Board (New Zealand).September 2001. Accessed April 11, 2008, athttp://www.cdhb.govt.nz/corpbrd/Documents/Prioritisation2.doc

28. Daniels N, Sabin J. Setting Limits Fairly: Can WeLearn to Share Medical Resources? 2002. England:Oxford University Press

29. Gibson JL, Martin DK, Singer PA. Evidence,Economics and Ethics: Resource Allocation inHealth Services Organizations. Healthcare QuarterlyVol. 8 No. 2 2005. Accessed April 11, 2008, athttp://www.utoronto.ca/cpsrn/html/documents/EthicsEconomicsGibson_000.pdf

30. Wong-Rieger D. Social Values and Ethical DecisionMaking: How to Incorporate into Process ofResource Allocations for Rare Disorders?PowerPoint presentation to the CanadianAssociation for Rare Disorders. Undated

31. Hunter D. Am I my brother’s gatekeeper?Professional ethics and the prioritization ofhealthcare. Journal of Medical Ethics 2007;33:522-526

32. Prioritising Health Services: A background paper forthe National Health Committee. Government of NewZealand, Wellington, New Zealand, October 2004.Accessed April 11, 2008. athttp://www.nhc.govt.nz/publications/prioritisation-backgroundpapers.pdf

33. Bill 36 (Chapter 4 Statutes of Ontario, 2006). An Actto provide for the integration of the local system forthe delivery of health services. Royal Assent March28, 2006.

34. Lomas J. Reluctant rationers: public input to healthcare priorities. Journal of Health Service Researchand Policy 1997; 2:1–9.

35. Kitzhaber J. A Brief History of Health ServicesPrioritization in Oregon. Archimedes Movement website, 03/31/2006. Accessed April 11, 2008, athttp://www.archimedesmovement.org/node/23

36. Richardson J, McKie J. Empiricism, ethics andorthodox economic theory: what is the appropriatebasis for decision-making in the health sector?Social Science & Medicine 60 (2005) 265–275.

References Page 45

Page 46 References

37. Kumar S, Williams AC, Sandy JR. How do weevaluate the economics of health care? EuropeanJournal of Orthodontics 28 (2006) 513–51. AccessedApril 11, 2008, athttp://ejo.oxfordjournals.org/cgi/reprint/28/6/513.pdf

38. Guidelines for Economic Evaluation ofEnvironmental Health Planning and AssessmentVolume 1 – The Guidelines. Department of Healthand Ageing and e-Health Council, Government ofAustralia. January 2003. Accessed April 11, 2008, athttp://www.nphp.gov.au/enhealth/council/pubs/pdf/eee_guides1.pdf

39. Yazbeck A. An Idiot’s Guide to Prioritization in theHealth Sector. World Bank, 2002

40. Evidence-Based Planning. Module 3, The HealthPlanner’s Toolkit. Health System Intelligence Project,Ontario Ministry of Health and Long-Term Care.Accessed April 11, 2008, athttp://www.health.gov.on.ca/transformation/providers/information/resources/health_planner/module_3.pdf

41. Edwards D, Peacock S, Carter R. Setting Priorities inSouth Australian Community Health III: RegionalApplications of Program Budgeting and MarginalAnalysis

42. Mitton C, Donaldson C. Health care priority setting:principles, practice and challenges, in CostEffectiveness and Resource Allocation 2004,2:3doi:10.1186/1478-7547-2-3

43. Sabin J. Fairness as a problem of love and the heart:a clinician’s perspective on priority setting. Centerfor Ethics in Managed Care, Harvard Pilgrim HealthCare and Harvard Medical School. Accessed April11, 2008, athttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1114018

44. Priorities for Action in Cancer Control 2001–2003.Government of Australia, Cancer Strategies Group.Accessed April 11, 2008, athttp://www.health.gov.au/internet/wcms/publishing.nsf/Content/7223527350FCF94ACA257082000E7F81/$File/priorpre.pdf

45. Mitton C, Donaldson C. Cost Effectiveness andResource Allocation 2004 2:3 doi:10.1186/1478-7547-2-3

46. Shiell A, Mooney G. Framework for Determining theExtent of Public Financing of Programs andServices. Discussion Paper No. 6 Commission on theFuture of Health Care in Canada. July 2002.Accessed April 11, 2008, at http://www.hc-sc.gc.ca/english/pdf/romanow/pdfs/Shiell_E.pdf

47. National Institute for Health and Clinical Excellence.Press release, June 2, 2006. Accessed June, 2008 at:http://www.nice.org.uk/niceMedia/pdf/2006_025_Citizens_Council_Health_Inequalities_meeting.pdf

Appendix A: Using Matched Pairs to Determine the Weight of Criteria Page 47

The example described below assumes that priority setters have identified seven criteria. They now want tocompare the importance of each criterion against every other criterion, to arrive at weights for each criterion.

The starting point is a simple grid, with the criteria listed on each axis. Letters rather than numbers are used toidentify the criteria, to avoid confusion when one gets to the number part of the process.

Appendix A

Using Matched Pairs to Determine theWeight of Criteria

A B C D E F G

Severity Prevalence Competence Cost Feasibility Respect Public

of problem of problem benefit for clients visibility

A Severity

of problem

B Prevalence

of problem

C Competence

D Cost benefit

E Feasibility

F Respect

for clients

G Public visibility

A B C D E F G

Severity Prevalence Competence Cost Feasibility Respect Public

of problem of problem benefit for clients visibility

A Severity

of problem

B Prevalence

of problem

C Competence

D Cost benefit

E Feasibility

F Respect

for clients

G Public visibility

Using this grid, the priority setters start comparing each criterion against all others, starting with row “A”. In eachcell, the priority setters enter the letter for the criterion that is more important. For instance, when criterion A iscompared to criterion C, criterion A is considered more important. The results of this process are shown below.

The priority setters can now rank order the sevencriteria, based on frequency with which each criterioncame out on top when compared to other criteria:

The priority setters now have a decision to make aboutcriterion G – public visibility. This criterion never beatout any other criterion, so it has a value of “0”. If it isleft in as a criterion, any subsequent calculation ofweights will still give it a nil value. So the prioritysetters can:

• eliminate it as a criterion; or• give it a value of, say, “1”.

For the sake of this example, the priority setters decideto eliminate it, leaving six valid criteria.

The priority setters then sum the cells with valid valuesin them (excluding any cells that involved a comparisonwith criterion G – public visibility – since this has beenexcluded as a criterion):

Assuming 100 points of value to distribute as weights,the priority setters then divide 100 by the number ofvalid cells:

100 divided by 15 = 6.67

They then multiply this value by the number of timeseach criterion came out on top when compared to othercriteria, to produce a weight for each criterion:

Page 48 Appendix A: Using Matched Pairs to Determine the Weight of Criteria

A Severity of problem 6

D Cost benefit 5

F Respect for clients 4

B Prevalence 2

C Competence 2

E Feasibility 2

G Public visibility 0

Criteria Calculation Weight

A Severity of problem 5 x 6.67 33.35

D Cost benefit 4 x 6.67 26.68

F Respect for clients 3 x 6.67 20.01

B Prevalence 1 x 6.67 6.67

C Competence 1 x 6.67 6.67

E Feasibility 1 x 6.67 6.67

100.05

(round to 100)

A Severity of problem 5

D Cost benefit 4

F Respect for clients 3

B Prevalence 1

C Competence 1

E Feasibility 1

G TOTAL 15

© 2

008,

Que

en’s

Pri

nter

for

Ont

ario