2.1 values and criteria (t)

Craig Mitton & Francois Dionne | Priority Setting & Resource Allocation | 1

Values and criteria


Resource allocation decisions = value-based decisions

How much priority to disease prevention?

Modest benefits for many vs. significant benefits for a few?

Best outcomes vs. fair chances?

Rule of rescue?

Urgent vs. likelihood of success?

Individual choice vs. collective good?


How are values integrated in resource allocation decisions?•Priority setting processes (such as PBMA

which will be presented in this workshop) are based on formal comparisons of possible courses of action i.e. possible investments or disinvestments

•These comparisons involve the application of decision making or evaluation criteria

• It is in the selection of these criteria and of their weights that values are reflected


Multi-Criteria Decision Analysis (MCDA)

•The comparison part of a priority setting process such as PBMA uses MCDA

•MCDA is well-suited to contexts that involve multiple objectives (such as, for example, health improvement, equity, knowledge generation, staff safety…)


Goal of MCDA

•To produce recommendations regarding resource allocation decisions that are as relevant to decision-makers as possible

•Where relevance is defined in terms of the ability of the recommendations to reflect the real life setting in which the decisions have to be made (as opposed to a Random Controlled Trial, for example)


Principles of MCDA

•All relevant information is included in the process, acknowledging varying degree of quality

•Analysis can apply to a multitude of possible alternatives

•Objective is to produce a composite assessment of each possible alternative


Principles of MCDA (2)

•Ignoring a specific consideration is equivalent to assigning it a weight of zero i.e. there is no getting away from MCDA, the only choice is whether it will done explicitly or not, and by who

•Relevant considerations directly linked to the scope of the decisions required


MCDA Criteria

•Basis for process recommendations

•Must be specified at outset of process

•Must be explicitly weighted

•Operational enough to assess proposals

•Should not overlap (mutually exclusive)

•Need to be clearly defined

•Rating scale specific to each criterion


Criteria

Basis for process recommendations

Points to the importance of aligning criteria with organizational objectives and reflecting social values as the criteria will determine the direction of the resource allocation


Criteria

Must be specified at outset of process

Criteria do not vary by proposal for change. All criteria are applied to all possible courses of action considered. Allows comparisons of totally disparate courses of action


Criteria

Must be explicitly weighted

Relative importance of criteria should be identified.

Weighting can’t be avoided- no weighting is accepting equal weights

• Methods to elicit relative importance weight for criteria

• Relative importance should be plausible and intuitive


CriteriaOperational enough to assess proposals

Criteria are not selected on the basis of the availability of data or evidence but on the relevance to the decision making process. However, criteria must be defined in a way that allows the development of a measurement metric.


Criteria

Should not overlap (mutually exclusive)

Obvious overlap must be avoided. Some overlap will exist. The goal is to avoid obvious double counting yet remain pragmatic- the process is designed for decision makers not mathematicians.


Criteria

Need to be clearly defined

Criteria must be defined with clarity sufficient to ensure consistency of interpretation both by stakeholders submitting proposals for changes and by reviewers


CriteriaRating scale specific to each criterion

Rating is not a “meet/does not meet” a given criterion. Rating is about the impact in terms of each criterion. And impact measurement is more than “positive”, “null” or “negative”. Typically a scale with 3 or 4 grades is sufficient to capture all relevant information


Results of the application of the criteria

• Each possible course of action (investment or disinvestment) gets a score

• The score provides a single measure of each possible action’s net benefit or impact

• The scores depend on the criteria and their weights

• Scores are comparable across all courses of action because net benefits are measured on the same basis


Managing the evidence in MCDA

•Education on process and expectations on using evidentiary base

•All relevant information is used

•Business case template used for query submission adapted to the process


Summary•Priority setting involves values, it can’t be

avoided

•Objective is to make these value judgements explicit and transparent

•Values are reflected in the decision making or evaluation criteria and their weights

•Criteria, in turn, determine content of resource allocation recommendations

Technology

2.1 values and criteria (t)