Metrics for One Health benefits: key inputs to create an economic

evidence base

Minutes of the expert workshop taking place on

12th September 2013 in London

2

Contents

1. Aim and Objectives ........................................................................................................... 2

2. List of Participants ................................................................................................................. 3

2.1 Workshop organisers .................................................................................................... 3

2.2 Participants ....................................................................................................................... 3

3. Workshop agenda .................................................................................................................. 4

4. Literature review .................................................................................................................... 4

5. Workshop findings ................................................................................................................. 5

5.1 Break-out sessions part I ............................................................................................. 5 5.2 Break-out sessions part II ......................................................................................... 10

6. Conclusions ............................................................................................................................ 13

7. Future plans ........................................................................................................................... 13

1. Aim and Objectives The aim of the workshop was to agree on and recommend a set of metrics and associated methods that are most effective and practical for the standardized assessment of selected One Health benefits. The objectives were:

1) To describe the usefulness, advantages and disadvantages of metrics and methods available to measure One Health benefits (based on a literature review conducted before the workshop)

2) To select a set of metrics by consensus to recommend for application in the international (scientific) community

3) To draft a plan for further activities on this topic (e.g. publication, case studies, submit a proposal)

3

2. List of Participants

2.1 Workshop organisers

Dr Barbara Häsler Dr Laura Cornelsen Prof Jonathan Rushton Prof Bhavani Shankar

2.2 Participants

Dr Adamson David University of Queensland

Mrs Bennani Houda Royal Veterinary College

Dr Betson Martha Royal Veterinary College

Dr Bonnet Pascal Centre de Coopération International en Recherche Agronomique pour le Développement

Prof Carpenter Tim Massey University

Dr de Balogh Katinka Food and Agriculture Organisation of the United Nations

Dr Dehove Alain World Organisation for Animal Health

Dr Gamba Paul Egerton University, Njoro

Dr Kunkel Rebekah Centers for Disease Control and Prevention

Dr Hilde Kruse World Health Organisation

Ms Ladbury Georgia Glasgow University

Dr Lapar Lucy International Livestock Research Institute

Dr Lee Jeo Centre for Environment, Fisheries and Aquaculture Science

Dr Martin Jerry Development Alternatives Inc.

Dr McIntyre Marie University of Liverpool

Dr Paini Dean Commonwealth Scientific and Industrial Research Organisation Biosecurity Dr Pica Ciammarra Ugo Food and Agriculture Organisation of the United Nations

Dr Pradère Jean-Paul World Organisation for Animal Health

Prof Rich Karl Norwegian Institute of International Affairs

Dr Roth Felix Swiss Tropical and Public Health Institute

Dr Rubin Carol Centers for Disease Control and Prevention

Dr Ruegg Simon VetSuisse Faculty, University of Bern

Dr Scoones Ian Institute of Development Studies, University of Sussex

Dr Shaw Alexandra AP Consultants Ltd

Dr Vandersmissen Alain One Health Global Network/in his personal capacity

Prof Welburn Sue University of Edinburgh

4

3. Workshop agenda The workshop agenda was as follows: - Welcome and introduction

- Presentation of results from literature review and discussion

- Introduction to discussion points/case studies for the break-out groups

- Break-out sessions part I: Which metrics and methods available to assess One Health benefits are suitable or not suitable for selected One Health benefits and why? Description of advantages and disadvantages (participants divided into four groups)

- Facilitators report back from break-out groups part I, discussion and pre-selection of suitable metrics for selected benefits.

- Break-out sessions part II: Detailed definition of the selected metrics and methods, description of data requirements, technical requirements and recommendations for best-practice application for One Health questions.

- Facilitators report back from break-out groups part II, discussion and final selection of suitable metrics and methods to recommend for the use in One Health economics.

- Summary of workshop agreements, definition of next steps and close.

4. Literature review Prior to the workshop a literature review was conducted in the scientific and grey literature searching for articles that assessed the value of One Health or Ecohealth and to screen the metrics that were used in such studies. The search term combination used was [(”One health” OR ecohealth) AND (effectiv* OR efficien* OR useful* OR benef* OR profit OR utility OR gain OR advantage OR value OR “losses avoid*” OR “cost avoid*” OR “costs avoid*” OR “cost sav*” OR “costs sav*”)]. 621 articles were screened with 154 articles reaching full text review. Of these, 54 papers were found to discuss some benefit from One Health approach while 10 had made an attempt in measuring these. The benefits described in the literature with regards to One Health interventions range from rather specific measures focusing on one type of output (e.g. “15% reduction in costs”) to “all-inclusive” expected benefits referring to one or more hazard(s) that could be disaggregated into their respective single outcomes (e.g. “prevent, detect and combat future pandemics of H1N1”) and very broad benefits that are difficult to disaggregate (e.g. “ecosystem resilience”). The literature review specifically focused on articles that had defined themselves as taking a One Health or Ecohealth approach which, while offering specificity, is

5

also a limitation. Some literature does exist that assesses the benefits of collaborative actions across human, animal and environmental health but these sources have not defined themselves as One Health or Ecohealth. Workshop participants were invited to add any such literature they know of to the review and these studies are separately incorporated into the database of the existing literature.

Possible approaches and metrics from disciplines such as economic impact evaluation, environmental impact evaluation, and economic evaluation in health care were briefly discussed to provoke thought on the usability and transferability to evaluation of One Health benefits. Following the presentation, some points were raised and identified as important for discussion:

Metrics can operate at micro- (e.g. health) and macro-level (e.g.

knowledge, well-being). It would be useful to investigate metrics used in

ecology, geography, ecosystems services and other disciplines, which can

be broad and integrated and thereby link the different disciplines.

The focus should be on metrics that inform/persuade decision makers.

It is always tempting to look for “accessible data”, but it is worth thinking

differently and investigating new metrics.

How do you measure cooperation/interdisciplinarity/multidisciplinarity?

5. Workshop findings

5.1 Break-out sessions part I The participants were divided into four groups and each group was given a different group of benefits, identified from the literature review to consider during the first break-out session: The specific task was to discuss the detailed benefits associated with each group and potential metrics to measure these. The outcomes of the group discussions are summarized here. The summary also incorporates the feedback from the workshop participants on the relevant topics.

5.1.1 Group 1: Monetary and non-monetary benefits associated with protection of the environment, healthier ecosystems. The group discussed whether it would be more appropriate to apply a single “magic formula” when evaluating One Health interventions or whether context-specific metrics should be applied. To attract resources to One Health, policy makers need to be targeted and therefore it was considered that one “formula” would be more beneficial. The group recognized that they needed to identify what policy makers are looking for and what has previously been produced by looking at case studies from the likes of UN agencies and NGOs. They highlighted the risk of neglecting existing discussion on and products from “ecosystems

6

services” and advocated checking existing policy-level eco-environment regulation. There was discussion around and recognition of the ‘old debate’: does health mean “absence of disease”? A framework was suggested in which policy domains/drivers are identified in relation to relevant themes (e.g. climate change, water security and management, biodiversity, airborne diseases), then processes or causal links (i.e. pathways), followed by metrics which may be specific or universal (Figure 1). The group suggested that millennium development goals should be considered when looking for a suitable metric, suggesting a One Health index or the use of existing valid metrics (e.g. DALY, monetary measure) that could be combined for One Health approaches.

Policy domains/Ecosystem drivers

Processes,Causal linksCase specific oruniversal

Metrics

HowWhoWhere

Climatechange

Water security& management

DistributionAccess

Quality

ManagementConflict prevention

Link with soil fertility/productivityBiodiversity

Habitatfragmentation

Airborne diseasesBio-hazards

IDEAL:1 common indexper driver

- Specific- Universal- Case studies- Agreed- Exist- OH- Common/can be shared

Figure 1: Overview of outcome of group 1 discussions

It is assumed that the outcome of the One Health approach is what is being measured. There is a need to start with standard/compatible metrics so they can be compared across disciplines. The group concluded that it was important to involve policy makers in the process and that existing approaches to measurement were still valid. It was pointed out that a different set of metrics may apply according to the unit of observation (e.g. household level, farm level, national level). The selection of drivers and processes was deemed important and the most suitable should be chosen on its ability to demonstrate One Health added value. It was also stated that collaborations are possible and that data already exists or is easily collected.

5.1.2 Group 2: Monetary and non-monetary benefits associated with higher quality or quantity of information, data; better knowledge, skills, innovation. The group discussed the value of surveillance and linking human and animal data. They noted that having knowledge about something enables action but questioned whether it would be possible to measure a link between information

7

and effective action. Further, there would be a trade-off between the cost of information and the value of the information. Data mining was discussed and it was suggested that this might enable new risk factors to be identified or be an effective way of making counterintuitive discoveries. The need for a link with policy makers and to identify what influences policy was discussed. It was suggested that network maps and stakeholder analysis could be employed as a method to measure interdisciplinarity. The importance of active investigations and the cost of acquiring knowledge as well as the need to prioritise information were also discussed. Further, the group highlighted the potential benefit of better insurance based on innovation and improved surveillance through technology. The possibility of having an index for data sharing and communication was discussed. It was concluded that the health impact would be the most obvious measure to look at but that this would not be adequate by itself.

Table 1. Benefits discussed by group 2

Benefit Metrics Increased jobs, trade, productivity Economic impact,

Cost effectiveness (index), Returns on investment

Communication & information sharing (e.g. confidential information)

Measuring interdisciplinarity - network analysis or stakeholder analysis e.g. in Zambia

Better agricultural/ livestock insurance – link to improved disease outcomes

No. of insured farmers, informal insurance mechanisms (micro-level)

Reduced incidence/prevalence of disease, Behaviour change

KAP – communities, specialists/practitioners, use questionnaires or direct observation

Improved surveillance through technology – syndromic surveillance e.g. tablets, mobile phones

Uptake of technology, return on investment – disease avoided per unit cost

Integration of data collection (→ improved surveillance and control)

Measure of integration of data collection

Improved surveillance/control through data mining

Amount of data analysed, New risk factors identified

Wildlife health Biodiversity index, tracking of species (animals, insects) – chipping, GPS etc.

5.1.3 Group 3: Monetary and non-monetary benefits associated with improvement in human health and well-being and animal health and welfare

Group 3 identified a long list of possible benefits from taking the One Health approach in comparison to ‘traditional approaches’ that all lead to an improvement in health for either or both humans and animals and which eventually lead to economic gains. From this list following benefits were identified as key:

- Reduction in disease prevalence (human and animal, both infectious and

non-infectious) which has many positive effects, for example stronger

workforce, increased life expectancy, more income, facilitated

8

trade/access to markets, improved fertility, more time for production or

leisure as less time is spent caring for sick animals or people, improved

education as children can attend school and pay attention, decreased

pressure on land use

- Higher availability of food, safer food and better quality of food, improved

food security

- Improved access to services through shared resources (e.g. Laboratory

equipment, assess to diagnostic services)

- Improvement in animal welfare (including animal health), reduced animal

suffering, protection of animal rights

- Female empowerment; reduction in socio-economic and health

inequalities

A brief discussion followed on what metrics could be used to measure these benefits (Table 2). Table 2. Benefits discussed by Group 3

Benefit Metrics: Reduction in disease

prevalence (human and animal)

Disease incidence/prevalence; life expectancy;

QALY/DALY; vaccination rate (animals); production

(animals); morbidity/mortality

Improved food security Household hunger score, anthropometrics, calories

per capita, productivity of animals, production yield

Improved access to services Number of visits to health practitioner; number of

tests per population, time taken from suspicion of

disease to result of test (pos. or neg. diagnosis);

distance to services, travel time to services,

availability of services (i.e. clinics, labs) per sq/km or

per population

Improved animal welfare Animal mortality and morbidity (animal health);

access to feed/feed availability; productivity.

Consensus that there is a lack of good metrics to

measure animal welfare and that the data availability,

especially in poorer countries, is very limited.

Reduction in socio-economic

and health inequalities

GDP, income, UNDPI human development index

From the feedback of participants it was also noted that it is crucial to make a difference between outputs and impact of these outcomes. For example the impact of vaccinated animals is the same regardless of it being done through One Health or non-One-Health approach. However, One Health approach can lead to a much larger number of animals being vaccinated. Therefore it is important to measure the outputs and develop ways to aggregate the various outputs in one only indicator. Further the question was discussed about what needed to be valued and to identify where it is easiest to show the added value of One Health.

9

5.1.4 Group 4: Monetary and non-monetary benefits associated with early (earlier) detection of threat and timely, effective or rapid response; better/improved/more effective disease control and/or biosecurity measures

The key benefits discussed in this group were:

- Cost-savings by merging resources - Reduced time from occurrence of an event to a decision - Local, national and international collaboration - Number of human or animal cases detected and avoided - Trust - Change in income

A big advantage of One Health that was described was the “broader defense” aspect, i.e. One Health keeps everything on the radar which allows us to be better prepared. The group firstly considered how sectors could collaborate more effectively. Different possibilities for measuring collaboration were discussed; for example the number of people from different sectors working on a programme, shared resource allocation, integrated reporting channels, number of reports from different stakeholders, number of contacts/meetings, number of people we know who work in the other field. It was recognized that it is easier to calculate costs than to measure benefits and that the assessment of cost-savings would be a first step. The difficulty in measurement was considered for time of response to an outbreak (e.g. consider time from introduction to an action/intervention or time from introduction to a decision and can this be a useful for proxy for better animal or human health?). Surveillance was identified as the key to better and earlier detection, which consequently leads to reduced negative impacts on animal and human health and fewer trade barriers. It was suggested that transparency in surveillance allows countries to be scored and ranked. It was agreed that trust between sectors was imperative for One Health, with one indicative definition of trust being:

Trust = Credible x Responsiveness x Intimacy

Self interest The group considered that a starting point for a One Health intervention might be a qualitative evaluation and quantification of resources needed. It was decided that it would be necessary to examine competencies at the animal: human interface and that more indicators were required for assessors. It was concluded that better coordination, collaboration and free exchange of information between services is required.

5.1.5 Morning plenary discussion The groups came together after the first break out session to feedback and discuss topics in further detail. It was noted that at the 2013 World Assembly meeting of the OIE the Gates Foundation has recognised the Evaluation of

10

Performance of Veterinary Services (PVS) Pathway as an appropriate tool for measurement (more than 70% of OIE member countries have asked for an evaluation) and it is useful for management decisions. It was found that more collaboration between public and animal health is needed. Difficulties in compiling costs were highlighted, although some progress is made through the increased collaboration between WHO (IHR costing tool) and OIE (PVS Gap Analysis, the PVS costing tool). It was also suggested that governments know how much they spend but not how much the private sector spends. The Living Standard Measurement (public or private) measures amount spent on livestock/animal health in developing countries. It was suggested that National trade associations (e.g. poultry) compile information on total value of vaccines, feed etc. which could be utilised. Distortions in the way in which money is spent was outlined giving the amount spent by Britain on TB or America on antibiotics as examples.

While WHO and OIE train people from different sectors to work together for food safety/control zoonoses (OIE national focal points for food safety and regional seminars for national focal points), it can be difficult to measure related outcomes. It was suggested that a One Health Index could be applied to different countries to see how good they are at applying a One Health approach. Such an index would require a combination of different metrics that allow measurement of One Health benefits. It was questioned whether this needed to be standardised in order to compare outcomes of different systems (is a “one size fits all” approach possible?). There was discussion about how to validate such an index. It might be beneficial to study early adopters of interdisciplinary work in different disciplines. The importance of using metrics useful for decision-makers was highlighted. Finally, the need for and benefit of a One Health approach was questioned and it was agreed that an evidence base was required in order to test hypotheses.

5.2 Break-out sessions part II A secondary break out session took place where participants again split into four groups. Because of the outcomes of the morning discussion, the aims of this session were changed and groups were asked to address specific questions related to their discussions in the earlier break-out session.

5.2.1 Group 1: How can metrics from different disciplines be combined to evaluate One Health benefits? The group decided that the appropriate method to measure benefits was to disaggregate the issues associated with a particular problem as much as possible, doing this for different domains, departments and regions and examining trade-offs between them. The group used the example of development of dams in Mekong to illustrate how the method would work. They determined that dam development would reduce vector borne diseases, having positive social and cultural effects, but would reduce protein (fish, insects, rice) and biodiversity, which would have negative environmental, social and cultural impact. The resulting generation of electricity would have positive social and cultural but negative environmental effects.

11

5.2.2 Group 2: What is the value of communication and information and data sharing? Much of the group discussion focused on methods of measuring amount of communication and information and data sharing across disciplines and between ministries. One suggested approach was to look at resource allocation between ministries, where donor money went and whether/how money was actually spent. In terms of data sharing (e.g. disease reports, risks, reactions), it was proposed that one could assess speed of availability, circulation, where data are published and whether data are accessed. To determine level of communication between ministries, departments etc., the group thought that formal and informal networks could be investigated, as well as public records of meetings to determine how often they take place, who attends and the decision-making power of participants. Further discussion of the use of mobile phones, web searches and social networking to predict outbreaks and follow disease spread took place. The group also examined whether historical analysis of outbreak management and intersectoral collaboration could be carried out. Finally, it concluded that the ultimate value of communication and information and data sharing lies in their positive impact on health outcomes. From feedback to the workshop the following suggestions were made that could help measure collaboration (which can also be identified as an input so a care has to be taken to avoid double-counting):

(i) Level of collaboration between the various actors involved in One Health as measured by simple indicators such as number of meetings; phone calls; email exchanges; etc. (these are very rough proxies of collaboration and not necessarily easily available); (ii) Budget allocated to the various ministries involved in the One Health approach. The more equally the budget is distributed across the various ministries, the more the benefits for society; (iii) Household expenditures on public and private human and animal health services or drugs which will give an indication of the share of public and private investment.

5.2.3 Group 3: What values have the most value? How do we prioritise the various possible metrics to inform decision making? What is the right amount of information? Which values translate to money? Which should be prioritised? What should we do if there is insufficient data? The group decided that perfect data will never be available so an appropriate approach was to start with whatever data was available. For this reason pilot projects to collect the right data are especially important. These pilots should be in a case-control or randomized controlled trial design to allow measuring the added benefit of taking the One Health approach. It was deemed that monetary cost is the best metric for policy makers because “money talks”. The group

12

thought it would be possible measure many things in monetary terms e.g. the cost of disease, outbreak control, loss of labour productivity, trade, tourism. However, links to food security, and social stability, for example, would be harder to demonstrate. Also, some important data is not available in many of the poorer countries, especially relating to the burden of livestock and fish diseases (contagious and non-contagious). What complicates things further is the time dimension of the realisation of costs and benefits. If these take place in the medium or long run, then discount factors for costs and benefits are needed. Another important factor is the unit of measurement which should be as standardised as possible (e.g. household, individual, herd, zone, compartment, etc.) and which determines data need. Therefore, it should be agreed upon before pilot projects take place so that the right data are collected. A system could be created that allows converting common and usable metrics into one single metric or a measure using conversion rates or coefficients. Further discussion centred around the fact that a One Health platform requires an investment cost, therefore an economic justification is required, which might be easier in countries which have had epidemics, although there is the risk of “pandemic fatigue” or “One Health fatigue”. The group concluded that the main pre-requisites of One Health working are not so costly, however, and entailed good working relationships, knowing and trusting each other and establishing good communication, systems of information sharing and a chain of command.

5.2.4 Group 4: What is the value of cross-sectoral collaboration? The group highlighted the importance of private-public partnerships and the fact that the private sector needs to be included in any collaborative effort. It was mentioned that the HLTM Mexico meeting in November 2011 identified common objectives and common benefits to public and private sectors. It was agreed that equitable sharing of resources is essential to promote cross-sectoral collaboration, as well as high-level commitment, e.g. from policy makers. Decision-makers in the process would both include those who allocate budgets and those who implement initiatives, i.e. work with a budget. In addition, incentives are necessary to stimulate collaboration and this should occur at a national and local level. It was proposed that cross-country comparisons are needed and that a common methodology across countries is required. Instead of defining a set of standardised metrics to apply, a better approach may be to develop a best practices guide which would include definition of the context (e.g. identification of all stakeholders, their needs, and how they cooperate), the objective of the One Health initiative (what is to be achieved with the cooperation, performance targets), identification/description of benefits referring to the context (e.g. existing value system), selection/determination of appropriate metrics, assessment of benefits, and common reporting processes. It was highlighted that a form of comparison would be crucial to assess the value

13

of the collaboration, for example a design including a comparison “before and after” implementation of a One Health/collaborative approach. It was pointed out that human and animal health sectors need different reasons to make decisions on resource allocations. Therefore it is necessary that metrics reflect the objectives of both sectors. From the feedback to the group it was also suggested that countries could be ranked according to their level of One Health approach as many countries de facto, somewhat use it. This can be done by administering staff in the various ministries with questionnaires (qualitative / quantitative).

6. Conclusions It might not be necessary to “re-invent the wheel” and might be possible to

adopt and/or adapt metrics used by ecology, ecosystems, human and animal

health services or others for measuring One Health benefits.

Development of a One Health index would allow for the combining of existing

(validated) metrics from different disciplines.

Measuring benefits in cost or monetary terms might be the best metric for

influencing policy makers, although this can be challenging for measurement

of some benefits.

Common/agreed methodologies and best practice guidelines should be

developed and implemented to allow comparisons between countries and

institutions.

Equitable sharing of resources, communication, exchange of information,

collaboration and interdisciplinarity are key to the success of One Health.

It is important to make sure that both the public and private sector are

involved in assessment of One Health benefits.

Processes should be evaluated through the schematic of impact evaluations

that involve participatory methods from all relevant stakeholders. Without

common understanding and agreement it is difficult to ensure that the right

data is consistently collected and collaborations last in the long-run. These

evaluations have to find the right balance with being context specific and also

allow for comparisons across projects – information that is crucial for

ministers who allocate budgets.

Need to consider the input side of the equation (resource use) even though

more work is done in this area in comparison to benefit measurement.

7. Future plans A plan was formulated to submit a proposal to a suitable funding body. This would allow the establishment of a formal interdisciplinary research network to continue developing a framework and creating a measurement system for the evaluation of One Health.