AMC MONITORING AND EVALUABILITY STUDY FOR THE AMC FOR PNEU… · advance market commitment for pneumococcal vaccines report of the monitoring and evaluability study november 13, 2008

ADVANCE MARKET COMMITMENT FOR

PNEUMOCOCCAL VACCINES

REPORT OF THE MONITORING AND EVALUABILITY STUDY

November 13, 2008

i

CONTENTS

Contents.............................................................................................................................. i

Foreword .......................................................................................................................... iii

Executive Summary........................................................................................................ iv

List of Acronyms............................................................................................................. ix

1. Introduction ........................................................................................................... 1

1.1 Purpose of the Report ....................................................................................... 1

1.2 Methodology...................................................................................................... 2

1.3 Objectives and Goal of the Pneumococcal Vaccine AMC ........................... 2

2. M&E Strategy......................................................................................................... 5

2.1 Scope of the AMC M&E ................................................................................... 5

2.2 M&E Issues......................................................................................................... 6

2.3 Data Sources..................................................................................................... 12

2.4 Components of M&E Strategy....................................................................... 13

2.5 Monitoring and Evaluation Matrix............................................................... 24

2.6 Proposed Counterfactuals.............................................................................. 34

2.7 M&E Challenges .............................................................................................. 40

3. Proposed M&E Implementation Plan .............................................................. 46

3.1 M&E Implementation Structures and Required Resources ...................... 46

3.2 Costs and Timeframe for M&E...................................................................... 48

3.3 M&E Timeframe .............................................................................................. 52

Appendix A: List of Interviews.................................................................................... 55

Appendix B: Definitions................................................................................................ 56

ii

Appendix C: AMC Context .......................................................................................... 59

1. Introduction ......................................................................................................... 59

2. Pneumococcal Vaccine Context ........................................................................ 61

Appendix D: Pneumococcal Vaccine Profile.............................................................. 69

1. Conceptual Model of the Pneumococcal Vaccine AMC................................ 69

2. AMC Stakeholders .............................................................................................. 73

3. AMC Beneficiaries .............................................................................................. 83

4. AMC Funding Arrangements and Resources................................................. 84

5. Pneumococcal Vaccine AMC Logic Model ..................................................... 85

5.1 Activities and Outputs.................................................................................... 88

5.2 AMC Outcomes (Expected Results) ............................................................. 97

iii

FOREWORD

I am pleased to introduce this monitoring and evaluability report for the Advance Market Commitment (AMC) for Pneumococcal Vaccines. The report was commissioned by the Monitoring & Evaluation Subgroup of the AMC Donor Committee (comprising the governments of Italy, the UK, Canada, Russia and Norway, and the Bill & Melinda Gates Foundation) in August 2007 and provides an in‐depth and pioneering monitoring and evaluation framework for an initiative worth USD $1.5 billion.

The monitoring and evaluation proposals contained in this report have been designed to enable the effectiveness, efficiency and eventual impact of this pilot AMC to be measured over time and to assist with its effective management.

Extensive consultations took place prior to the framework’s finalisation. Comments were sought from all donors involved in this initiative and detailed consultations held with GAVI‐eligible countries, civil society, industry and experts in Washington and Geneva. This was carried out to ensure all relevant stakeholders’ opinions and views could be taken into consideration in order to develop a credible framework to support ongoing monitoring and rigorous evaluations.

On behalf of the Evaluation Departments of both the UK’s Department for International Development and the Canadian International Development Agency, who jointly funded this piece of work, I would like to extend my gratitude to the Monitoring and Evaluation Subgroup that managed and oversaw this exercise. Members included Yohanna Loucheur (CIDA), Abdallah Bchir (GAVI), Tania Cernuschi (GAVI), Chris Collinson (DFID) and James Bianco (DFID).

I would also like to take this opportunity to thank Steve Mendelsohn and Sheila Dohoo Faure of Goss Gilroy Inc, as well as HLSP and other members of the consulting team who approached this work with both diligence and rigour.

Nick York Deputy Director Head of Evaluation Department, DFID November 13, 2008

iv

EXECUTIVE SUMMARY

This report recommends a monitoring and evaluation (M&E) framework for a pilot Advance Market Commitment (AMC) for a pneumococcal vaccine. It was prepared for the Monitoring and Evaluation Sub‐Group, on behalf of the AMC Donors’ Committee.

The report explores the M&E issues for the pilot AMC, develops an M&E framework and strategy with specific activities and timeframes, identifies the necessary indicators and data sources, and estimates the anticipated costs for an effective M&E program.

The M&E activities recommended in this report will permit donors to test the effectiveness of the AMC mechanism, thereby providing evidence on whether and to what extent the AMC mechanism will likely be effective as a mechanism for other interventions.

Given the importance of being able to evaluate the design and implementation of the pilot AMC, the issues for evaluation have been grouped into four categories:1

• Rationale issues: These issues deal with the need for the AMC initiative, and whether there were alternatives and perhaps more cost‐effective ways of achieving the same objectives;

• Design issues: These issues deal with the fundamental concept of the AMC and the terms, conditions and mechanics of the AMC concept. The goal here is to offer guidance on how potential future AMCs could be structured, and to follow up on issues considered during the Economic Expert Group/ Implementation Working Group (EEG/IWG) consultation process;

• Process issues: These issues deal with the overall efficiency of the AMC structures, whether the governance and oversight processes were adequate and whether timelines and budgets were respected; and

1 Exhibit 1 cross‐references these “issues” with the key Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) evaluation criteria – relevance, efficiency, effectiveness, impact and sustainability.

v

• Outcome issues: These constitute the core issues of the evaluation over the longer term and deal with whether the targeted results were achieved. The core issue for the initiative is whether the AMC stimulated additional/new pneumoccocal vaccine development, new vaccine production and the introduction of pneumococcal vaccines tailored to the needs of Global Alliance for Vaccines and Immunization (GAVI)‐eligible countries.

The scope of the recommended AMC M&E framework does not include issues that relate to evaluating the effectiveness of the complementary activities of other stakeholders (e.g. whether GAVI, the World Health Organization (WHO) etc. carried out their roles effectively). However, the implementation of these activities will need to be monitored throughout the implementation of the AMC. To the extent that information is available on the implementation of the complementary activities, it can also be seen as explanatory – i.e., what has contributed to, or detracted from, the achievement of the AMC objectives or long‐term goal.

A variety of data sources are proposed for the M&E of the pilot AMC. They include:

• Literature and document review – This would include a review of the literature reflecting the context for, and concept of, the AMC, as well as documents on the implementation of the AMC;

• Surveys/interviews – The framework proposes a series of attitude and intention surveys and interviews of key stakeholders (including past and current representatives of the EEG, IWG, donor agencies, industry, GAVI‐eligible country stakeholders including health authorities) on the AMC processes and outcomes;

• Industry data – In order to assess the impact of the AMC on industry behaviour, data will need to be collected from industry directly, including firms serving GAVI‐eligible countries and potential new entrants, or from existing public sources of data on the pharmaceutical industry; and

• Immunization and health data – This includes the use of country data that is reported regularly by the United Nations Children’s Fund (UNICEF) and WHO, including immunization coverage, dates of introduction of new vaccines, mortality/morbidity data, and special studies relevant to pneumococcal disease generated by the WHO Global Burden of Disease Project. WHO prepares annual GAVI progress reports (available on the WHO‐

vi

Immunization Coverage Estimates and Trajectories (ICE‐ T) database that includes coverage data, disease burden estimates and projections of deaths averted.

The M&E framework described in this report has four components:

• A baseline study to determine the point of comparison for future M&E, including the development of counterfactuals;2

• Annual monitoring of both the AMC and the complementary activities required to support the public health goal of the AMC;

• A process evaluation about two years after the launch of the AMC to assess whether the AMC mechanism is working as expected and to obtain information on the AMC design issues; and

• Outcome evaluations every four years after the signing of the first AMC Supply Agreement. The outcome evaluations will focus on assessing the extent of achievement of the AMC objectives, as well as addressing design issues, as required. They will include an analysis of the counterfactuals. The first outcome evaluation is expected in 2013, provided a supply agreement is signed in late 2009.

Specific structures and dedicated resources are required to ensure satisfactory implementation of the M&E activities. It is proposed that the AMC Secretariat be responsible for AMC monitoring, including:

• Managing the monitoring activities, with accountability to the GAVI Board for these activities;

• Managing the independent contractors required for the implementation of the AMC baseline study; and

• Producing annual reports, as identified in the requirements for annual monitoring.

2 Counterfactuals are a simulation of what would have happened without policy intervention, or if the policy intervention had taken a different (but specified) form. Assessing the difference between the situation under the AMC and the counterfactuals contributes to the assessment of impact of the AMC.

vii

It is proposed that the GAVI evaluation unit be responsible for managing the evaluations. An independent steering committee (including all key stakeholders) should provide oversight and report evaluation results to the IAC and Stakeholder Committee.

Both the AMC Secretariat and GAVI’s evaluation unit will require adequate human and financial resources. Details on these requirements are provided in the report.

The table on the following page provides an overview of the expected timelines and costs of the proposed M&E framework.

viii

Summary of Monitoring and Evaluation Deliverables and Associated Costs

Year

Type of M&E

Ann

ual m

onito

ring

repo

rts

Surv

ey o

f ind

ustry

re

pres

enta

tives

Surv

ey o

f cou

ntry

le

vel h

ealth

aut

horit

ies

Dev

elop

men

t of

mod

els f

or

coun

terf

actu

als

Eval

uatio

n of

im

plem

enta

tion/

pr

oces

s and

des

ign

issu

es

Indu

stry

leve

l dat

a co

llect

ion

and

anal

ysis

Cou

ntry

leve

l dat

a co

llect

ion

and

anal

ysis

Eval

uatio

n of

des

ign

and

outc

ome

issu

es

Cos

t-eff

ectiv

enes

s st

udy

Costs Total for year

Annual monitoring $150,000 $100,000 $100,000

2009 Baseline study

$75,000 $425,000 2010 Annual monitoring $150,000 $150,000


2011 Process evaluation

$150,000 $550,000 2012 Annual monitoring $150,000 $150,000 2013 Annual monitoring $150,000 Outcome evaluation $100,000 $100,000 $150,000 $500,000 2014 Annual monitoring $150,000 $150,000 2015 Annual monitoring $150,000 $150,000 2016 Annual monitoring $150,000 $150,000


2017 Outcome evaluation

$150,000 $500,000 2018 Annual monitoring $150,000 $150,000 2019 Annual monitoring $150,000 $150,000 2020 Annual monitoring $150,000 $150,000

Annual monitoring $150,000 $100,000 $100,000 $150,000

2021 Outcome evaluation

$100,000 $600,000

Total $3,775,000

ix

LIST OF ACRONYMS

ADIP Accelerated Development and Introduction Plan

AMC Advance Market Commitment

ARI Acute Respiratory Infections

CDC Center for Disease Control

CRA CRA International

CVI Children’s Vaccine Initiative

DAC Development Assistance Committee (of OECD)

DALYs Disability Adjusted Life Years

DC Donor Committee

DCVMN Developing Countries Vaccine Manufacturers Network

DPT Diphtheria, Pertussis and Tetanus

EEG Economic Expert Group

EMEA European Medicines Agency

FDA Food and Drug Administration

FIRM Financial Implications and Risk Model

EPI Expanded Programme of Immunization

GAO Government Accountability Office

GAVI Global Alliance for Vaccines and Immunization

GDP Gross Domestic Product

GSK Glaxo Smithkline Ltd

HIV Human Immunodeficiency Virus

Hib Haemophilus influenzae type B

IAC Independent Assessment Committee

ICE‐T Immunization Coverage Estimates and Trajectories

IFPMA International Federation of Pharmaceutical Manufacturers & Associations

x

IWG Implementation Working Group

JHU John Hopkins University

M&E Monitoring and Evaluation

NGO Non‐Government Organization

OECD Organisation for Economic Co‐operation and Development

PAHO Pan‐American Health Organisation

PATH Program for Appropriate Technology in Health

PCV Pneumococcal Conjugate Vaccine

PCV 7, 10, 13 Pneumococcal Conjugate Vaccine that addresses 7, 10 or 13 serotypes respectively

PneumoADIP Pneumococcal Accelerated Development and Introduction Plan

PYLL Potential Years of Life Lost

R&D Research and Development

ROI Return On Investment

RotaADIP Rotavirus Accelerated Development and Introduction Plan

SAGE Strategic Advisory Group of Experts

SEC Security Exchange Commission

TPP Target Product Profile

UK United Kingdom

UNDP United Nations Development Programme

UNICEF United Nations Children’s Fund

USA United States of America

USAID United States Agency for International Development

WHO World Health Organization

1

1. INTRODUCTION

This chapter presents the overall structure of the report and the objectives of the Advance Market Commitment (AMC) for a pneumococcal vaccine.

1.1 PURPOSE OF THE REPORT

The purpose of this report is to recommend a framework, or plan, for the monitoring and evaluation (M&E) of the AMC. The framework will guide the activities of those involved in the M&E of the AMC. The report explores the M&E issues for the pilot AMC, develops a framework and strategy with specific activities and timeframes, identifies the necessary indicators and data sources, and estimates the anticipated costs for an effective M&E programme.

This report is divided into three sections:

• Section 1.0 is the introduction, including the methodology for developing the M&E strategy and the objectives of the AMC for pneumococcal vaccines;

• Section 2.0 outlines the M&E strategy, including identifying the key issues for M&E, providing an overview of the M&E strategy, describing the indicators, data sources and the types of M&E studies and proposing the counterfactuals to be used for assessing the impact of the AMC; and

• Section 3.0 describes the M&E implementation plan, including its specific components, timeframes and estimated costs.

There are four appendices:

• Appendix A: a list of people interviewed for the development of the M&E strategy;

• Appendix B: a list of definitions of key terms in the strategy;

• Appendix C: a description of the context of AMC initiative; and

• Appendix D: a profile of the AMC initiative.

2

1.2 METHODOLOGY

The development of the M&E framework was based on:

• An extensive review of AMC and other documents (including documents on the Global Alliance for Vaccines and Immunization (GAVI) website);

• Interviews and consultations with key AMC stakeholders, technical and programme experts (see Appendix A); and

• Working sessions with members of the M&E Sub‐Group.

This framework was updated in July 2008, taking into account the results of the Economic Expert Group (EEG) Report, which was completed April 1, 2008 and the results of an Implementation Working Group (IWG) Report released July 10, 2008. The latter currently represents the latest recommendations on the implementation of the AMC, provided by the IWG.

1.3 OBJECTIVES AND GOAL OF THE PNEUMOCOCCAL VACCINE AMC

The objectives of the pneumococcal vaccine AMC, as agreed by the AMC Donor Committee (DC) are3:

1. To accelerate the development of pneumococcal vaccines that meet developing country needs (e.g. serotype composition and vaccine presentation) as specified in the Target Product Profile (TPP);

2. To bring forward the availability of effective pneumococcal vaccines for developing countries by guaranteeing the initial purchase price, for a limited quantity of the new vaccines, that represents value‐for‐money and incentivizes manufacturers to invest in scaling‐up production capacity to meet developing country vaccine demand;

3 “Advance Market Commitment for Pneumococcal Vaccines: Implementation Working Group Report”, July 10, 2008, p. 5 – 6

3

3. To accelerate vaccine uptake by ensuring predictable vaccine pricing for countries and manufacturers, including binding commitments by participating companies to supply the vaccines at low, long‐term and sustainable prices after the AMC finances are depleted; and

4. To pilot test the effectiveness of the AMC mechanism as an incentive for needed vaccines and to learn lessons for possible future AMCs.

Further, it is recognized in the IWG report that the AMC can only achieve the above objectives if:

• Vaccine suppliers participate in the AMC;

• The resource requirements, including the financial demands on GAVI and on recipient country governments, fall within the willingness and ability of those entities to pay;

• The terms and features are reasonably robust to a range of unforeseen changes, such as unanticipated variations in demand and cost increases associated with regulatory changes; and

• The “mechanics” or procedures can be implemented with reasonable transaction costs.

The implementation, M&E activities will allow donors to test the effectiveness of the AMC mechanism and will provide information on AMC achievements to contribute to demonstrating the effectiveness of the mechanism for potential use with other interventions.

AMC documents4 have identified the overarching goal of the pneumococcal vaccine AMC and its associated demand stimulation activities to reduce morbidity and mortality from pneumococcal diseases and, specifically, to prevent an estimated 7 million childhood deaths by 2030.5 While the AMC should contribute substantially to achieving this public health impact,

4 For copies of these documents please see the GAVI website, www.vaccineamc.org

5 Estimates developed by Applied Strategies.

4

complementary activities undertaken by partners will be critical to achieving both the goal and objectives:

• Stimulating the demand for pneumococcal vaccines (e.g. estimates of disease burden, local effectiveness of vaccines, informing local decision makers);

• Subsidizing, to varying degrees, country co‐financing for the vaccines; and

• Strengthening health systems to ensure the effective delivery and use of the vaccines (e.g. effective cold chain, training for medical staff).

In the long‐term, the AMC is expected to contribute to a sustainable supply market for the pneumococcal vaccines in GAVI‐eligible countries (after the depletion of the AMC and after the “tail period”) and sustained health outcomes for GAVI‐eligible countries.

5

2. M&E STRATEGY

This chapter presents the scope of the M&E strategy, including the key issues for M&E, the indicators for measuring these issues, data sources and the periodicity of data collection and reporting.

2.1 SCOPE OF THE AMC M&E

The activities of the AMC and the complementary activities that need to be carried out to ensure the success of the initiative are described in detail in Appendix D. The Appendix also highlights the stakeholders that are involved in the establishment and implementation of the AMC (the donors, the pharmaceutical industry, GAVI, World Bank, ad hoc committees, WHO, the procurement agent(s) and GAVI‐eligible countries), as well as those that are expected to play a role in supporting the introduction and use of new pneumococcal vaccines in GAVI‐eligible countries (GAVI, Pneumococcal Accelerated Development and Introduction Plan (Pneumo‐ADIP), WHO and the United Nations Children’s Fund (UNICEF)).

Before presenting the M&E strategy it is important to define the scope of the M&E of this initiative. The scope of the M&E is defined by the AMC objectives and long‐term goal.

With respect to the AMC objectives (to accelerate the development of pneumococcal vaccines that meet developing country needs as specified in the TPP, bring forward their availability, accelerate vaccine uptake, and demonstrate AMC mechanism effectiveness) the M&E strategy should cover the assessment of the extent to which:

• The AMC rationale and design were appropriate for these objectives (rationale and design issues);

• The activities expected to lead to this objective are being implemented (process issues);

• The expected impact of the complementary activities required to achieve this objective has been realized (outcome issues); and

• The objective has been achieved (outcome issues).

6

With respect to the long‐term goal (to reduce morbidity and mortality from pneumococcal diseases) the M&E strategy should build on existing modelling information from other sources (e.g. GAVI) and assess the achievement of the long‐term goal in order to explain the extent to which the AMC has contributed to this long‐term goal.

The scope of the AMC M&E strategy does not include issues that relate to evaluating the effectiveness of the complementary activities of other stakeholders (e.g. whether GAVI, WHO etc. carried out their roles effectively). However, the implementation of these activities will need to be monitored throughout the implementation of the AMC. To the extent that information is available in the evaluations on the implementation of the complementary activities, it can also be seen as explanatory – explaining what contributed to, or detracted from, the achievement of the AMC objectives or long‐term goal.

2.2 M&E ISSUES

As part of the process of defining “what is to be evaluated”, it is necessary to develop a list of evaluation issues and questions. To identify these issues, the team:

• Considered the initial list of issues provided to the study team at the outset of the study;

• Augmented the list during the many discussions and interviews with primary stakeholders (including the donors, the World Bank, GAVI, the AMC Secretariat, the World Health Organization (WHO) and the EEG), as well as with other experts (e.g. Applied Strategies) interviewed for the study (see list of interviewees in Appendix A);

• Consulted with staff of Applied Strategies who completed much of the modelling underlying the conceptual design of the AMC;

• Prepared a causal map of the programme, and obtained agreement on this from the donor M&E Sub‐Group. This was used as a basis for generating the key questions that must be answered;

• Further refined the list of issues in two workshops held during September and October 2007; and

7

• Consulted with some of the members of the EEG and IWG in July 2008 in order to update the report to reflect recent changes in the AMC conceptual design.

The Organization for Economic Cooperation and Development (OECD), Development Assistance Committee (DAC) defines standard evaluation practice issues: relevance, effectiveness, efficiency, impact and sustainability.6 (See Appendix B for definitions of these issues).

For the purposes of this framework, given the importance of the evaluation of the process issues related to the design and implementation of the AMC, the issues have been grouped into slightly different categories:7

• Rationale issues: These issues deal with the need for the AMC initiative, and whether there were alternatives and perhaps more cost‐effective ways of achieving the same objectives;

• Design issues: These issues deal with the fundamental concept of the AMC and the terms, conditions and mechanics of the AMC concept. The goal here is to offer guidance on how potential future AMCs could be structured, and to follow up on issues considered during the EEG/IWG consultation process;

• Process issues: These issues deal with the overall efficiency of the AMC structures, whether the governance and oversight processes were adequate and whether timelines and budgets were respected; and

• Outcome issues: These constitute the core issues of the evaluation over the longer term and deal with whether the targeted results were achieved. The core issue for the initiative is whether the AMC stimulated additional/new pneumoccocal vaccine development, new vaccine production and the introduction of pneumococcal vaccines tailored to the needs of GAVI‐eligible countries.

6 DAC Criteria for Evaluating Development Assistance, 21 November 2000

7 A table (Exhibit 1) following the presentation of these AMC evaluation questions provides a cross‐referencing of the issues in this evaluation with the standard DAC issues.

8

Each issue area includes one overarching issue and a number of potential evaluation questions and sub‐questions. It should be noted that the specific issues should be reviewed and confirmed prior to each evaluation.

Rationale: Was there a clear rationale for the development of this AMC initiative?

1. Was there a valid need for the AMC mechanism in order to accelerate the provision of pneumococcal vaccine in GAVI‐eligible countries outside of the traditional GAVI support?

2. Were there more cost‐effective alternatives to achieving the same outcomes from a public health perspective?

Design: Was the design of the AMC initiative appropriate?

3. Were the assumptions behind the design of the AMC reasonable? Were the inputs to, and assumptions behind, the studies and modelling accurate?

4. Was the AMC design appropriate, easily understood, and did it provide the basis for an effective and transparent programme?

5. How did the various aspects of the design (price‐fixing, firm order timing, upfront binding supply, and purchase commitments) contribute to the AMC objectives?

a. Was the tail price ceiling too high/too low?

b. Was the treatment of India appropriate?8

8 India has a large birth cohort (25 million children per year). Given the size of the country and the potential pressure on GAVI resources, GAVI has set a cap of $350 million from 2008 to 2011 for all types of GAVI support. In 2011, the cap for the following 4‐5 years will be reviewed. Should the cap be maintained at a similar level and time period (4‐5) years, depending on the speed of introduction of pneumococcal vaccine, the cap would be reached before India is able to fully introduce pneumococcal vaccine with GAVI support. This could be reached as soon as 2017 or 2018. In consideration of this, during the IWG’s work, it was proposed that a discount be applied to the total demand value. While demand forecast stands at 210 million doses for 2025, it was proposed that the cap be set at 200 million doses to account for the possibility of a limited support to India. It was decided that this 200 million is to be used as the basis for calculating supply allocations.

9

6. Was the TPP an appropriate standard for product development, or should the TPP terms have been more or less stringent? Was the design choice of single versus multi dose vials appropriate?

7. Were the requirements and timeframes for companies to bid appropriate? Was the time frame for bidding appropriate? Did the timing requirements preclude entry of firms?

8. Was the process for establishing the AMC design using input of the EEG and IWG efficient? Were there lessons learned?

9. To what extent is the AMC concept consistent with donor practices (DAC principles) and compatible with industry practices?

Process: Was the AMC implemented in a cost‐effective manner to maximize the achievement of the expected outcomes?

10. Were there adequate consultations during the development phase of the AMC initiative?

11. Was the AMC governance structure (including management processes, agreements, terms, conditions and responsibilities) appropriate? Was it clearly documented, understood and supported? Was it effective?

12. Was financial management efficient, effective, transparent and timely?

13. What were the estimated costs of setting up and implementing the mechanism? Were they reasonable? Were the resources for designing the mechanism sufficient?

14. Was the AMC management and oversight adequately resourced for both the design and implementation phases?

a. Were the management functions performed effectively and in a timely manner?

b. Was the AMC DC the most effective and efficient way to oversee the AMC design phase?

15. Were the mechanisms to stimulate demand and support the introduction of pneumococcal vaccines in GAVI‐eligible countries adequately monitored? If so, was the information used in decision‐making?

10

Outcomes: Did the AMC achieve its expected outcomes? What, if any, were the unintended outcomes of the AMC?

Impact on the production and availability of pneumococcal vaccines

16. Did the AMC mechanism lead to an acceleration of investment in incremental production facilities and availability of supply of new pneumococcal vaccines for GAVI‐eligible countries by 2012?

17. Did the AMC initiative result in affordable and sustainable prices and supply of the vaccine after the tail period?

Health‐related impacts in GAVI‐eligible countries

18. Did the AMC contribute to increased confidence in investing in immunization programmes in GAVI‐eligible countries?

19. Did the AMC contribute to the accelerated adoption of pneumococcal vaccines in GAVI‐eligible countries? Were the forecasted immunization targets achieved?

20. What impact did the AMC have on GAVI‐eligible countries’ pattern of health expenditures (national government and donors) and health delivery systems during and after the AMC?

21. Was the appropriate pneumococcal vaccine procured, from a social efficacy point of view? (Cost‐benefit)

Unintended impacts

22. Did the AMC foster competition? What was the impact of any competition that occurred?

23. Did any vaccine producers reduce or eliminate their investment, at any point, in Research and Development (R&D) and/or production of pneumococcal vaccines for non‐GAVI‐eligible countries as a result of the AMC?

24. Did the AMC have a detrimental impact on any vaccine producers?

25. Did the AMC result in a change in the structure of the industry and/or in the behaviour of firms in non‐GAVI‐eligible countries?

11

26. Did participation in the AMC have an impact on the reputation of the pharmaceutical industry in non‐GAVI‐eligible countries?

27. What impact did the AMC have in non‐GAVI‐eligible countries on vaccine availability and sales and public health?

28. Were there any other unintended impacts of the AMC?

Lessons learned

29. Have the key lessons learned from this pilot project been identified and articulated so that they could be applied to other AMCs? What are they?

a. Are there lessons to be learned regarding what types of products may be most appropriate for AMCs?

b. Are there lessons to be learned regarding how AMCs should be designed differently for technologically close products, relative to technologically distant products?

c. Are there lessons to be learned concerning whether it makes sense to incorporate purchase guarantees and if so, under what conditions?

d. What lessons can be learned regarding the tail price? Was the tail price ceiling too high or too low?

These questions cover the standard DAC evaluation issues, cross‐referenced in Exhibit 1.

Exhibit 1: Cross‐reference of AMC Evaluation Questions with OECD DAC Evaluation Issues

OECD DAC Evaluation Issues AMC Evaluation Questions Relevance 1, 2 Effectiveness 10,11,12,13,14,15 Efficiency 3, 4, 5, 6, 7, 8, 9 Impact 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,28,29 Sustainability 20

12

2.3 DATA SOURCES

This section provides a general overview of the data sources being proposed for the M&E strategy.

2.3.1 DATA SOURCES

A variety of data sources are proposed for AMC M&E. They include:

• Literature and document review – This would include a review of the literature reflecting the context for, and concept of, the AMC, as well as documents on the implementation of the AMC;

• Interviews – At various points in the life of the AMC, it will be important to interview key stakeholders (including past and current representatives of the EEG, IWG, donor agencies, industry, GAVI‐eligible country stakeholders) on the AMC processes and outcomes;

• Surveys/interviews – The framework proposes a series of attitude and intention surveys/interviews of both vaccine producers and stakeholders in GAVI‐eligible and non‐GAVI‐eligible countries to include health authorities, bilateral donors, non‐governmental organizations (NGOs) and international agencies. These would be carried out using structured telephone interviews;

• Industry data – In order to assess the impact of the AMC on industry behaviour, data will need to be collected from industry directly, including firms serving GAVI‐eligible countries and potential new entrants, or from existing public sources of data on the pharmaceutical industry (see Section 2.4.1); and

• Immunization and health data – This includes the use of country data that is reported regularly by UNICEF and WHO, including immunization coverage, dates of introduction of new vaccines, mortality/morbidity data, and special studies relevant to pneumococcal disease generated by the WHO Global Burden of Disease Project. WHO prepares annual GAVI progress reports (available on the WHO‐Immunization Coverage Estimates and Trajectories (ICE‐T) database) that include coverage data, disease burden estimates and projections of deaths averted.

13

2.4 COMPONENTS OF M&E STRATEGY

The M&E strategy described in this report has four components9:

• A baseline study to determine the point of comparison for future M&E. It will establish the environment for the AMC and the counterfactuals10 at the starting point of the AMC. It will document both the situation prior to the AMC and the counterfactuals, as a starting point for measuring the impacts of the initiative. It should address to the extent possible both the situation with respect to the pneumococcal vaccine industry (vis‐à‐vis investment in R&D and production capacity) and the status of immunization and health in GAVI‐eligible countries.

• Annual monitoring of both the AMC and the complementary activities required to support the public health goal of the AMC. Monitoring is the ongoing reviewing and reporting of programme accomplishments, particularly progress toward pre‐established goals.11 It is typically conducted by programme or agency management and results in periodic reports assembled annually. These reports are provided to consultants conducting periodic evaluations.

• Evaluations. Evaluations are independent, systematic studies conducted periodically or on an ad hoc basis to assess how well a programme is working, addressing a broader range of information on programme performance and its context than is feasible to monitor on an ongoing basis. There are typically two types of evaluations:

o Process Evaluations – A process evaluation about two years after the launch of the AMC to assess whether the AMC mechanism is

9 The definitions for monitoring and evaluation draw heavily on material from the United States Government Accountability Office (GAO) and the OECD DAC (Glossary of Key Terms in Evaluation and Results Based Management, DAC Working Party on Aid Evaluation, OECD, 2002.)

10 A counterfactual is a simulation of what would have happened without policy intervention, or if the policy intervention had taken a different (but specified) form. Measuring the difference between the situation under the AMC and the counterfactuals contributes to the assessment of impact of the AMC.

11 United States Government Accountability Office, Performance Measurement and Evaluation, Definitions and Relationships

14

working as expected and to obtain information on the AMC design issues. Process evaluation are also called implementation or formative evaluations. Process evaluations “verify what the programme is and whether or not it is delivered as intended to the targeted recipients.”12 The process evaluation will also investigate design issues with a view towards providing input on whether additional AMCs would be useful and, if so, to inform how future AMCs should be designed.

o Outcome evaluations every four years after the signing of the first AMC Supply Agreement. Outcome evaluations are also called impact or summative evaluations. They assess the extent to which a programme achieves its objectives and, specifically, the impact of the programme on the target group. They focus, for the most part, on outcomes (including unintended effects) to judge programme effectiveness but will also address design issues to the extent that these issues contribute to explaining the extent that outcomes have been achieved. The outcome evaluations will focus on assessing the extent of achievement of the AMC objectives. They will also address design issues, as these issues help to explain the extent to which the outcomes have been achieved, and an analysis of the counterfactuals. The first outcome evaluation is expected in 2013, provided a supply agreement is signed in late 2009.

Each is described in more detail in the following sections.

2.4.1 BASELINE STUDY

Baseline data establishes the environment for the programme at the beginning of the intervention. A baseline study will have to be conducted as soon as possible to collect both industry and country‐level data to establish the baseline for the AMC. It will also include the development of the counterfactual models.

Industry level data

The industry focus in the baseline study will include:

12 Evaluation, A Systematic Approach, Rossi, Lipsey and Freeman. Sage Publications 2004.

15

• Collecting information available from the industry consultations that were used for the AMC‐Financial Implications and Risk Model (FIRM). This should include information from the earliest date possible, but is expected to be about 2005; and

• Conducting an intentions survey of pneumococcal vaccine producers in terms of investment in development and production of vaccines for GAVI‐eligible countries, and on trends in industry’s confidence in the AMC strategic forecasts and the UNICEF short‐term forecasts, in addition to attitudes and internal factors affecting company decisions.13

Based on the consultations with industry, the evaluability study team believes that industry is not likely to provide very specific quantitative data on R&D or investment and production. An intentions survey is suggested, the purpose of which would be to establish qualitative trends in expenditures based on opinions of industry stakeholders. The focus could be establishing trends in terms of “change in perceptions” where, hopefully, 2006 can be considered the base year. In addition to this there may be some information available (for firms with securities traded on US Exchanges) from the company’s United States of America (USA) Security Exchange Commission (SEC) reports, the USA Food and Drug Administration (FDA), public information releases and annual reports.

Specifically, the study will collect information on:

• Perceptions of changes in R&D effort since 2005 for pneumococcal vaccine development, by type of market. Specifically the changes in R&D investments for vaccine development for GAVI‐eligible countries will be requested;

• Perceptions of changes in plant investment since 2005 for vaccine production and pneumococcal vaccine;

• Perceptions of changes in licensing or production agreements with emerging country suppliers for pneumococcal vaccines (if any); and

13 The industry and country level surveys should be repeated periodically. It is suggested that these surveys be included in the periodic outcome evaluations of the AMC.

16

• Perceptions of changes in the annual number of doses manufactured targeted for GAVI‐eligible and if possible, non‐GAVI eligible countries14 since 2005 for pneumococcal vaccines.

The data suggested above have been discussed with industry.

Country level data

Country level data will come from two sources: a survey at the country level and existing health data.

The country level study will collect baseline data regarding:

• Actual and forecast demand for pneumococcal vaccines (in terms of numbers of doses);15

• Perceptions regarding the importance and priority of the pneumococcal burden of disease versus other health problems;

• Willingness and ability to purchase or subsidize pneumococcal vaccines (country willingness to contribute to their share of the cost per dose);

• Level of confidence in pneumococcal vaccine immunization programmes; and

• Surveillance data showing trends in the burden of pneumococcal disease, etc.

This data will be collected through a survey of representatives of health authorities and organizations supporting the health sector in GAVI‐eligible countries, including staff from health ministries, immunization programmes, bilateral donors, NGOs and international agencies with country/regional offices, such as UNICEF and WHO.

This information may be obtained or supplemented by other existing reports from GAVI and/or UNICEF/WHO. Existing reports include information on budget forecasts for vaccines, timeframe for introduction, perceived obstacles to

14 A proxy could be the number of dedicated finishing lines and capital costs for multi‐dose vial filling.

15 This can be based on UNICEF short‐term forecasts.

17

procurement, distribution and use, projections for payment/subsidies of pneumococcal vaccines, results from cost‐effectiveness studies, etc.

Similarly the WHO Burden of Disease Project will provide baseline data on morbidity from pneumococcal disease among children less than five years of age as a minimum. It is available for 2000 globally, regionally, sub‐regionally and by country. A pneumococcal disease analysis conducted by WHO in collaboration with PneumoADIP from 2004 ‐ 2007 was completed in 2008. These sets of data would constitute good baseline data for determining public health impact during the periodic evaluations. WHO plans to do time series/projections of the numbers of pneumococcal cases and related deaths annually. These projections may be informed by small studies based on actual data (sentinel studies). WHO also analyzes and reports on the GAVI contribution to vaccine coverage and future deaths averted annually.

Counterfactual models

The development of the counterfactuals is included in the baseline study.

The counterfactuals will serve as the benchmark for testing the incrementality of the AMC and the attribution of results to the AMC concept. This means that significant effort will have to go into developing a counterfactual model that is credible. The study proposes using the rotavirus vaccine and/or Haemophilus influenzae type B (Hib) development and implementation as counterfactuals.

In essence, the counterfactuals should help demonstrate whether or not the AMC has contributed to advancing take up, availability, distribution and application of the advanced pneumococcal vaccines.

2.4.2 ANNUAL MONITORING

Data from various sources should be compiled and reported annually to the Independent Assessment Committee (IAC) and the donors. It should be the role of the AMC Secretariat to compile the data and prepare these annual reports and make the data available to external independent contractors conducting periodic evaluations.

Annual monitoring will cover key issues identified in Section 2.2 that can be grouped, as follows:

18

• Implementation of the AMC – implementation of the various legal agreements and roles and responsibilities, signing of Supply Agreements, financial inflows and disbursements, status of the Supply Agreements, etc;

• Changes at the industry level – monitoring the acceleration of R&D, and the investment in incremental production facilities for new pneumococcal vaccines and the extent to which the AMC has fostered competition; and

• Changes at country level – monitoring country demand for pneumococcal vaccines and expected immunization levels, improvements in public health in GAVI‐eligible countries (and the impact in non‐GAVI‐eligible countries) and the pattern of health expenditures in GAVI‐eligible countries.

AMC Implementation

The AMC Secretariat should be required to monitor and report on key events in the AMC implementation, with particular comparisons to timelines, plans, and projections. This will require the preparation of an annual implementation plan so that variances from the plan can be demonstrated. The plan will include activities and milestones for all stakeholders, including all parties involved in the AMC agreements. Progress achieved will be derived from reports provided by partners, such as GAVI, the World Bank, UNICEF, WHO and GAVI‐eligible countries. The Secretariat will provide annual monitoring reports to the IAC and donors, highlighting achievements and problems/obstacles.

The World Bank should be required to provide annual reports to the donors. The report will include status of income and expenditures of donor money and breakdowns of each. Income will include line items for donor contributions by donor and earnings on funds invested. Expenditures will include line items for funds expended for purchase of vaccine and for administrative costs including overhead. Actual AMC expenditures will be compared to planned expenditures. Actual financial contributions compared to commitments by donors will be reported, as well as reasons for any variances.

Industry‐level monitoring

The industry‐level monitoring will include primarily publicly available information, based on news releases, company reports, and the SEC filings. Indicators tracked could include (subject to data availability):

19

• New trials for pneumococcal vaccines;

• New investments in production capacity for pneumococcal vaccines targeted at GAVI eligible countries; and

• Trends in pneumococcal vaccine doses shipped to GAVI‐eligible and, if possible, non‐GAVI‐eligible countries.

Country‐level monitoring

In order to ensure that the complementary activities required for the success of the AMC are being carried out, there should also be monitoring at the level of both GAVI‐eligible and non‐GAVI‐eligible countries, which would include:

• Updates from the WHO Burden of Disease Project that provides mortality data, burden of disease, and projections through its annual report to GAVI;

• Updates from GAVI/WHO/UNICEF on the implementation of activities to support the introduction and use of pneumococcal vaccines, including Progress Reports of countries to GAVI and information on the subsidization of co‐payments, activities to stimulate and forecast demand and investments in in‐country health system strengthening; and

• Information from UNICEF on procurements (e.g. volume of pneumococcal vaccine purchased annually per country).

2.4.3 PROCESS EVALUATION

The first evaluation should focus on the AMC implementation processes, specifically on assessing the extent to which the AMC is being implemented as planned (in comparison to the various agreements and with respect to forecasts) and whether the complementary activities to support the introduction and demand for pneumococcal vaccines are occurring. The specific issues to be covered are presented in Section 2.5. However, they should be confirmed and/or adjusted as required by the evaluation Steering Committee, immediately prior to undertaking the evaluation.

The following activities will be required for the process evaluation:

20

AMC implementation

• Review whether the planned structures have been implemented as per the Agreements;

• Assess the perceptions of stakeholders of how well the AMC implementation has worked;

• Review the quality and reliability of annual monitoring, as well as the use of the monitoring information.

Industry level

• Compare the latest available AMC‐FIRM model and the CRA International model projections to actual emerging trends. The cause of variances should be assessed and consideration should be given to how these variances could be addressed in possible future applications of the AMC approach;

• Analyze perceptions of industry trends in R&D, investment, production for pneumococcal vaccines;

• Assess industry perceptions of future demand for pneumococcal vaccines from GAVI‐eligible countries, the likely price and profitability of vaccine products from their perspective, industry’s assessment of GAVI country absorptive capacity and donors’ willingness to finance the essential complementary activities (e.g. infrastructure, training, subsidizing of country co‐payments); and

• Repeat the intentions survey carried out for the baseline study, including identifying industry intentions for R&D, clinical trials, licensing, production, and investment in production capacity for pneumococcal and counterfactual vaccines, and sales in GAVI, middle‐income and high‐income country markets for the pneumococcal and counterfactual vaccines.

Country level

• Repeat the survey of country‐level health authorities and supporting organizations in GAVI‐eligible countries to assess changes in attitudes and plans for vaccine procurement, trends in the government health budgets, as well as donor and NGO plans;

21

• Analyze trends in procurement of pneumococcal vaccines immunization rates and disease burden for pneumococcal in GAVI‐eligible and non‐GAVI‐eligible countries; and

• Case studies, based on site visits, of a sample of GAVI‐eligible countries (for example, the early adopter countries and some non‐adopting countries) to assess the implementation of the complementary activities to ensure demand generation and the development of the capacity for the delivery of pneumococcal vaccines.

A further step in the AMC process evaluation will be to look at the pneumococcal vaccine AMC through the lens of whether future AMCs would be useful and if so, to draw lessons on how these future AMCs should be designed.

This exercise should build on the work of the EEG/IWG report. It would be fruitful for the process evaluation to be undertaken by a similar team, consisting of: (1) economists with expertise in industrial organization and mechanism design; and (2) individuals with knowledge of the pharmaceutical industry and of vaccination programmes in the developing world. However, it would be very useful for this team to be in close contact with the other evaluation teams to build on their data gathering and evaluation activities.

2.4.4 OUTCOME EVALUATIONS

It is expected that the AMC will lead to the availability of pneumococcal vaccines targeted to GAVI‐eligible countries as early as 2009 but potentially not until four or more years after the expected 2008 launch of the AMC. The first outcome evaluation should be conducted no later than four years after signing the first Supply Agreement (expected to be in 2009 or 2010). This evaluation, expected for 2013 earliest, will focus more on the achievements of AMC outcomes than on processes – on what has changed as a result of the AMC initiative. The evaluation will go a step further to assess causality between the AMC intervention and its results through the comparisons with the counterfactuals. It will also examine the unintended impacts (both negative and positive) of the AMC. The specific issues to be covered are presented in Section 2.5. However, they should be confirmed and/or adjusted as required by the evaluation Steering Committee, immediately prior to undertaking the evaluation. Depending on the status of the implementation of the initiative and the achievement of the

22

outcomes, some design issues may also need to be addressed in the outcome evaluations.

This outcome evaluation should include assessments of:

• Rationale and alternative approaches;

• Whether the AMC has performed as anticipated, including, if required, addressing issues related to AMC design processes and oversight;

• Linkages between inputs and outcomes and the contribution of the AMC to achieving success or not in attaining outcomes/results; and

• Changes, both negative and positive, generated by the AMC including those described in Appendix D, Section 5.2 of this report.

The evaluation builds on the annual monitoring data and the results of the first evaluation, but will also include specific data collection for this evaluation.

The evaluation would include, inter alia, the following:

Industry level

• Repeat of the intentions survey carried out for the baseline study, including identifying industry intentions for R&D, clinical trials, licensing, production, investment in production capacity for pneumococcal and counterfactual vaccines, and sales in GAVI, middle‐income and high‐income country markets for the pneumococcal and counterfactual vaccines;

• Assessment of the impact that the AMC has had on the pharmaceutical firms, in terms of perceptions of changes in R&D expenditures by different firms on pneumococcal vaccines appropriate for GAVI countries, increased plant investment and production capacity, the number and type of firms supplying vaccines and changes in the structure of the industry, industry behaviour and the nature of the competition. The results will be compared to the baseline. The trends will be compared to the counterfactuals to assess the incremental impact of the AMC over policy alternatives;

• Comparison of the AMC‐FIRM model and CRA International model projections to actual trends, in order to assess how the original projections compare to what actual trends have emerged. The source of variations

23

between the model projections and what has actually happened may be a result of changes in the environment or faulty assumptions in the model;

• If possible, comparisons of the volume shipped to GAVI‐eligible countries of pneumococcal vaccines with shipments to non‐GAVI‐eligible countries during and after the AMC;

• Assessment of the unintended impacts of the AMC;

Country level

• Repeating the country‐level survey of local health authorities and supporting organizations in GAVI‐eligible countries to assess changes in attitudes and plans for vaccine procurement, as well as trends in the government health budgets as well as donor and NGO plans;

• Assessment of trends in the demand side for the pneumococcal vaccine. This would also involve comparing actuals to the projections of the AMC‐FIRM and CRA International model projections;

• Assessing the health impacts of the pneumococcal vaccine to assess overall efficacy of the AMC in both GAVI‐eligible and non‐GAVI‐eligible countries, including:

o Trends in immunization rates and disease burden;

o Immunization rates (available from UNICEF and WHO);

o Disease specific mortality and morbidity (available from the WHO);

o Disease burden statistics (available annually from the WHO ICE‐T database); and

• Conducting case studies, based on site visits, of a sample of GAVI‐eligible countries (for example, the early adopter countries) to assess the implementation of the complementary activities to support the introduction of pneumococcal vaccines.

After the first evaluation focusing on AMC outcomes, subsequent evaluations should be conducted about every four years to ensure that the AMC is achieving its longer‐term outcomes. The approach and methodology would be similar to the outcome evaluation planned for 2013.

24

2.5 MONITORING AND EVALUATION MATRIX

The M&E matrix in Exhibit 2 reflects, for each evaluation question, the specific indicators and data sources and the type of M&E studies that should be used to address the question.

25

Exhibit 2: Monitoring and Evaluation Issues

Performance Issues Indicators Data Sources M&E Component Rationale: Was there a clear rationale for the development of this AMC initiative? 1. Was there a valid need for the AMC

mechanism in order to accelerate the provision of pneumococcal vaccine in GAVI‐eligible countries outside of the traditional GAVI support?

‐ Historical trends in vaccine availability in high‐income countries compared to GAVI‐eligible countries ‐ Comparison of actual (and projected) outcomes with counterfactuals in terms of lag time to availability of vaccines, costs, trends in immunization rates, etc. ‐ Comparison of time lags for other vaccines ‐ Opinions of stakeholders (lags)

‐ Literature review of studies on historical delays in vaccine availability in lower income countries

‐ First outcome evaluation (2013)

2. Were there more cost‐effective alternatives to achieving the same outcomes from a public health perspective?

‐ Comparison of actual (and projected) outcomes with counterfactuals in terms of lag time to availability of vaccines, costs, trends in immunization rates, etc. ‐ Opinions of stakeholders

‐ Monitoring reports & evaluation studies for pneumococcal and the counterfactuals

‐ Outcome evaluations

Design: Was the design of the AMC initiative appropriate? 3. Were the assumptions behind the

design of the AMC reasonable? Were the inputs to, and assumptions behind, the studies and modeling accurate?

‐Evidence that stakeholders agreed with the assumptions behind the AMC design ‐ Events occur as projected ‐ Evidence of delays or challenges in process ‐ Variance between model predictions and outcomes

‐ Industry survey ‐ Interviews with IAC members ‐ Interviews with stakeholders

‐ Process evaluation ‐ Outcome evaluations

26

Performance Issues Indicators Data Sources M&E Component 4. Was the AMC design appropriate,

easily understood, and did it provide the basis for an effective and transparent programme?

‐ Evidence that stakeholders understand AMC structures and processes ‐Opinions of stakeholders ‐Results of economic modelling

‐ Monitoring reports ‐ Interviews with stakeholders

‐ Process evaluation

5. How did the various aspects of the design (price‐fixing, firm order timing, upfront binding supply, and purchase commitments) contribute to the AMC objectives? a. Was the tail price ceiling too high/too low? b. Was the treatment of India appropriate?

‐ Comparison of actual (and projected) outcomes with counterfactuals in terms of lag time to availability of vaccines, costs, trends in immunization rates, etc. ‐ Extent to which GAVI‐eligible countries required external assistance to meet co‐payment and tail prices ‐ Opinions of industry ‐ Opinions of stakeholders

‐ Interviews with Industry ‐ Country stakeholder interviews in GAVI‐eligible countries ‐ Monitoring Reports and Counterfactuals

‐ Baseline study ‐ Process evaluation

6. Was the TPP an appropriate standard for product development, or should the TPP terms have been more or less stringent? E.G. Was the design choice of single versus multi dose vials appropriate?

‐ Number of requests for changes by the IAC ‐ Evidence that the TPP meets GAVI‐eligible country requirements ‐ Opinions of industry representatives ‐ Opinions of IAC members ‐ Opinions of stakeholders



7. Was the process for establishing the AMC design using input of the EEG and IWG efficient? Were there lessons learned?

‐Evidence that stakeholders agreed with the assumptions behind the AMC design ‐ Events occur as projected ‐ Evidence of delays or challenges in process



27

Performance Issues Indicators Data Sources M&E Component ‐ Variance between model predictions and outcomes

8. To what extent is the AMC concept consistent with donor practices (DAC principles) and compatible with industry practices?

‐ Perception of design as being consistent with donor and industry practices (e.g. provisions of general budget or sector support)

‐ Interviews with donors ‐ Interviews with Industry

‐ Process evaluation ‐ Design evaluation

Process: Was the AMC implemented in a cost‐effective manner to maximize the achievement of the expected outcomes? 9. Were there adequate consultations

during the development phase of the AMC initiative?

‐ Evidence of engagement of various stakeholders (industry, regulators, GAVI‐eligible countries, donors, etc.) ‐ Opinions of stakeholders

‐ Interviews with stakeholders ‐ Document review


10. Was the AMC governance structure (including management processes, agreements and terms and conditions and responsibilities) appropriate? Was it clearly documented, understood and supported? Was it effective?

‐ Evidence that the outputs from structures and processes are on schedule ‐ Evidence that all stakeholders understand governance ‐ Evidence that the governance structure is functioning as planned ‐ Evidence that legal agreements were enforceable at reasonable cost (time and money)

‐ Monitoring Reports ‐ Interviews with GAVI and other stakeholders ‐ Governance documents and meeting minutes ‐ Legal agreements ‐ Interviews with the World Bank

‐ Annual monitoring (first three years only) ‐ Process evaluation ‐ Outcome evaluations

11. Was financial management efficient, effective, transparent and timely?

‐ Evidence that donor financing provisions and commitments are respected ‐ Evidence of effective cash management

‐ Annual financial reports ‐ Interviews with stakeholders ‐ Internal audits

‐ Annual monitoring ‐ Process evaluation ‐ Outcome evaluations

28

Performance Issues Indicators Data Sources M&E Component ‐ Stakeholder perceptions of efficiency and effectiveness of the disbursement procedures ‐ Evidence of appropriate currency risk management ‐ Adequate financial accountability

12. What were the estimated costs of setting up and implementing the mechanism? Were they reasonable? Were the resources for designing the mechanism sufficient?

‐ Financial costs for the AMC mechanism and related administrative costs ‐ Perceptions of adequacy of resources and analysis to set up the mechanism

‐ World Bank, GAVI and AMC financial reports ‐ Annual monitoring ‐ Process evaluation ‐ Outcome evaluations

13. Was the AMC management and oversight adequately resourced for both the design and implementation phases? a. Were the management functions performed effectively and in a timely manner? b. Was the AMC DC the most effective and efficient way to oversee the AMC design phase?

‐ Resource and financial commitments met for AMC Secretariat ‐ Evidence of oversight reports produced on schedule ‐ Delays/challenges in decision‐making

‐ Interviews with donors ‐ Interviews with AMC Secretariat


14. Were the mechanisms to stimulate demand and support the introduction of pneumococcal vaccines in GAVI‐eligible countries adequately monitored? If so, was the information used in decision‐making?

‐ Regular reporting on demand and introduction in GAVI‐eligible countries ‐ Evidence of actions taken in response to information provided through monitoring

‐ Reports from AMC Secretariat ‐ Minutes of meetings of DC, IAC


Outcomes: Did the AMC achieve its expected outcomes? What, if any, were the unintended outcomes of the AMC? Impact on the production and availability of pneumococcal vaccines 15. Did the AMC mechanism lead to an

acceleration of investment in ‐ Periodic comparison of original projections to actuals

‐ Industry survey and consultations ‐ Evidence from industry, public information

‐ Baseline study ‐ Annual monitoring

29

Performance Issues Indicators Data Sources M&E Component incremental production facilities and availability of supply of new pneumococcal vaccines for GAVI‐eligible countries by 2012?

for R&D spending, investment in production facilities, and shipments for pneumococcal and for the counterfactuals ‐ Opinions of industry ‐ Opinions of stakeholders

‐ Comparison of the Applied Strategy and CRA model forecasts with actuals


16. Did the AMC initiative result in affordable and sustainable prices and supply of the vaccine after the tail period?

‐ Comparison with immunization levels of other vaccines ‐ Opinions of stakeholders in GAVI‐eligible countries, especially NGOs and country health authorities

‐ Survey of GAVI‐eligible country health authorities and other stakeholders


Health‐related impacts in GAVI‐eligible countries 17. Did the AMC contribute to increased

confidence in investing in immunization programmes in GAVI‐eligible countries?

‐ Number of doses of vaccine purchased, compared to forecasts ‐ Changes in doses of vaccine purchases over time ‐ Immunization coverage in GAVI‐eligible countries compared to targets

‐ Procurement agency data on purchase and delivery of vaccines ‐ Comparisons of model forecasts to actuals ‐ Country reports on vaccine procurement and immunization coverage ‐ Country stakeholder interviews in GAVI‐eligible countries

‐ Annual monitoring ‐ Outcome evaluations

19. Did the AMC contribute to the accelerated adoption of pneumococcal vaccines in GAVI‐eligible countries? Were the forecasted immunization targets achieved?

‐ Trends in health expenditures and vaccination expenditures for pneumococcal vaccines compared to other health expenditures ‐ Comparison of expenditures for pneumococcal vaccines with expenditures for other vaccines

‐ UNICEF data ‐ GAVI and World Bank Health expenditure data ‐ Survey of country health authorities


20. What impact did the AMC have on GAVI‐eligible countries’ pattern of

‐ Costs vs. benefits of: ‐ Lives saved /deaths averted

- WHO ICE‐T database data on disease burden, mortality and morbidity data

‐ Baseline study ‐ Annual monitoring

30

Performance Issues Indicators Data Sources M&E Component health expenditures (national government and donors) and health delivery systems during and after the AMC?

(all cause) in 3‐59 month old children vaccinated with pneumococcal conjugate vaccine in GAVI‐eligible countries ‐ Lives saved/deaths averted among unvaccinated children and adults ‐ Estimated disease burden caused by pneumococcus (cases, deaths, Disability Adjusted Life Years) globally and in GAVI‐eligible countries

‐ Country routine coverage and campaign data (joint WHO/UNICEF reporting forms) ‐ GAVI projection models on herd immunity ‐ WHO Global Burden of Disease Project ‐ Special (sentinel) studies supported by GAVI and others ‐ Country stakeholder interviews in GAVI‐eligible countries


21. Was the appropriate pneumococcal vaccine procured, from a social efficacy point of view? (Cost‐benefit)

‐ Trends in health expenditures and vaccination expenditures for pneumococcal vaccines compared to other health expenditures ‐ Comparison of expenditures for pneumococcal vaccines with expenditures for other vaccines

‐ Procurement agent data on purchase and delivery of vaccines ‐ Interviews with country stakeholders in GAVI‐eligible countries

‐ Baseline study ‐ Annual monitoring ‐ Outcome evaluations

Unintended impacts 22. Did the AMC foster competition?

What was the impact of any competition that occurred?

‐ Unintended impacts identified by stakeholders ‐ Observed competition as per new Supply Agreements

‐ Interviews with stakeholders ‐ Numbers of Supply Agreements over the AMC implementation period and during the post AMC period ‐ Evidence of new suppliers for pneumo vaccines becoming active ‐ Trends in second generation R&D

‐ Baseline study ‐ Process evaluation ‐ Outcome evaluations

31

Performance Issues Indicators Data Sources M&E Component 23. Did any vaccine producers reduce or

eliminate their investment, at any point, in R&D and/or production of pneumococcal vaccines for non‐GAVI‐eligible countries as a result of the AMC?

‐ Number of Supply Agreements and AMC share for each one ‐ Evidence of other suppliers with competitive products (outside of AMC) ‐ Opinions of new producers and potential producers not participating ‐ Comparison with baseline and counterfactuals (on the number of pharmaceutical companies engaged in the pneumococcal vaccine markets)

‐ AMC reports ‐ Survey of vaccine producers ‐ Monitoring data on vaccine production, distribution and price ‐ AMC‐FIRM and CRA model projections


24. Did the AMC have a detrimental impact on any vaccine producers?

‐ Evidence of reduced R&D and production investments in the non‐GAVI country markets

‐ Industry survey ‐ Survey of health authorities in non‐GAVI‐eligible countries

‐ Baseline study ‐ Process evaluation ‐ Outcome evaluations

25. Did the AMC result in a change in the structure of the industry and/or in the behaviour of firms in non‐GAVI‐eligible countries?

‐ Perceptions of expected impacts on specific vaccine suppliers ‐ Perceptions of equity of allocation of AMC funds

‐ Industry survey ‐ Outcome evaluations

26. Did participation in the AMC have an impact on the reputation of the pharmaceutical industry in non‐GAVI‐eligible countries?

‐ Perceptions of industry representatives

‐ Industry survey ‐ Baseline study ‐ Outcome evaluations

27. What impact did the AMC have in non‐GAVI‐eligible countries on vaccine availability and sales and public health?

‐ Opinions of stakeholders in non‐GAVI‐eligible countries, especially NGOs and country health authorities

‐ Interviews with stakeholders ‐ Survey of industry


32

Performance Issues Indicators Data Sources M&E Component 28. Were there any other unintended

impacts of the AMC? ‐ Changes in availability and unit cost of vaccines in industrialized and middle‐income countries vs. demand ‐ Extent producers were able to respect conditions of Guarantee and Supply Agreements ‐ Extent to which countries required external assistance to meet co‐payment and tail prices ‐ Incidence rates (mortality and morbidity) in non‐GAVI countries

‐ Survey of Industry ‐ UNICEF data on vaccine availability/sales and WHO disease burden, mortality and morbidity data

‐ Baseline study ‐ Annual monitoring ‐ Outcome evaluations

Lessons learned 29. Have the key lessons learned from

this pilot project been identified and articulated so that they could be applied to other AMCs? What are they? a. Are there lessons to be learned regarding what types of products may be most appropriate for AMCs? b. Are there lessons to be learned regarding how AMCs should be designed differently for technologically close products, relative to technologically distant products? c. Are there lessons to be learned concerning whether it makes sense to incorporate purchase guarantees and

‐ Lessons learned identified and summarized for decision‐makers ‐ Opinions of stakeholders and participants

‐ Interviews with stakeholders ‐ Evaluation Reports (AMC Process and Outcome)


33

Performance Issues Indicators Data Sources M&E Component if so, under what conditions? d. What lessons can be learned regarding the tail price? Was the tail price ceiling too high/too low?

34

2.6 PROPOSED COUNTERFACTUALS

An important aspect of the M&E framework is comparing what happens as a result of the AMC to what would likely happen without it. This will be based on assessing the AMC against simulated policy alternatives. These policy alternatives are called counterfactuals.

A counterfactual is defined by the OECD DAC as “The situation or condition, which hypothetically may prevail for individuals, organizations, or groups were there no development intervention.”16 Any evaluation of a policy’s effects should be made relative to what would otherwise have happened. Usually it is not enough to describe the starting position or ‘baseline’ since this is likely to change over time.17

Counterfactuals are used often in economic analysis to assess what would happen under alternative potential scenarios. The counterfactual analysis is generally a simulation or “what if” scenario. The scenario can be determined by running a model under a different set of assumptions or by comparing the policy intervention to a “control” which has not had the same intervention. A primary issue for the M&E framework is how the AMC has affected the timing, availability, quantity, quality and price for pneumococcal vaccines – all of which need to be estimated relative to some benchmark of what would have happened to these outcomes in the absence of the AMC.

The baseline study team will be tasked with: (1) selecting comparator vaccine(s); and (2) making adjustments to the comparator vaccine(s) to allow for a reasonable counterfactual or range of counterfactuals for pneumococcus. For example, ideally a comparison vaccine would control for the influence of time‐varying policy factors affecting vaccines for low‐income countries (such as GAVI funding), control for technological characteristics which may change the price or production of vaccines (such as conjugate vaccine production technologies), control for “supply side” factors (such as the number of firms in the market at the time of first entry), control for “market size” in the sense of the disease

16 Glossary of Key Terms in Evaluation and Results Based Management, DAC Working Party on Aid Evaluation, OECD, 2002, p. 19.

17 Environment Agency, Government of the UK.

35

burden in the developed and middle‐income world as well as GAVI countries (which would be important if firms are reluctant to sell at low prices in GAVI countries because they are worried about being able to maintain prices in middle‐income countries, for example). Another issue, which should be addressed when choosing comparator vaccines, is technological challenges facing potential additional suppliers and emerging market suppliers in particular, in the development of new versions of the vaccine.

Almost surely, no vaccine will be an excellent match for each characteristic. The goal is not to identify the “perfect” comparator vaccine but rather to identify one or several reasonable comparator vaccines, which can be adjusted to construct reasonable counterfactuals.

The study team has preliminarily identified two potential counterfactual vaccines: rotavirus and haemophilis influenzae b (Hib).

2.6.1 ROTAVIRUS

To evaluate the usefulness of rotavirus as a counterfactual for pneumococcus in terms of market structure, the baseline study team will be tasked with exploring how the structure of the market for rotavirus vaccine sales compares to the structure of the market for pneumococcal vaccine sales. Like pneumococcal diseases, rotaviruses impose a non‐trivial disease burden in developed countries but their primary disease burden in terms of morbidity and mortality falls on children in low‐income countries. Rotaviruses are the most common cause of severe diarrhoeal disease in infants and young children worldwide; in 2004, rotavirus infections were estimated to cause approximately 527,000 (range of 475,000–580,000) deaths, predominantly in developing countries. Although the viral strains show considerable diversity, five serotypes are responsible for the majority of human rotavirus disease.

To evaluate the usefulness of rotavirus as a counterfactual for pneumococcus on the basis of technological characteristics, the baseline study team will be tasked with exploring how the technological characteristics of the rotavirus vaccine compare to the technological characteristics of the pneumococcal vaccine.

To evaluate the usefulness of rotavirus as a counterfactual for pneumoccocus on the basis of supply side factors, the baseline study team will be tasked with exploring how the firms involved in R&D on rotavirus vaccines at the time of its launch (and subsequent entrants) compare to the firms involved in R&D on

36

pneumococcal vaccines (and subsequent entrants). Two new live, oral, attenuated rotavirus vaccines were licensed in 2006: the monovalent human rotavirus vaccine (Rotarix™ from Glaxo Smithkine (GSK)) and the pentavalent bovine–human, reassortant vaccine (RotaTeq™ from Merck). Both vaccines have demonstrated very good safety and efficacy profiles in large clinical trials in western industrialized countries and in Latin America. Early reports of a possible emerging country supplier in China have not been confirmed in recent GAVI and RotavirusADIP documents.

WHO has given a strongly positive, but qualified, recommendation for the use of these rotavirus vaccines. The qualifications in WHO’s recommendation contrasts with the unqualified recommendations that it provided for pneumococcal vaccines, and may be influential in affecting the country demand for the vaccines.

To evaluate the usefulness of rotavirus as a counterfactual for pneumoccocus on the basis of policy factors, the baseline study team will be tasked with exploring how policies such as GAVI funding and ADIPs varied for rotavirus vaccines relative to pneumococcal vaccines. The rotavirus and pneumococcal ADIPs were conceived using the same assumptions (e.g. the need to: establish the value of the vaccine, communicate that value to local decision makers/donors and deliver value by ensuring that sufficient production capacity is available to produce the vaccines for the developing countries).

Both ADIPs were created at the same time by GAVI, and with one key exception both have proceeded through the GAVI management system at the same pace, including the decision to create the ADIPs and the November 29, 2006 decision by the GAVI Alliance Board to funds both ADIPs for a phase implementation programme which total US$ 200 million for both programmes. Both ADIPs seek to address the same lag inducing factors. Both ADIPs appear to use similar state‐of‐the‐art methods to address the “pull” factors associated with establishing and communicating the value of vaccines to local decision makers and donors, and developing local human capacity and infrastructure to receive and administer the vaccines effectively.

The major difference between the two ADIPs is the procurement and supply strategies. RotavirusADIP has yet to announce its longer‐term procurement and supply strategy. However, it has to date followed a more traditional approach with regard to the direct funding of late stage development costs and trials to

37

prove the effectiveness and safety of the rotavirus vaccines in the least developed countries.

Both ADIPs are making use of a Phase 1 programme structure to introduce modern vaccines to GAVI countries as part of the initiative to accelerate availability and to extract lessons about effective implementation, effectiveness and cost‐effectiveness. In the case of PneumoADIP, Prevenar™, an established 7‐valent pneumococcal vaccine will be available in 6‐8 GAVI‐eligible countries in 2008/09. The 7‐valent pneumococcal vaccine will be available in the first GAVI countries about 8 years after it was first licensed in the North America and Europe, rather than the historic lag of 10 to 15 years for a modern vaccine to be available in the developing world. The phase 2 plan for pneumococcal vaccines and the AMC is targeted on the next generation of vaccines containing 10 or more valents that address more of the serotypes that cause health problems in the developing world.

The Phase 1 plan for RotavirusADIP is to make the two rotavirus vaccines available in Latin America upon prequalification. Countries applied for GAVI support in 2007 and were able to receive vaccines in spring 2008. These vaccines were licensed for use in the USA and Europe in 2006, and have passed late stage drug trials for safety and efficacy in Latin America. Since there is some concern about a possible side effect, this introduction will be closely monitored. Simultaneously late stage trials will be conducted in Africa and Asia, at least partially supported by GAVI, the RotavirusADIP and perhaps other donors. If the RotavirusADIP Phase 1 plan is implemented as planned the initial rotavirus vaccines will be available only two years after their availability in the developed world, rather than the historic lag of 10 to 15 years.

2.6.2 HAEMOPHILUS INFLUENZAE TYPE B (HIB)

Hib vaccines were introduced into industrialized countries in the late 1980’s; the first vaccines for infants were licensed for use in the US in 1990. WHO was encouraged to take the lead in preparing for its introduction in developing countries in the early 1990s, but declined, due to its perceived need to focus on maintaining delivery of the six traditional vaccines, rather than introducing new vaccines. It should be noted that in 1990, the six vaccines addressed only a fraction of the burden of infectious disease affecting children in developing

38

countries; for example, diarrhea and pneumonia constituted over half of the disease burden18.

In 1990, the Children’s Vaccine Initiative (CVI) – co‐sponsored by WHO, UNICEF, Rockefeller Foundation, United Nations Development Programme (UNDP), and the World Bank – was launched and stimulated activities in the early 1990s on newly available or soon to be available vaccines. In 1993, a generic framework for evaluating potential vaccines for inclusion in EPI was developed, but it was not used by WHO/EPI to evaluate potential vaccines for meningitis or pneumonia. Activities such as an effectiveness trial of Hib vaccine delivered as a component of the routine immunization programme in the Gambia were carried out mainly funded by United States Agency for International Development (USAID), not in collaboration with EPI but with the WHO ARI Programme. A protocol was developed by WHO and Centers for Disease Control and Prevention (CDC) to assess the Hib disease burden.

In 1996, the CVI took the lead in developing a five‐year plan for the development and introduction of vaccines against acute respiratory infections (ARI) into routine infant immunization programmes. Activities included documentation of disease burden, effectiveness/efficacy trials in developing countries, promotion and support for introduction, an international meeting outlining prospects for prevention for ARIs and activities critical for vaccine supply, including estimates of demand, product specifications and pricing options. However, there continued to be little involvement of WHO/EPI, the primary unit involved in immunization policy at WHO headquarters and in the technical aspects of vaccine implementation at the country level. In 1998, WHO issued a Position Paper on Hib indicating concern for the small amount of disease burden data from Asia and declining a firm recommendation for global use.

As CVI was phased out, GAVI was created in 2000. However, GAVI’s early years focused on financing of vaccines and the Hib agenda for introduction was left to individual organizations and units of WHO to carry out.

As of 2002, less than 10% of children in GAVI countries had received the vaccine. The slow uptake was a result of several factors:

18 Additional historical context on vaccine introduction in developing countries is provided in Appendix C, section 2.

39

• Uncertainty regarding the need for this vaccine ‐ disease burden largely unknown;

• Uncertainty regarding price and availability of the vaccine;

• Limited supply of preferred presentation of vaccine (as 4‐ and 5‐valent combination vaccines);

• Lack of global supply/introduction strategy; and

• Delays in introduction and implementation at the country level.

Observers have noted that GAVI was slow to address the Hib introduction problems, aside from providing financing for countries to purchase the vaccines. The GAVI Financing Task Force worked with countries on financial sustainability but did not develop a supply strategy. Since an ADIP was not created for Hib, manufacturer contacts were with UNICEF or WHO.

By 2005, the situation had not improved much. Disease burden for Hib remained unclear in many countries; as a result WHO disseminated a weak message regarding introduction, and countries using the vaccine were unable to document its impact. Demand forecasts were not robust, as demand in countries without reliable data on disease burden could not be quantified. There still was no supply strategy; supply remained uncertain with just one supplier and one buyer. Prices did not fall and in fact prices for the preferred Diphtheria, Pertussis and Tetanus (DPT) combination products increased.

The Hib Initiative (HI) was created in 2005 by GAVI with funding at $28M plus $9M for the India Hib Vaccine Probe Study. The Initiative was run by a geographically dispersed Executive Committee representing a consortium of four institutions: Johns Hopkins University (JHU), CDC, WHO and the London School of Tropical Medicine and Hygiene. The objective of the Initiative was to expedite and sustain evidence‐based decisions regarding the use of Hib vaccine to prevent childhood pneumonia and meningitis based on the fact that the availability of financing alone had not resulted in optimal uptake. The Initiative’s mandate specifically excluded dealing with supply and pricing issues, per the management committee; these issues were to be addressed by the Supply Task Force within GAVI.

As of 2007 there were two suppliers for a pentavalent vaccine, including Hib.

40

The suitability of rotavirus and Hib as comparators for pneumococcal on the basis of factors such as those discussed in the beginning of this chapter is an important part of the M&E strategy.

2.7 M&E CHALLENGES

This section highlights some of the challenges of implementing an M&E strategy for the AMC and proposes solutions that are detailed in other sections of the report.

2.7.1 AMC ASSUMPTIONS

A key challenge for the AMC itself lies in the fact that meeting both the AMC objectives and the long‐term public health goal of the AMC also requires the successful completion of complementary activities, outside the scope of the AMC, carried out by GAVI, other international organizations and donors in order to support the introduction and use of pneumococcal vaccines in GAVI‐eligible countries. The need for these complementary activities affects the AMC in two ways.

• These complementary activities are required to achieve the long‐term goal of reducing morbidity and mortality from pneumococcal diseases. Unless the vaccines are introduced and used successfully, the public health goal will not be achieved.

• Secondly, the AMC will not achieve its own objective to stimulate investment in pneumococcal vaccine research and production, unless vaccine manufacturers believe that there will be a credible demand for the vaccines from GAVI‐eligible countries. The creation of this demand also requires the successful completion of these complementary activities.

The challenge then for the M&E strategy is to integrate approaches that allow those responsible for implementation of the AMC to monitor the achievements of the complementary activities even though they are beyond the scope of the AMC. Two approaches have been included:

• Monitoring of the outcomes of the complementary activities (e.g. demand for vaccines, immunization rates) is included in the annual monitoring; and

41

• Case studies, based on visits to selected GAVI‐eligible countries, have been included in the process and outcome evaluations to allow for in‐depth exploration, through interviews with a broad range of stakeholders, of the factors affecting the introduction and use of pneumococcal vaccines.

2.7.2 DATA AVAILABILITY

A second major challenge is the availability of data to address the M&E issues. The proposed M&E strategy relies on the availability of considerable information to address the M&E issues. In some cases, new (primary) data will be required to address the issues. This includes:

• Development of the comparator counterfactuals;

• Interviews with stakeholders (donors, host and technical agencies, industry, other stakeholders);

• Survey of industry representatives (repeated every 2 – 3 years);

• Survey of health representatives, bilateral donors, NGOs, and international agencies in GAVI‐eligible countries; and

• Case studies in selected GAVI‐eligible countries.

However, to the extent possible, the M&E strategy is based on the use of existing (secondary) data, in order to reduce M&E costs and the burden on stakeholders of providing or collecting the information. This includes data on:

• Data from literature and document reviews, including M&E reports for other vaccine initiatives;

• Data on the implementation of the AMC (e.g. financial reports, procurement data);

• Data on the pharmaceutical industry, including industry SEC reports, public information releases and annual reports and production capacity;

• Data on vaccine production, shipments and procurement;

• Trends in pneumococcal immunization versus the counterfactual vaccines for GAVI‐eligible country as well as disease burden for both GAVI‐eligible and non‐GAVI‐eligible countries; and

42

• Corporate data on R&D, investments, production, sales, pricing, etc.

AMC Implementation data

Information will be required from the following stakeholders:

• World Bank – reports on financial status of the AMC. This should include financial flows into and out of the AMC account, investment of free cash balances, financing of procurement, and donor contributions versus commitments;

• AMC Secretariat – information on the implementation of AMC activities; status of governance activities, reports and meetings; and

• UNICEF as the procurement agent – information on the purchase of vaccines (by country) and issues associated with procurement processes.

Industry data

Ideally, the M&E strategy would request information from vaccine producers on their perceptions of investment trends in R&D and production capacity. However, it appears unlikely that the vaccine producers will be willing to make this information public. As a result the evaluation will likely have to:

• Make use of publicly available information from sources such as the United States’ SEC and FDA; and

• Conduct intention surveys with industry representatives19 periodically to obtain information on their attitudes and intentions towards R&D, investment, production in the pneumococcal vaccine, as well as an estimate of the confidence in GAVI‐eligible country demand as an indicator of their likely intention to invest. This survey would provide proxy information, such as trend data (e.g. percent change in R&D, investment, production, shipment) and should be updated as part of each periodic evaluation to assess their confidence in future demand, how it is affecting investment and production plans, etc. Since they will only be conducted about every three to four years

19 Industry consultation indicated that companies are not likely to provide detailed information on their products, shipments, etc. The proposal is to request “range estimates” only, and if possible for these to be collected by a trusted, neutral third party.

43

and could be seen as part of an ongoing consultation process with industry, they are not expected to represent an excessive response burden on industry.

• Make use of an independent third party to collect data and to maintain the confidentiality of data collected.

Potential country‐level data

The measurement of the outputs and outcomes at the country level will draw on existing immunization and health data:

• WHO ICE‐T database – provides routine coverage estimates (number of children vaccinated with three doses) and campaign data (number of children vaccinated with one dose). Countries collect this on joint WHO/UNICEF reporting forms. Based on this data, WHO estimates coverage data and the number of deaths averted based on data from the vaccine clinical trials (for example, 7.4 deaths per 1000 vaccinated in the Gambian trial). From this, Disability Adjusted Life Years (DALYs) can be calculated globally and in GAVI‐eligible countries. Deaths averted are converted into life‐years lost and DALYs to quantify societal burden.20 Death estimates are based on proportion of pneumonia deaths worldwide attributable to pneumococcus.

• GAVI/PneumoADIP – conducts special studies to address issues that cannot be covered as part of routine surveillance. These may cover the incidence of pneumonia, pneumonia in Human Immunodeficiency Virus (HIV)‐infected children, invasive disease, outpatient visits and hospitalizations.

• WHO Global Burden of Disease Project – has estimated the number of cases and deaths from pneumococcal disease among children less than 5 years of age globally, regionally, sub‐regionally and by country for the year 2000. The pneumococcal/Hib work was conducted by WHO, PneumoADIP and HI from 2004‐2007 and is expected to be available in early 2008. WHO plans to

20 WHO defines DALY as “… a health gap measure that extends the concept of potential years of life lost due to premature death (PYLL) to include equivalent years of ‘healthy’ life lost by virtue of being in states of poor health or disability (1). The DALY combines in one measure the time lived with disability and the time lost due to premature mortality. One DALY can be thought of as one lost year of ‘healthy’ life and the burden of disease as a measurement of the gap between current health status and an ideal situation where everyone lives into old age free of disease and disability.” WHO website www.who.int/healthinfo/boddaly/en/index.html

44

do time series/ projections of cases and deaths annually. A more in‐depth update would be carried out by GAVI every five years to include published data input into the model.

2.7.3 IDENTIFICATION OF AMC BASELINE

Given that the AMC has been under consideration for several years, it is challenging to identify the baseline from which the evaluation can measure change and attribute change to the AMC. A reasonable suggestion appears to be that the baseline could be the date on which the donors made their formal commitments to provide the $1.5B in funding for the AMC – that is, February 2007. However, given that the AMC discussions have been underway for some time, it would be preferable (where historical information exists and to the extent possible) to set the baseline at the point where pneumoccocal vaccines were first proposed as a pilot (2005), in order to establish the situation before the formal announcement.

Getting information for this date will be possible at least for any components of the baseline for which historical data is publicly available, or will be reported by industry (either directly or through the industry intentions survey).

2.7.4 INTRODUCTION OF PREVENAR™

It is expected that the 7‐valent vaccine, Prevenar™, will be available for purchase, through GAVI, in a limited number of GAVI‐eligible countries in 2008/09. Countries can introduce this vaccine into their immunization programmes and then decide later whether to switch to the newer pneumococcal vaccines when they become available.

To the extent that countries use this vaccine and that it has an impact on morbidity and mortality, it will confound the measurement of the impact of the newer pneumococcal vaccines made available as a result of the AMC. However, since the 7‐valent vaccine is only expected to be taken up in a limited number of countries, the impact on the measurement of the health changes in GAVI‐eligible countries overall is not expected to be significant. The outcome evaluations will have to focus on those countries that did not introduce the 7‐valent vaccine, as they will provide a clearer measure of the impact on health of the newer vaccines for which introduction may have been advanced due to the AMC.

45

2.7.5 ATTRIBUTION OF HEALTH AND OTHER BENEFITS TO AMC

A cost‐benefit analysis of the AMC can be completed once enough data exists to assess the incremental immunization that has occurred because of AMC. This assumes that all of the AMC‐funded immunizations would not have happened if there were no AMC initiative.

The cost side of the equation includes all of the AMC investment costs including the administrative costs to oversee the AMC initiative. The benefits will include the estimated life years and DALYs saved through immunization. This provides a benefit on a year‐by‐year basis as the immunization progresses. The benefits and the costs must then be discounted back to the present based on an acceptable social rate of return.

The attribution of benefits to AMC will be based on assessing what would likely have happened if the AMC did not exist. This requires an analysis of the outcomes that would likely have occurred using the two proposed counterfactuals.

46

3. PROPOSED M&E IMPLEMENTATION PLAN

The purpose of this section is to summarize the proposed implementation plan for the AMC M&E. It includes the identification of the resources, timeframes and costs for the specific M&E activities.

3.1 M&E IMPLEMENTATION STRUCTURES AND REQUIRED RESOURCES

The strategy requires specific structures and dedicated resources to ensure satisfactory implementation of the M&E activities. These would include structures with responsibility for the evaluation, internal resources for M&E management and external resources to undertake the evaluations.

3.1.1 IMPLEMENTATION STRUCTURES

It is important that there be clear responsibilities for the implementation of the M&E strategy. It is proposed that the AMC Secretariat be responsible for the monitoring aspect, which includes managing the monitoring activities and the independent contractors required for the AMC baseline study, as well as producing annual reports, as identified in the requirements for annual monitoring.

It is also proposed that the GAVI evaluation unit be responsible for managing the evaluation aspect, which includes the establishment of the evaluation Steering Committee, providing logistic and management support and managing the independent contractors required to conduct the evaluations. The evaluation unit should establish an evaluation Steering Committee that would include representation from all key stakeholders – donors, GAVI‐eligible countries, industry and civil society organizations, and whose chair would be approved by the IAC. This group, ideally between ten and twelve people independent of groups involved in AMC implementation, should include a balance of sector and evaluation specialists. The Steering Committee would be responsible for reviewing and confirming the evaluation issues, approving the terms of reference for the evaluation (including the profile of, and terms for, the external resources), reviewing and accepting the evaluation report and presenting the results to the Donor Committee and the IAC.

47

Finally, it is proposed that changes to the structure (main components, timelines) or oversight of the M&E strategy be approved by the Donor Committee.

3.1.2 INTERNAL RESOURCES

Dedicated resources need to be allocated in both the AMC Secretariat and the GAVI evaluation unit to ensure that the M&E activities outlined in this report are implemented.

Both units will require resources that have, first and foremost, strong M&E skills and experience, including strong statistical analysis skills. Ideally they will be knowledgeable of the vaccine industry, health economics and the international development health context. They need to have strong writing, reporting and presentation skills. They need also have strong management skills to manage the work of the independent contractors.

3.1.3 EXTERNAL RESOURCES

In addition to the internal resources, the proposed M&E activities will require independent external resources in the form of independent contractors (either individuals or a firm) to implement the baseline study and the periodic evaluations. Depending on the components, these resources need to have a mix of knowledge and experience in:

• International development health evaluation (for the country level surveys and the case studies);

• Health, the pharmaceutical industry and vaccination programmes in the developing world (for developing the counterfactuals, analysis of industry trends and outcomes); and

• Economics with expertise in industrial organization and mechanism design.

They would need strong quantitative and qualitative skills, including experience with interviewing and survey practices, and economic analysis and modeling.

All evaluations should adhere to the OECD‐DAC Evaluation Quality Standards (www.oecd.org/dataoecd/34/21/38686856.pdf).

48

3.2 COSTS AND TIMEFRAME FOR M&E

This section outlines the costs and timeframe for the M&E activities proposed in this report.

All costs are provided in current US dollars. The cost figures are based on estimates in 2008‐year dollars, and will likely have to be adjusted for inflation. Internal resources may be through contract or permanent staff.

The timeframes for these resources are based on two assumptions:

• That the launch of the AMC will occur when the Stakeholders Agreement is signed and that this will be in late 2008. This will be the beginning of the timeframe for the baseline study and process evaluation; and

• That the first Supply Agreement will be signed in 2009 and that the first vaccines will be available by 2013 at the latest. This will be the beginning of the timeframe for the outcome evaluations.

If either of these assumptions does not hold up, then the timeframes for the various evaluation products will need to be changed.

3.2.1 INTERNAL M&E RESOURCES

Given that there are a number of options for staffing the dedicated internal resources for M&E, it is difficult to cost them. However, in order to define the scope of the M&E costs, the team has costed a separate M&E resource within the AMC Secretariat.

Exhibit 3: Anticipated Costs for Internal Resources

Internal Resources Cost One full‐time person for M&E in the AMC Secretariat

$150,000 per year including estimated overhead21

These internal M&E costs are not included in the tables for each of the studies below but are included in the total costs in the summary in Section 3.2.6.

21 $150,000 is based on an estimated $100,000 salary, plus 20% for benefits and 30% for overhead. These numbers will of course vary depending on the nature of the contract for the resource and the location.

49

3.2.2 BASELINE STUDY

The baseline study should be undertaken as soon as possible to establish the baseline for the measurement of the AMC impacts. The baseline study includes data from three studies:

• Collection of industry baseline data – a combination of existing data and a survey of industry representatives;

• Collection of country level data – a combination of existing data and a survey of country level representatives; and

• Development of the models for the counterfactuals.

Timeframe: Early 2009.

Exhibit 4: Anticipated Responsibilities and Costs for Baseline Study

Responsibilities Internal Resources External Resources

Estimated Cost

Management of independent contractors Compilation of reports from contractors

Ongoing AMC cost

Survey of industry representatives

$100,000

Survey of country level health authorities

$100,000

Development of models for counterfactuals

$75,000 estimate

Total $275,000 plus internal resources

3.2.3 ANNUAL MONITORING

Responsibility for annual monitoring will lie with the internal M&E resources. They will prepare an annual report that provides an update on AMC implementation and includes industry and country level data from existing data sources. The annual report will be prepared for the AMC Secretariat and the oversight group.

Timeframe: Annually, beginning in 2009.

50

Exhibit 5: Anticipated Costs and Responsibilities for Annual Monitoring


Estimated Cost

Monitor and report on key events in the AMC implementation Compilation of reports and synthesis for annual reports

Ongoing AMC cost

Total Internal costs only

3.2.4 PROCESS EVALUATION

The process evaluation will focus primarily on assessing the implementation of the AMC activities and the extent to which the complementary activities (outside the scope of the AMC) are being carried out to support the introduction and use of pneumococcal vaccines in GAVI‐eligible countries. However, it will also include an assessment of whether or not the AMC is on track to achieve its objectives.

Timeframes: An evaluation focusing on process (including design) issues should be completed two years after the signing of the AMC (evaluation expected in 2011).

Exhibit 6: Anticipated Responsibilities and Costs for Process Evaluation


Estimated Cost


Ongoing AMC cost

Evaluation of AMC implementation/ process and design issues

$150,000

Industry level data collection and analysis

$100,000

Country level data collection and analysis (including GAVI‐eligible country case studies)

$150,000


51

3.2.5 OUTCOME EVALUATIONS

The outcome evaluations will focus on what has changed as a result of the AMC initiative. The evaluation will be based on existing data from the annual monitoring reports as well as specific data collection that would be contracted to independent contractors. The outcome evaluations will also address design issues, since the design will ultimately affect the AMC outcomes. The last outcome evaluation will also include a study on cost‐effectiveness.

Timeframes: First evaluation focusing on outcomes should be completed four years after the signing of the first Supply Agreement (evaluation expected in 2013) and then every four years (2017 and 2021).

Exhibit 7: Anticipated Responsibilities and Costs for Each Outcome Evaluation


Estimated Cost


Ongoing AMC cost

Evaluation of design and outcome issues

$100,000

Industry level data collection and analysis

$100,000

Country level data collection and analysis (including GAVI‐eligible country case studies)22

$150,000


Final outcome evaluation Above plus Cost‐effectiveness study $100,000


22 Cost may vary depending on the number of countries included in the case studies.

52

3.2.6 TOTAL M&E COSTS

The total costs of M&E for the AMC are $US 3.675M (current dollars) (see Exhibit 9). This amounts to just over 0.3% of the total expenditure forecast for the AMC or about $290K a year. Given the size and sensitivity of the AMC and the potential for the AMC model to be applied more widely, the study team believes that this magnitude of expenditure is reasonable.

3.3 M&E TIMEFRAME

Exhibit 10 provides a summary of the timeline for the M&E activities, in relation to the AMC timeline.

53

Exhibit 8: Summary of M&E Requirements ($USD 2008)

Component 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 Total

Internal M&E resources $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $150,000 $1,950,000

Baseline study $275,000 $275,000

Annual monitoring (see internal M&E resources)

$0

Process evaluation (including design issues)

$400,000 $400,000

Outcome evaluations $350,000 $350,000 $450,000 $1,150,000

Total $425,000 $150,000 $550,000 $150,000 $500,000 $150,000 $150,000 $150,000 $500,000 $150,000 $150,000 $150,000 $600,000 $3,775,000

55

APPENDIX A: LIST OF INTERVIEWS

Andrew Jones, Senior Programme Officer GAVI Dana Dunne, Consultant Susan McAdams, Director, Multinational and Innovative Financing Department World Bank

Kevin Mitchell, Senior Financial Officer Ruth Levine, Vice President for Programmes and Operations, Center for Global Development, Washington DC

Michael Kremer, Gates Professor of Developing Societies, Department of Economics, Harvard University, Cambridge Massachusetts

Economic Expert Group & Implementation Working Group

Jonathan Levin, Associate Professor, Department of Economics, Stanford University

Orin Levine, Executive Director, Associate Professor, International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore MD

PneumoADIP

Angeline Nanni, Director of Vaccine Finance and Supply

RotaADIP John Wecker, Director, Immunization Solutions, PATH, Seattle, Washington

Applied Strategies Consulting Sandra Wrobel, President and CEO

Industry Kate Taylor, Vice President, Market Access

Glaxo Smithkline Ltd. Walter Vandersmissen, Director, Public Partnerships

Serum Institute of India Ltd. Dr. Suresh Jadhav, Executive Director

GAVI Eligible Countries Cameroon Dr. Nomo Emmanuel, EPI Manager, Cameroon

Civil Society Organizations

56

PATH Danielle Altares, Policy and Advocacy Associate

CPHA James Chauvin Director, Global Health Programmes

AMC Donors Sue Szabo, Senior Adviser, Development Policy, Finance Canada

Canada Catherine Palmier, A/Manager, Multilateral Health Institutions and Global Programmes Group, CIDA

Italy Carlo Monticelli, Director for International Financial Relations, Ministry of Finance

Norway Ine Måreng

Russia Alexey Proskuryakov, Advisor to ED, World Bank

U.K. Sebastian Ling, Programme and Policy Manager, GFDD, DFID

Gates Foundation Hannah Kettler, Programme Officer, Global Health Policy and Finance

UNICEF Ann Ottosen, Contracts Officer UNICEF Immunization Team

APPENDIX B: DEFINITIONS

Activities

Represent the main types of work carried out or the major allocation of resources for the initiative. They represent all the activities that are required for the initiative to reach its objectives (outcomes), but do not normally include the administrative enablers (finance, HR, systems, etc.) of the initiative.

AMC period

The period during which AMC funds are available for the purchase of pneumococcal vaccines.

57

Baseline Study

Defined in the OECD DAC glossary of terms as “An analysis describing the situation prior to a development intervention, against which progress can be assessed or comparisons made.”

Effectiveness

A measure of the extent to which an aid activity attains its objectives.

Efficiency

A measurement of the outputs – qualitative and quantitative – in relation to the inputs. It is an economic term that signifies that the aid uses the least costly resources possible in order to achieve the desired results.

Evaluations

Individual systematic studies conducted periodically or on an ad hoc basis to assess how well a programme is working. They typically examine a broader range of information on programme performance and its context than is feasible to monitor on an ongoing basis.

Impact

The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended.

Indicator

Provides evidence that a certain condition exists or certain results have, or have not, been achieved.

Long‐term outcomes

Represent the final or strategic impacts of the initiative and are dependent upon achieving the immediate and intermediate outcomes. Often the long‐term outcomes are subject to influences beyond the initiative and it is difficult to measure them and/or attribute their achievement to the initiative. However, being able to demonstrate the achievement of the immediate and/or intermediate outcomes helps to construct the argument that they have contributed to long‐term outcomes.

58

Monitoring studies

Monitoring is the ongoing monitoring and reporting of programme accomplishments, particularly progress toward pre‐established goals. It is typically conducted by programme or agency management. Performance measures may address the type or level of programme activities conducted (process), the direct products and services delivered by a programme (outputs), or the results of those products and services (outcomes). A “programme” may be any activity, project, function, or policy that has an identifiable purpose or set of objectives.

Outputs

Represent the key tangible products (goods/services) that arise from the main activities. They are, thus, the first direct results of the initiative.

Outcomes

Reflect why the initiative is doing the activities and producing the outputs. Most outcomes in a logic model typically have an action word associated with them (e.g. better, improved, ensure) to more proactively represent (and purposely measure) the (desired) consequences of the activities and outputs.

Outcome Evaluations

Outcome evaluations assess the extent to which a programme achieves its outcome‐oriented objectives and, specifically, the impact of the programme on the target group. They focus on outputs and outcomes (including unintended effects) to judge programme effectiveness but may also include issues process‐related issues to understand how outcomes are produced.

Process Evaluations

Also called implementation or formative evaluations. Process evaluations “verify what the programme is and whether or not it is delivered as intended to the targeted recipients.” They focus on the extent to which a programme is operating as it was intended. They typically assess programme activities’ conformance to statutory and regulatory requirements, programme design, and professional standards or customer expectations.

59

Post‐tail period

The period after the expiry of all the pharmaceutical firms’ obligations to provide vaccines.

Relevance

The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.

Sustainability

Measurement of whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.

Tail period (post‐AMC)

The period after the AMC funds have been exhausted but during which pharmaceutical firms are under obligation to provide vaccines at specific prices and in specified quantities to GAVI‐eligible countries.

APPENDIX C: AMC CONTEXT

1. INTRODUCTION

Vaccines are cost‐effective solutions to infectious diseases causing mortality and morbidity in developing countries. Pneumococcal disease is one of the leading causes of mortality in children under five in the poorest countries23 and the development of effective vaccines for this population is well underway. Accelerating the provision of these vaccines at affordable and predictable prices for use in the world’s poorest countries has been identified as a priority by many in the international community. Widespread introduction and use of new pneumococcal vaccines is expected to lead to significant reductions in under‐five mortality and disease burden.

23 WHO Statistical Database (WHOSIS 2008).

60

In 2005 an AMC Working Group published “Making Markets for Vaccines: Ideas to Action.”24 The report noted that vaccines are cost‐effective public health interventions and that about half of the global investments in health research R&D can be attributed to the private sector. The report considers how to promote private investment for vaccine development, incentives for commercial investment and options for financing R&D for diseases of developing countries. The authors concluded that an AMC is the best option to accelerate the development and introduction of new vaccines and provides input for design, estimates for the size of an AMC, industry requirements, a description of an Independent Adjudication Committee to oversee the process, and model term sheets for Framework Agreements and Guarantee and Supply Agreements.

In December 2005, the Italian Finance Minister Giulio Tremonti presented a report to the G‐7 Finance ministers entitled “AMC for Vaccines; a New Tool in the Fight Against Disease and Poverty.” Largely based on the work of the AMC Working Group, completed with strong support of the World Bank (which led much of the analytical work) and with input from a team at the Italian Ministry of Economics and Finance, the report galvanized support for the AMC concept among some of the G‐8 countries.

Based on an estimated fifteen‐year lag in the availability of new vaccines to the developing world, the next generation of pneumococcal vaccines that address serotypes most relevant to disease in the developing world would not begin to be introduced in the developing world before 2025. Establishment and implementation of a pneumococcal vaccine AMC, modelled on the proposal put forward by the AMC Working Group, is anticipated to provide at least two new vaccines at affordable and predictable prices by 2013, and potentially a third vaccine within the AMC timeframe.

Since 2005, the AMC conceptual design has evolved somewhat. The working papers produced by the EEG (April 2008) and the IWG (July 2008), have further refined the conceptual design of the AMC. The IWG was asked to develop recommendations that are sufficiently specific, detailed and operationally

24 “Making Markets for Vaccines: Ideas to Action”, Center for Global Development, Advanced Market Commitment Working Group, Ruth Levine, Michael Kremer, Alice Allbright, Co‐Chairs, Washington DC, 2005.

61

feasible to allow the donors to finalize the offer to be presented to industry in the legal documentation.

2. PNEUMOCOCCAL VACCINE CONTEXT

2.1 HISTORICAL PERSPECTIVE: SUPPLY AND DEMAND FOR VACCINES IN DEVELOPING COUNTRIES

In the late 1970s, following the successful eradication of smallpox, WHO prepared for the expansion of immunization to protect against six diseases which were endemic throughout the world and for which suitable vaccines already existed: tuberculosis, diphtheria, whooping cough, tetanus, measles and polio. UNICEF joined WHO in 1982 to set a target to increase immunization from under 10% to 100% in 8 years.

At that time, the vaccine industry had an excess of capacity gained through production efficiencies. This excess capacity was sold to WHO and UNICEF at costs just above marginal cost – considerably lower than the pricing in industrialized markets. However, fully servicing the potential ten‐fold increase resulting from demand from the public sector markets of developing countries required considerable investments in production capacity. The major supply and demand challenge during this period was not to increase production capacity to meet the demand, but to manage the demand so it did not exceed the available supply; this would have increased the per dose cost of vaccines beyond affordable levels. UNICEF and industry worked together to reduce the risk for industry while still allowing it to produce according to its best efficiencies.

Managing the demand for vaccines without slowing down immunization programmes was more difficult. The large developing countries continued to supply locally produced vaccines to their populations even though the vaccines did not meet international standards. As a result, WHO established a department to work with national regulatory authorities in order to bring them up to international standards and hence to bring local production capacity up to international standards. This resulted in multiple benefits. The large developing country populations could now receive vaccines that were known to be safe and effective.

62

Expansion occurred throughout the 1980s and, by 1990, WHO and UNICEF declared success, as more than 80% of the world’s children had been immunized against the six diseases.

At the World Summit for Children in 1990, the concept of the CVI was launched. The purpose of the CVI was to harness new technologies and vaccines to advance the immunization of children. It was proposed that the ideal CVI vaccine must be: administered as a single dose; effective near birth; heat stable, contain multiple antigens; effective against diseases not currently targeted in the EPI, and be affordable. At that time the six vaccines included in the EPI addressed only a fraction of the burden of infectious diseases affecting children in developing countries. For example, the six vaccines included in the EPI did not target diarrhoeal disease and pneumonia, which accounted for over 50% of under‐five deaths in developing countries in 1990.25 As CVI evolved, issues of vaccine production and supply came to the forefront.

Since the vaccines administered through EPI had been supplied at pennies per dose, the international community expected that new vaccines would also be supplied at the same price level. However, industry was not prepared to make the investments necessary to provide new vaccines to developing country markets that would not pay prices high enough to cover the costs of development and production incurred to service these markets.

After a one‐year review of immunization‐related activities and issues by interested partners, the Global Alliance for Vaccines and Immunization (GAVI) was formally launched in 2000 at the World Economic Forum in Davos. The Global Fund for Children’s Vaccines was established by GAVI with an initial grant of $750 million from the Bill and Melinda Gates Foundation. A major objective of the Fund was to procure new vaccines and make them accessible to and affordable by the poorest countries. The result was the availability of new vaccines such as Hepatitis B and Hib by the Global Fund with financial support provided to developing countries based on a tiered system according to per capita Gross Domestic Product (GDP).26

25 State of the World’s Children, 1993.

26 The Global Alliance for Vaccines and Immunization Information Note, 6/11/99.

63

2.2 ACCELERATED DEVELOPMENT AND INTRODUCTION OF PLANS (ADIPS)

One of GAVI’s strategic objectives is to accelerate the development and introduction of new vaccines and technologies into developing countries.27 A 2002 study conducted by McKinsey & Company for GAVI indicated that, in countries traditionally poorly served by the pharmaceutical industry, earlier access to vaccines could be achieved through the active engagement of public sector stakeholders. ADIPs for rotavirus vaccines and pneumococcal vaccines were launched in 2002 to shorten the lag between vaccines being proven safe and effective for use in the industrialized world and their introduction in developing countries. Each ADIP received approximately $50 million over 2003‐2008.

The ADIP activities include:

• Assessing regional/country specific burden of disease, estimating the effectiveness and cost‐effectiveness of the vaccine once it is in use, and establishing the vaccine’s potential impact;

• Identifying and reducing the perception and operational barriers for vaccine introduction where needed; and

• Working within public‐private partnerships to ensure sufficient supply for the health benefiting products to children in developing countries.

2.3 RATIONALE FOR ACCELERATING THE INTRODUCTION OF PNEUMOCOCCAL VACCINES

Pneumococcal disease is the leading infectious cause of child mortality worldwide.

In 2005, WHO estimated that 1.6 million people die of pneumococcal disease every year, including the deaths of 0.7 ‐ 1 million children of less than 5 years of age, 90% of whom are estimated to live in developing countries.28

A pneumococcal vaccine has been shown to be effective in industrialized countries. In 2000, a licensed 7‐valent pneumococcal conjugate vaccine (PCV‐7)

27 Material for this section is drawn from the GAVI website.

28 Weekly Epidemiological Record, 23 March 2007.

64

became available and is in use in more than a dozen industrialized countries. The vaccine has resulted in a 69% decrease in the incidence of invasive pneumococcal disease in children under two in the United States. More than twice as many cases of pneumococcal disease were prevented among the unvaccinated. This illustrates the herd effect of the vaccine, as well as a significant decrease in disease among adults.

Pneumococcal vaccines with a different serotype composition than the current 7‐valent vaccine will be best suited for developing countries, due to the predominance of serotypes not included in the current vaccine formulation. Such vaccines, which are in late‐stage development, have the potential to prevent a significant proportion of pneumococcal disease, including pneumonia, meningitis, and sepsis, and otitis media, in the poorest countries. Introduction of an effective pneumococcal vaccine as a public health tool in developing countries is especially important because:

• HIV infection and other conditions associated with immunodeficiency greatly increase the likelihood of contracting pneumococcal disease.

• Resistance of S. pneumoniae to commonly used antibiotics is increasing.

• Prevention of pneumococcal meningitis by vaccination will prevent the serious disabilities incurred by survivors.

• Pneumococcal vaccines are cost effective.29

• The impact of vaccination with pneumococcal conjugate vaccines is likely to exceed its direct effects due to herd immunity, protecting unimmunized children and vulnerable adults.

However, although industry has the production technologies, it lacks the incentives to make the investments to produce this vaccine in sufficient quantities to fill the requirements of the developing countries. Developing

29 A Harvard University study concluded that: vaccinating all infants in GAVI‐eligible countries at DPT rates would prevent approximately 470,000 deaths per year among children under 5 years of age and provide a weighted average cost‐effectiveness ratio of $22 per DALY (disability adjusted life year) or $690 per death averted, and the net costs of vaccination would be $324 million annually, based on reduction of medical expenditures by more than $558 million per year and costs of procuring and delivering vaccine estimated at $882 million annually.

65

countries are increasingly aware of the potential benefits of a pneumococcal vaccine, but they lack the resources to pay for it. Industry is unlikely to scale‐up production to accommodate the potential demand for this vaccine by the poorest countries without the assurance of a return on its investments.

2.4 PNEUMOCOCCAL VACCINE PIPELINE

Since the 1940s, polyvalent polysaccharide pneumococcal vaccines have been used for older adults or high‐risk persons of more than two years of age, but these vaccines produce poor immune responses in children under two years and are not licensed for use in this age group.

Wyeth Pharmaceuticals produces a PCV‐7, marketed as Prevnar™ in North America, or Prevenar™ outside of North America, the only commercially available pneumococcal conjugate (protein linked to polysaccharide) vaccine. Prevnar protects against pneumococcal serotypes 4, 6B, 9V, 14, 18C, 19F, 23F. The vaccine is licensed in more than 70 countries for use in children of less than five years of age and has been introduced into immunization programmes in more than a dozen high‐income countries. In a large field study in the US, protective immunity against invasive disease caused by vaccine serotypes was 97.4% among children receiving three doses and 93.9% among children receiving at least one dose. Surveillance data from the US indicates that the herd immunity effect prevents twice as many cases as the direct effects of vaccination alone.30

Relatively little information is available on the outcome of PCV‐7 in developing countries, but it is not expected to confer the same level of immunity because the distribution of serotypes in most developing countries differs from that in the high‐income countries. In particular, it does not provide protection against serotypes 1 and 5 that, together with serotype 14, are the most frequent isolates in GAVI‐eligible countries.

Nevertheless, in March 2007, WHO issued guidance calling for the introduction of pneumococcal conjugate vaccines into immunization programmes in

30 Centers from Disease Control and Prevention, Direct and Indirect Effects of Routine Vaccination of Children with 7‐Valent Pneumococcal Conjugate Vaccine on Incidence of Invasive Pneumococcal Disease‐United States, 1998‐2003, MMWR 2005; 54:893‐897.

66

developing countries, beginning with the currently licensed PCV‐7.31 Countries can begin saving lives with the available vaccine and then decide whether to switch to one of the newer vaccines when they become available.

Other pneumococcal conjugate vaccines that expand the serotypes coverage of PCV‐7 are in late stages of development.

PCV‐9, comprised of PCV‐7 serotypes plus serotypes 1 and 5 in a lyophilised formulation (produced by Wyeth Pharmaceuticals), has been evaluated in Africa. A randomized controlled trial in the Gambia found that the efficacy of three doses against vaccine‐type invasive pneumococcal disease was 77%, and Gambian children experienced a 16% reduction in all‐cause mortality.32 A study in South Africa found 83% efficacy in HIV‐negative children and 65% efficacy in HIV‐positive children.33

Two additional conjugate vaccines, with broader serotype coverage suitable for use in the developing world, are expected to be approved for use in industrial countries in the near future. A 10‐valent vaccine containing PCV‐7 serotypes plus 1, 5, and 7F (PCV‐10 produced by GSK) is being evaluated in Panama and Argentina using pneumonia as an outcome measure. GSK has now submitted its dossier to the European Medicines Agency (EMEA) for licensure, implying that studies have already been completed in the US and Europe. GSK has also submitted a Product Summary File to WHO in order to achieve a pre‐qualification of the vaccine.

Wyeth Pharmaceuticals has indicated that it expects licensure of a 13‐valent vaccine containing PCV‐10 plus serotypes 3, 6A and 19A by late 2009 ‐ 2010. The FDA has granted fast‐track designation for the vaccine and based, on the completion of the filing during first quarter 2009, the first national licensure could be achieved late 2009. Wyeth is evaluating the 13‐valent vaccine in Phase 3

31 World Health Organization, Pneumococcal Vaccines, Weekly Epidemiological Record, No. 12, 2007, 8:93‐104.

32 Efficacy of a Nine‐Valent Pneumococcal Conjugate Vaccine against Pneumonia and Invasive Pneumococcal Disease in The Gambia: Randomized, Double‐Blind, Placbo‐Controlled Trial, The Lancet, Vol. 365, 2005.

33 A Trial of a 9‐Valent Pneumococcal Conjugate Vaccine in Children with and without HIV Infection, NEJM, 349; 14, 2003.

67

trials in India. It is anticipated that the 13‐valent vaccine will replace the 7‐valent vaccine worldwide. Efficacy trials will not be necessary for regulatory approval; non‐inferiority will be necessary for approval, but this will require only Phase 1 and Phase 2 trials. A major benefit of the 13‐valent vaccine for the USA is the presence of serotype 19a, which is a drug‐resistant strain.

The Pneumococcal Global Serotype Project Version 1.0 estimated34 the proportion of invasive pneumococcal disease in children of less than 5 years of age caused by serotypes included in existing or developed vaccine formulations (assuming that all vaccine formulations with 6B would also provide cross protection against serotype 6A) for all World Bank Regions. In every region, the serotypes included in the existing 7‐valent vaccine are estimated to account for 54‐75% of invasive pneumococcal disease among children of less than 5 years of age, with substantial regional variability; the lowest estimated coverage is in Africa and Asia. The serotypes included in the 10‐valent formulation produced by GSK are estimated to account for 75‐90% of serotypes causing invasive pneumococcal disease in children less than 5 years old, with considerable increases over the 7‐valent serotype coverage for the Latin America and Caribbean, Africa and Asia regions. The serotypes included in the 13‐valent formulation produced by Wyeth Pharmaceuticals account for 80‐92% of serotypes causing disease in children less than 5 years old in every region; regional variations are minor.

Approximately twenty other pneumococcal vaccines35 are in early stages of development. Several of these products could be licensed between 2015 and 2019. Potential emerging market suppliers for conjugate vaccines include manufacturers in Brazil, Cuba, India and China; producers in these countries may manufacture their own vaccines or negotiate a parallel licensure and technology transfer from a multinational to produce an existing vaccine. Several multinational companies have discontinued work on their pneumococcal conjugate vaccines but could potentially resume their efforts. In addition, “common protein” vaccines, which have the potential to offer broader protection, are in development; one company is expected to start Phase 1 trials in 2008 and its product could possibly be available during the AMC period.

34 Angeline Nanni, personal communication

35 Anon. Pneumococcal for childhood immunization. WHO Position Paper. In press, 2007

68

Pneumococcal vaccine production has some characteristics which are different from other vaccines – for example, conjugate vaccines such as pneumococcal are thought to currently have longer, more complicated, and more expensive production processes than non‐conjugate vaccines. However, pneumococcal conjugate vaccines are similar to other vaccines in the sense that the industry needs a good probability of earning substantial revenues to consider making the necessary investment in R&D, licensing and production. Developing countries cannot afford to buy vaccines at the prices and volumes that will give potential producers a satisfactory return on investment (ROI).

2.5 OUTLOOK FOR SUPPLY AND DEMAND FOR PNEUMOCOCCAL VACCINE

For pneumococcal vaccines, unlike traditional EPI vaccines, there are several factors that complicate the supply and demand equation. S. pneumoniae serotypes responsible for causing pneumococcal disease vary from region to region, country to country and within countries with implications for vaccine design, demand and supply. The efficacy of a specific conjugate vaccine will vary from population to population, making licensing of the vaccine in each country more dependent on local conditions and local data than for other vaccines. Producers may produce vaccines designed to address different serotypes, which makes each vaccine more or less effective in different epidemiologic regions.

Large multinational producers may replace products most appropriate for industrialized countries with “global” vaccines composed of the most important serotypes worldwide.

Different conjugation processes are currently being used by different producers with varying degrees of efficiency. Differences between yields associated with different conjugation processes are the key cost driver. Once regulatory approval is obtained using one process, firms will typically not want to switch to another process, since this will require new regulatory approval. In consequence, it is unlikely that production costs for already existing plants will decline substantially below forecasted levels.

69

APPENDIX D: PNEUMOCOCCAL VACCINE PROFILE

This report reflects the situation as of August 2008 and incorporates recommendations of the EEG in April 2008 and the IWG in July 2008. The AMC conceptual design must still be operationalized through the legal offer that will be presented to industry.

1. CONCEPTUAL MODEL OF THE PNEUMOCOCCAL VACCINE AMC

Although vaccines are among the most effective tools of modern public health, the private sector has proven to be reluctant to make the investments to develop and produce vaccines specifically for the poorest countries because of limited potential for commercial returns. The lack of predictable demand inhibits the rapid development of, and access to vaccines that would prevent diseases that kill millions in the developing world.

In the past, some vaccines became available through donor support to the developing countries many years (e.g. 15 years) after the product had first been introduced in the industrial and middle‐income countries. At the time of launching a new product, the producers priced their vaccines in the industrial and middle‐income countries at levels that were adequate to recover the full development and production costs. In later stages of the product life cycle, after producers have recovered the costs of R&D and the fixed cost investments in production capacity, and when the unit cost of producing the vaccines has declined significantly due to economies of scale and improved production systems, producers have made the vaccines available to developing countries at lower prices, either because of competition, or because of pressure from the major donors and international health organizations. Under these conditions, vaccine producers can profitably supply vaccines to developing countries, as long as the now low variable costs of production are covered. Even at these low prices many of the poorest developing countries have been unable to afford the vaccines.

The pilot AMC for pneumococcal vaccines is a $1.5 billion financial commitment from donors to subsidize the future purchase of a vaccine that is not yet

70

available, provided that the vaccine meets the TPP performance standard36 and is made available at a pre‐established price. The donor commitment is expected to encourage one or more vaccine suppliers to invest in product development and scale up production capacity to serve developing countries’ needs by creating market‐like incentives. The qualifying manufacturer(s) are assured access to a donor subsidy if they develop a product which is in demand by the countries, and subsequently offer a lower, long‐term post‐AMC price for a defined number of doses after their share of donor funding is depleted.

The AMC will be available for suppliers until the $1.5 billion has been committed or until 2020. Funds that are not committed at this point will be returned to AMC donors or transferred to GAVI for use in future procurement of pneumococcal vaccines.

During the life of the AMC fund and during the post‐AMC period, developing countries are expected to pay part of the guaranteed price through the GAVI co‐financing schedule. GAVI, with additional support from donors, has committed to provide financial support for the vaccine procurement during the AMC subsidy period and the post‐AMC price to be added to the co‐financing contribution paid by recipient countries, in accordance with its standards of support for countries of different income levels.

The EEG recommended linking a manufacturer’s potential share of AMC funds to an explicit long‐term supply commitment. In essence under the Supply Agreement the manufacturer would commit to make product available to satisfy a fraction of the long term GAVI demand. In return the company would be eligible for a share of the AMC funds. The IWG further recommended a minimum 10‐year commitment, starting no later than 5 years after signing the Supply Agreement to allow build‐up of capacity. UNICEF with financial backing from the GAVI Fund would provide a procurement guarantee of 20% of the annual committed supply for the first year of the 10 year commitment, 15 percent in the second year and 10% in the third year to partially offset the risk that demand does not materialize as forecast.

Key factors that are expected to contribute to the AMC achieving its objectives are:

36 http://www.who.int/immunization/sage/target_product_profile.pdf

71

• Creating incentives – Manufacturers that have registered and been approved as AMC‐eligible manufacturers and that subsequently have their product pre‐qualified and approved by the IAC to meet the TPP, will receive an AMC price of $7 per dose for a certain quantity of doses, as well as an allocation of a portion of the demand that materializes. This is based on the supply commitment they make. The frontloaded price is intended to allow manufacturers to recoup investment costs in establishing dedicated manufacturing capacity sooner. Furthermore, the limited purchasing guarantee for the first three years of annual supply from the dedicated capacity is expected to provide additional incentives to industry.

• Ensuring a predictable low, long‐term and sustainable price and predictable supply in the post‐AMC period – Manufacturers that are approved as AMC‐eligible manufacturers with products approved as meeting the TPP, will set a tail price subject to a cap of $3.50. The tail price is the co‐payment that will be paid by GAVI and the supported countries throughout the AMC. The pre‐ and post‐AMC prices and supply requirements will be reflected in the Supply Agreements, but may be subject to inflation adjustment and renegotiation in extraordinary circumstances if market conditions warrant. As with the co‐payments, GAVI will finance the tail price and countries will make their contribution through the financing of a portion of the vaccine doses needed as requested under the GAVI co‐financing policy and in accordance with standard GAVI procedures. The Supply Agreements will commit the manufacturers to supply a certain number of vaccine doses on an annual basis to meet developing country demand for an agreed time period (10 years). The manufacturer will have a maximum of five years from signing of the Supply Agreement to establish the dedicated capacity, at which time the 10‐year supply commitment comes into force.

• Encouraging R&D to optimize vaccines for developing countries – The TPP specifies the minimum product specifications required for consideration for AMC financing. Pharmaceutical companies already working on new pneumococcal vaccines may have to carry out additional R&D to meet the TPP specifications.

• Demonstrating benefits of pneumococcal vaccines – The early availability of effective pneumococcal vaccines to the poorest developing countries is expected to demonstrate the benefits of improved health to the local decision

72

makers, and to encourage them to make use of the low post‐AMC price and to continue with the vaccination programme after the conclusion of the AMC.

For the AMC to be successful, the following have to occur:

• Donor commitments for the $1.5 billion must be finalized in a form that is legally binding;

• Manufacturers must determine that participating in the AMC under the terms and conditions established is consistent with their business model and business strategy overall. Manufacturers will also have to demonstrate the financial and probably corporate social responsibility impacts to their Boards;

• The demand forecast for pneumococcal vaccines must be credible to industry;

• Regulatory approval and licensing of the pneumococcal vaccines both at the country of producer and, if applicable, at recipient level proceeds in a timely manner;

• WHO pre‐qualification proceeds in a timely manner;

• AMC processes – supplier registration and vaccine approval – proceeds in a timely manner;

• Unanticipated technical/scientific obstacles do not arise (e.g. interference of additional serotypes with other EPI vaccines, etc.); and

• Donors, GAVI and recipient countries provide adequate financial resources and partners provide technical support to introduce the pneumococcal vaccines into existing GAVI‐eligible countries’ health systems and to deliver vaccines during the AMC period and post‐AMC.37

37 The “post‐AMC” period is defined as the period after the AMC funds have all been expended but industry is under obligation to supply vaccines at the agreed‐upon tail price. It is also referred to as the “tail period.”

73

2. AMC STAKEHOLDERS

There are seven stakeholders or groups of stakeholders in the AMC:

• Donors;

• IAC;

• Host agencies – GAVI, including the AMC Secretariat, and the World Bank;

• Technical agencies – WHO and UNICEF;

• Other ad hoc committees put in place for the AMC

• Industry; and

• GAVI‐eligible countries.38

The linkages among the stakeholders in each group are shown in Exhibit 11. The stakeholder model also represents stakeholders at the design, as well as the implementation, stage of the AMC. For example, the IWG and EEG are depicted in the model, but their role is now finished. Some of the roles and relationships are still being negotiated among the stakeholders. The model represents our understanding of the various relationships at the time of the study.

The roles and responsibilities for some stakeholders have not yet been finalized. This section reflects the information available in August 2008.

38 Material for this section is drawn primarily from “Framework Document: Pilot AMC for Pneumococcal Vaccines”, 26 October 2006 and from the IWG report. Additional information was taken from a few Terms of References for AMC committees and from interviews with key AMC stakeholders.

74

Exhibit 11: Stakeholder Map for Pneumococcal Vaccine AMC

Legend:

Donor Committee

Independent Assessment Committee

World Bank

TPP Expert Group

Economic Expert Group

M&E Sub-group

UNICEF (and procurement agent)

Industry

GAVI-eligible

countries

Technical agencies

AMC Donors

Provide data for AMC M&E

Fund

/ in

form

atio

n flo

ws

Advi

sory

role

Pro

gram

me

supp

ort

Adv

isor

y ro

le

Advi

sory

role

SAGE

WHO

Host agencies

Support

Advisory role Oversight

IAC Nomination Independent

Panel

Adv

isor

y ro

le

Programme support

AMC decision-making bodies

PneumoADIP (or post-PneumoADIP

entity)

GAVI

AMC Secretariat

Fund

flow

sVa

ccin

e su

pply

Prog

ram

me

supp

ort

Fund / information flows

Fund flows

Implemen-tation

Working Group

Ad hoc committees set up for AMC design

Fund

flow

sVa

ccin

e su

pply

Note:Some relationships (specifically with respect to fund flows) are not yet confirmed

75

2.1 DONORS

The current donors include the governments of Italy, the United Kingdom (UK), Canada, Russia and Norway and the Bill & Melinda Gates Foundation.

The key role of the donors is to provide credible financial commitments through pledged funding and to oversee the establishment and implementation of the AMC mechanism (including all its structures and governance mechanisms).

To do this, the donors have established a Donor Committee (DC), with representation from each of the six donors. It works with the World Bank to secure and structure financial commitments into a single credible commitment and agree on procedures to manage donor payments.

Subsequent to the establishment of the AMC mechanism, the DC would retain an oversight role but not be involved in the day‐to‐day management of the AMC Agreements. In preparation for this oversight role, the donors initiated the development and implementation of this M&E plan, though an M&E Sub‐Group.

2.2 INDEPENDENT ASSESSMENT COMMITTEE

The IAC is a body of nine representatives from the fields of public health, health economics, vaccine business development, contract/legal, client performance and delivery systems. An independent selection panel comprising the GAVI Executive Secretary, the World Bank, WHO, International Federation of Pharmaceutical Manufacturers & Associations (IFPMA) and the Developing Countries Vaccine Manufacturers Network (DCVMN) has appointed the IAC members.

The IAC responsibilities include:

• Overseeing the AMC processes;

• Approving the TPP, based on the recommendation of the WHO’s Strategic Advisory Group of Experts (SAGE);

• Reviewing the financial terms once the TPP is approved;

• Relating with WHO for the review of producer applications;

• Ascertaining, based on the WHO pre‐qualification process, whether the TPP is met by producers;

76

• Monitoring progress of the AMC through periodic reports provided by the and the World Bank; and

• Resolving disputes.

2.3 HOST AGENCIES

Two international agencies will be the host agencies for the AMC: GAVI and the World Bank.

GAVI ALLIANCE (INCLUDING PNEUMOADIP)

GAVI is a partnership of public and private sector resources with a single, shared focus: to improve child health in the poorest countries by extending the reach and quality of immunization coverage. It provides support for strengthening health systems to deliver immunization and other health services, accelerating the uptake of and improving vaccine supply security for new and underutilized vaccines, and financing vaccine purchases and/or developing alternative financing for vaccines, leading to long‐term predictability and sustainability.

GAVI carries out programmatic and operational functions for the pneumococcal AMC, mainly through an AMC Secretariat, along with UNICEF and the World Bank. The GAVI Board, which includes representation from many of the other AMC stakeholders (e.g. donors, developing countries, research and technical institutes, civil society and the pharmaceutical industry), provides oversight for the Secretariat and PneumoADIP (including approval of implementation plans, work plans, etc.).

Together with the World Bank, GAVI will work with stakeholders to draft, negotiate, and obtain signatures for agreement(s) that will clarify objectives and goals of the AMC and describe the roles and responsibilities of stakeholders to assure objectives and goals will be reached efficiently and effectively.

GAVI will produce a strategic (long term) demand forecast that projects out 10 or more years to be updated twice per year, and then UNICEF will produce a rolling supply chain forecast (short term) that projects 12 months, to be updated monthly based on country applications to the manufacturers to help them better match their supply and production capacity to long‐term demand expectations. GAVI will finance a portion of the vaccines and countries will pay for another

77

portion of the vaccine in accordance to GAVI’s requirements for co‐financing and to standard procedures during the AMC period and post‐AMC.

Unless otherwise specified, GAVI’s existing practices will be applied during AMC implementation. This includes, for example, rules regarding country co‐financing and GAVI contributions for countries in different “capacity to pay” categories.

GAVI has suggested that, through its evaluation unit, it provide the staff and funding for implementation of the evaluation exercises. To ensure independence, oversight of the evaluations will be provided by an independent Steering Committee and external consultants will carry out the actual evaluation activities.

PneumoADIP is a public‐private partnership created by GAVI and housed at the Johns Hopkins Bloomberg School of Public Health. PneumoADIP provided the strategic demand forecasts and technical input for the pneumococcal vaccine AMC. Since its inception, PneumoADIP, in collaboration with WHO and UNICEF, has helped to prepare countries for the introduction of pneumococcal vaccines. It has provided technical assistance for R&D activities at the global level and technical input for reports and other communication. Support at the country level includes strengthening surveillance capacity, providing laboratory training, cost‐effectiveness studies, refining demand forecasts, funding small grants, and providing technical information for advocacy and decision‐making.

Outside the scope of the AMC, but still critical to the success of AMC’s expected public health outcomes, GAVI (with partners) will take the lead role in providing technical and financial support to GAVI‐eligible countries introducing the new vaccines in order to help promote uptake of the vaccines and ensure governments have data for evidence‐based decision‐making. This includes PneumoADIP or other relevant organizations’ activities in surveillance, research, communications and advocacy, and financing to assist in strengthening vaccine delivery systems. This support may be used for social mobilization, injection supplies, transport and storage of vaccine, training of health workers, collection of data, demand forecasting and for other purposes. Major PneumoADIP activities will cease at the end of 2008. A competitive bidding process is underway to retain a new outsourced entity to conduct support activities around new vaccine introduction and is planned to be in place by the end of 2008.

78

AMC SECRETARIAT

The AMC Secretariat is hosted by GAVI. It will initiate programme activities to support the development and implementation of the AMC, as described in the formal Agreements. The Secretariat39:

• Coordinates AMC correspondence and documentation, convenes and reports on AMC meetings;

• Develops an overall AMC implementation plan;

• Develops an annual AMC work plan;

• Implements a transparent process, with the World Bank, to identify members for the IAC;

• Provides annual monitoring reports on GAVI activities to the IAC for their review;

• With technical input from relevant organizations, communicates activities and accomplishments of the pneumococcal vaccine AMC; and

• Monitors and provides data on the progress of vaccine developers/producers and demand for vaccine by countries and reports to the GAVI Alliance Board, the IAC and the DC.

WORLD BANK

The World Bank will provide financial, legal and contractual services related to donor funding for procurement of vaccine (producer /developer services) Specifically, the Bank is responsible for:

• Oversight of financial administration and financial reporting to donors;

• Reporting periodically to the IAC on the financial aspects of the AMC;

39 Terms of Reference for the Secretariat exist in draft form. Once the Donor Committee confirms them, the responsibility of the Secretariat may need to be updated to reflect these.

79

• Drafting and approving legal documents, including contracts and consensus agreements, mechanisms/structures for donor credible funding, supply and guarantee agreements;

• Implementing a transparent process, with the AMC Secretariat, to identify members for the IAC;

• Providing financial services, tracking and managing donor and country funds for the AMC;

• Providing funding to GAVI, in accordance with Supply Agreements, that will be combined with co‐payments from countries and provided to the procurement agency40;

• Assisting GAVI in establishing the IAC and drafting the legal agreements governing the AMC; and

• Providing data on the financial status of the AMC.

2.4 TECHNICAL AGENCIES

Two UN agencies are providing technical/operational assistance for the establishment and implementation of the AMC: WHO and UNICEF.

WORLD HEALTH ORGANIZATION

WHO will provide technical guidance and support related to the AMC and complementary activities at the headquarters, regional and country levels. It will also provide data to monitor and evaluate the AMC.

At the headquarters level, WHO’s key role in the AMC processes is identical to its normal role to review Product Summary Files as filed by industry and to pre‐qualify any pneumococcal vaccine that meets the requirements. Once the vaccine is pre‐qualified by WHO, it is eligible to be purchased by UN agencies. Other WHO roles include: provision of policy guidance regarding the introduction of new vaccines, analysis of health data (e.g. child health mortality data and

40 In actual fact the World Bank will be providing funds to an account based on cash transfer notes from UNICEF and GAVI will provide funds to the same account from where UNICEF can draw the funds. Country co‐financing will follow a different structure.

80

immunization coverage data), reports on progress and achievements in the implementation of the complementary activities to support the introduction of pneumococcal vaccines, and technical assistance/support for research related to new vaccines.

Regional offices play a role in coordinating the introduction of new vaccines and supporting surveillance networks. The Pan American Health Organization (PAHO), which serves as the regional office for the Americas and Caribbean, administers a Revolving Fund for the purchase of vaccines at predictable and affordable prices.

Country offices, in collaboration with UNICEF, assist in the integration of new vaccines into existing health systems, strengthening of health systems including training and surveillance systems, reporting of immunization‐related data for M&E and for annual reporting of global health indicators.

UNICEF

UNICEF will provide assistance at headquarters level (policy and advocacy), regional level and country level (assistance, in collaboration with WHO, in operational aspects of vaccine introduction and delivery). Similar to the support provided by WHO, these activities are outside the scope of the AMC, but are complementary and critical to the success of the AMC.

UNICEF has been identified as the procurement agency to purchase AMC eligible vaccines on behalf of GAVI‐eligible countries. UNICEF will enter into Supply Agreements with manufacturers and with financial support from GAVI, will commit to engage in firm contracting for fixed percentages of the committed supply in each of the first three years during which vaccine from dedicated capacity is made available. Working with GAVI, UNICEF will provide a forecasted demand to the manufacturers to help match supply to demand.

Unless otherwise specified, UNICEF’s existing practices will be applied during AMC implementation. This includes, for example, sourcing from multiple manufacturers to assure an uninterrupted sustainable supply of vaccines; consideration for country product preference; the allocation of available supply among countries in the event of shortage; and other practices.

81

2.5 AMC AD HOC COMMITTEES

In addition to the existing organizational structures supporting the AMC, a number of ad hoc groups have been set up to contribute to specific components of the AMC.

TARGET PRODUCT PROFILE EXPERT GROUP

The TPP Expert Group was set up by WHO based on WHO’s existing responsibilities for pre‐qualification of vaccines. The expert group established a TPP that was shared for review with the WHO SAGE at the November 2007 meeting. SAGE ‘s members approved the TPP subject to amendments and the TPP was publicized as of February 2008. Confirmation of the TPP by the SAGE means that the TPP becomes WHO official policy. WHO, in turn, recommends the TPP to the IAC for approval. The TPP Expert Group has finished its work.

ECONOMIC EXPERT GROUP

The EEG included members with an understanding of global health and development, industrial organization and market design economics, multinational industry/ manufacturing, pharmaceutical/vaccine economics, development economics, legal issues, developing country and health delivery and pneumococcal vaccines.41 It provided technical support needed to design, implement, monitor and evaluate the AMC mechanism. It was responsible for proposing the AMC pricing and design for approval by the DC. It addressed issues related to the subsidized price, relationship between the co‐payment and the tail price, supply obligation in the post‐AMC period and the ability and willingness of low‐income countries to pay and currency issues.

IMPLEMENTATION WORKING GROUP

The IWG included four experts from the EEG and representatives of the World Bank, GAVI and UNICEF. The IWG was set up by the DC to recommend “a specific proposal for the AMC structure and parameters…” including developing

41 Information for this section is taken from “Advance Market Commitment for Pneumococcal Vaccines: Expert Group Terms of Reference”, p. 1.

82

detailed, operational recommendations to allow the donors to finalize the terms and conditions of the AMC.42 The IWG has completed its work.

2.6 PHARMACEUTICAL INDUSTRY

The established vaccine manufacturers (mostly subsidiaries of large pharmaceutical companies) are currently conducting and will continue to conduct pneumococcal vaccine development and to optimize the vaccine formulations for application in low‐income countries, including clinical testing sufficient to ensure licensure and pre‐qualification to allow usage in GAVI‐eligible countries. They will then scale up production of these vaccines as committed to under Supply Agreements and make vaccines available in response to demand from GAVI‐eligible countries. In order to benefit from the AMC funds, emerging country vaccine makers, as well as independent biotech companies, will have to take their pneumococcal vaccine candidates through all clinical trial phases and invest in suitable production capacity.

The roles of the vaccine/biotech firms are to:

• Register to become an AMC‐eligible supplier;

• Provide input on policies and procedures for AMC;

• Complete clinical development and licensure;

• Scale up production capacity for pneumococcal vaccines;

• Submit the vaccine to WHO for pre‐qualification and to the IAC for TPP approval;

• Enter into Supply Agreements with UNICEF;

• Produce supplies of vaccine as specified in Supply Agreements, including during the tail period;43 and

42 IWG Terms of Reference, Appendix A of the “Advanced Market Commitment for Pneumococcal Vaccines: Implementation Working Group Report” July 10, 2008, p. 32.

43 It is assumed, for the long‐term sustainability of this initiative, that the industry will continue to supply vaccines in the period after the AMC. The “after tail period” is the period after the “post‐AMC” period – that is when manufacturers are no longer under obligation to provide vaccines.

83

• Provide information requested for M&E. This may include non‐confidential information on qualitative trends in investments and investment intentions in R&D and production capacity that derived from the AMC initiative. Consultations with industry did not provide definitive answers to what industry will or will not provide in information. Pharmaceutical firms did say that they would not likely provide sensitive information unless assured that information would be collected by a neutral third party and that information would be safeguarded as confidential.

2.7 GAVI‐ELIGIBLE COUNTRIES

There are currently 72 GAVI‐eligible countries that may access AMC funding for the pneumococcal vaccine. They will be responsible for:

• Making an evidence‐based decision on whether to introduce pneumococcal vaccines under the AMC;

• Preparing for the introduction of the vaccines appropriate to their epidemiological setting and within established EPI distribution systems;

• Making the necessary budgetary commitments to cover the co‐financing for such vaccines and the costs of their delivery; and

• Collaborating with multilateral partners to measure the impact of introduction of the pneumococcal vaccines.

3. AMC BENEFICIARIES

The ultimate targeted beneficiaries are children of less than five years of age receiving the pneumococcal vaccine immunization in GAVI‐eligible countries.

There are also potentially indirect beneficiaries, including:

• GAVI‐eligible country governments/health systems and families of vaccine recipients who will realize cost savings through prevention by immunization, as opposed to expenditures for treatment and disabilities;

• Non‐immunized children and vulnerable adults benefiting from herd immunity; and

• Developing country vaccine producers that secure first time access to international markets.

84

4. AMC FUNDING ARRANGEMENTS AND RESOURCES

The design of the AMC draws on a variety of financial and technical resources. The technical resources have been described in Section 2.3.

In February 2007, the donors (Italy, UK, Canada, Russia, Norway and the Bill & Melinda Gates Foundation) formally committed resources for vaccine purchase.44 At the time, the commitments (some of which were made in national currencies) were estimated to be US$ 1.5B in 2007 nominal dollars. Donors may provide financial resources as cash up‐front, or as a commitment for a given amount at one point in time, or a commitment to provide a stream of payments over time.

The first AMC payments will begin when the first eligible vaccine is available (currently estimated to be 2013) and will last for 9 ‐ 10 years.45 The vaccine deliveries and, hence, payments are expected to start at a low level and ramp‐up over time based on projected demand.

The total cost of providing access to pneumococcal vaccines in GAVI‐eligible countries includes, not only the $1.5B provided by the donors, but also a considerable contribution from GAVI over time – US$1.3B has been approved until 2015 ‐ as well as the co‐financing provided by GAVI‐eligible countries implementing the vaccine. Financial resources will be provided to support GAVI country demand creation and the introduction of new vaccines into existing health systems (including the delivery of vaccines). These costs will be incurred by both the international agencies supporting GAVI‐eligible countries and by the countries themselves (although possibly with assistance from GAVI and/or bilateral donors directly).

The administrative costs for the establishment of the AMC are currently being covered through a number of sources. The GAVI Board approved funding for start‐up costs (e.g. legal costs, the AMC Secretariat). In addition, each donor has been covering the costs of its own involvement in the DC. Beyond that, the administrative costs for implementing the AMC will be taken from the income

44 PneumoADIP, 9 February 2007 http://www.preventpneumo.org/pdf/PneumoADIP_AMC_9Feb07.pdf

45 AMC Pilot Proposal, p. 11.

85

from investing the donor contributions. There is no concrete arrangement on the coverage of future admin costs.

5. PNEUMOCOCCAL VACCINE AMC LOGIC MODEL

The logic model for the AMC initiative is provided in Exhibits 12 and 13 below. It describes the key elements of the AMC (activities, outputs and outcomes) in a logical sequence that reflects the causal linkages among them.

The purpose of the logic model is to contribute to a shared understanding of the AMC components and their inter‐linkages. Details of the definitions of the various components of a logic model are provided in Appendix B.

The AMC logic model is divided into two components: one reflecting the establishment phase and the second the implementation phase, which includes product development, activation and transaction processes. This section provides an overview of the activities and outputs, and the expected results (outcomes) of the pneumococcal vaccine AMC for the two phases.

86

Establishment Phase

Activ

ities

Imm

edia

te o

utco

mes

Out

puts

Long

er-te

rm o

utco

mes

1. Establishing the AMC organizational structures

and procedures (including M&E)

TPP document

3. Establishing/refining TPP and finalizing financial terms

Greater certainty for pharmaceutical companies

Credible donor funding

Effective AMC management and donor oversight

Donor Committee, AMC Secretariat, Expert Group,

IAC, IAC Charter and Bylaws and AMC Procedures

Memorandum

Effective and transparent AMC structures and

processes

4. Consulting with key stakeholders (industry,

donors and GAVI-eligible countries)

Stakeholders Agreement, Offer Agreement, Letters of Intent,

AMC Registered Manufacturer Application

Credible forecasts for pneumococcal vaccine

demand

Clear and shared understanding of AMC

structures and processes and reporting agreements

Clear technical requirements for

pneumococcal vaccine

Greater dynamic competition within the

industry

Models and forecastsUpdated forecasts

2. Developing models and forecasts

Entrance into market of new multinational and/or

emerging producers

Inte

rmed

iate

out

com

es

Exhibit 12: AMC for Pneumococcal Vaccine for GAVI-Eligible Countries: Establishment Phase

88

5.1 ACTIVITIES AND OUTPUTS

The activities and their outputs are grouped into nine key groups (the numbers correspond to the numbers on the logic model) that cover the full range of activities that are included in the AMC, as well as supporting activities that are beyond the scope of the AMC.

5.1.1 ESTABLISHMENT PHASE

1. Establishing the AMC Organizational Structures and Procedures

The first activity group includes the establishment of the AMC organizational structures, including the DC, the AMC Secretariat, the IAC and the EEG.

The outputs of these activities are the committees, structures and procedures themselves. They include:

• Establishing the AMC Secretariat;

• Establishing the roles and responsibilities of the AMC donors during the pilot AMC implementation phase;

• Defining terms and conditions of the offer agreement,

• Defining the IAC Charter and Bylaws setting out the roles and responsibility of the IAC, including approval of the TPP, review and approval of AMC Eligibility applications, review and modification of AMC terms and conditions (including price); and

• AMC Procedures Memorandum containing the processes and procedures for the IAC, the AMC Secretariat, vaccine manufacturers and eligible countries. It will outline application procedures for vaccine manufacturers to become AMC registered manufacturers, to have their vaccines become AMC‐eligible through the IAC approval process and the process for making an offer to the Procurement Agency. It will have monitoring and reporting procedures and application procedures for eligible countries to access IAC‐approved vaccines.

89

2. Developing Models and Forecasts

The AMC market analysis included:

• Strategic demand forecasting which was carried out by PneumoADIP;

• Development of the AMC‐FIRM modelling of potential industry response to the AMC, commissioned by the World Bank from Applied Strategies, a life‐sciences strategy consulting firm; as well as modelling undertaken by the EEG with assistance from CRA; and

• Cost of Goods studies conducted by Mercer Consulting.

The outputs were the models and forecasts. Strategic demand forecasts will be updated semi‐annually by GAVI in accordance with the country approval process and timeline (also considered outputs).

3. Establishing/Refining the Target Product Profile

The TPP establishes the expected efficiency (e.g. public health performance standards, maximum number of doses per treatment, compatibility, minimum duration of immunity, non‐interference) of the pneumococcal vaccine to be supported under the AMC. The expert advisory group recommends the TPP to SAGE – an advisory group on immunization policy and strategy to WHO’s Director General. If the TPP is adopted, it becomes WHO’s position on its required performance for the vaccine in GAVI eligible countries and likely leads to member states’ acceptance of TPP provisions. The TPP is then reviewed by the IAC, which makes a final decision on TPP. If, in the rare event, the TPP proves to be unattainable, the IAC can modify the TPP provisions (TPP provisions cannot be increased). The output is the TPP document.

4. Consulting with Key Stakeholders

The AMC has been based on extensive consultations with key stakeholders, including the donors, host institutions, GAVI‐eligible countries, industry and civil society organizations to develop consensus agreements. GAVI is consulting with eligible countries to identify the expected level of demand for the vaccine and the World Bank is working with donors to secure financial commitments and to establish financial management arrangements. The outputs are inputs to the design and administration of the AMC and documents that reflect the commitments of various stakeholders. These include:

90

• Stakeholders Agreement – the umbrella Agreement between the GAVI Fund, the World Bank and the donors. The Agreement describes the AMC roles and the responsibilities of the AMC stakeholders;

• Offer Agreement – a unilateral offer from the GAVI Fund and the World Bank, reflecting the arrangements among stakeholders in the Stakeholders Agreement, of $1.5 billion to industry under specified terms and conditions. The co‐payment from GAVI and countries would be reflected in the Supply Agreement. Certain fixed terms and conditions will be scheduled to this Agreement and will outline design features that are not subject to negotiation by an AMC‐eligible manufacturer – e.g. AMC contribution, tail price cap, supply commitment requirements, and others (see Exhibit 5);

• AMC Registered Manufacturer Application – the mechanism for manufacturers to express an interest to participate in the AMC and to create a contractual nexus between manufacturers and the World Bank and GAVI. Manufacturers will be expected to apply to become “AMC Registered Manufacturers.” During this application phase, a manufacturer would acknowledge the core AMC features and agree to abide by them – in particular, the AMC Terms and Conditions, the pro forma Supply Agreement and the roles and responsibilities of the IAC in the AMC framework. Each manufacturer would also be asked to voluntarily report to UNICEF as the procurement agency and the IAC on a regular (annual) basis an updated estimate of the timeline of progress towards WHO prequalification and intended availability of supply; and

• Letters of Intent from GAVI‐eligible countries.

Exhibit 5: Terms and Conditions of the Offer Agreement (Drawn extensively from the IWG Report)

These terms and conditions are fixed at the outset of the programme and apply across all firms that eventually participate in the AMC.

• AMC Price: The overall per‐dose price for vaccine sold during the time when a company is receiving an AMC subsidy should be $7. The per‐dose AMC subsidy should be $3.50 for products offered at a tail price of up to $3.50. For products offered under a Supply Agreement specifying a tail price below $3.50, the AMC subsidy should be raised accordingly, such that the overall AMC price remains at $7. This results in an average price over the duration of the AMC of up to $4.25.

91

• Supply commitment relationship to aggregate AMC subsidy: The AMC should provide an aggregate contribution (total subsidy) to a participating manufacturer that represents a share of the $1.5 billion directly proportional to the share of a total demand forecast of 200 million doses annually for which the firm has agreed to commit dedicated manufacturing capacity over 10 years (when the manufacturer enters into a Supply Agreement). This aggregate contribution will be distributed according to the doses ordered as described in the AMC Price section above of pneumococcal vaccine meeting the TPP for GAVI‐eligible countries. In extenuating circumstances, GAVI and UNICEF will be requested under the AMC Terms and Conditions to make any other purchases of pneumococcal vaccines for GAVI‐eligible countries in ways that do not provide a “better deal” to manufacturers than is available under the AMC.

• Tail price cap: To be eligible to enter into a Supply Agreement, the manufacturer should commit to providing pneumococcal vaccine, at an annual number of doses corresponding to the supply commitment, at a per‐dose price to GAVI/UNICEF (excluding the AMC subsidy) of no more than the tail price cap (initially set at $3.50). The tail price cap will apply for the duration of the supply commitment. The cap represents a ceiling; it is anticipated manufacturers will come in below this.

• Firm order timing: The AMC should guarantee a portion of the anticipated demand through the mechanism of firm order timing, which is described in the EEG report as a means of mitigating demand risk. For any participating firm entering into a Supply Agreement, UNICEF, with financial support from GAVI, should commit to firm orders for 20 percent of the committed supply on an annual basis in the first 12 months during which vaccine from dedicated capacity is made available; 15 percent of committed supply in the second 12 months during which vaccine from dedicated capacity is made available; and 10 percent of committed supply in the third 12 months during which vaccine from dedicated capacity is made available. The guarantee is for the entire AMC price, including both the AMC subsidy for amounts paid under firm order timing during the AMC period as well as GAVI and country co‐pay, provided that the firm has not received its entire AMC subsidy at the point in time when the purchasing guarantee comes into force.

• Sales out of existing headroom: Manufacturers entering into a Supply Agreement should be able to use existing manufacturing headroom (before dedicated capacity comes on‐line) to fill orders for their product for a maximum of 5 years. For these sales, they should receive the AMC price of $7 (subsidy + tail price) so long as their AMC subsidy amount under the Supply Agreement remains. Once their AMC subsidy has been exhausted, they should receive the tail price. As a condition of sale from existing headroom, once sales begin, the company should be required to continue to supply vaccine at the same or greater level until the full supply capacity comes online. GAVI should provide a biannual forecasted demand to the manufacturer and UNICEF the short‐term supply chain forecast, which UNICEF will issue to each supplier with a goal of matching supply to demand to guard against shortages. For these sales, as for all other AMC sales, the per‐dose payment should be made by UNICEF, consolidating the AMC contribution, and GAVI/country co‐payments.

92

The Supply Agreement specifies a 10‐year agreement to provide vaccine to GAVI countries. The length of the agreement would be revised depending on the extent of the provision of vaccines during the headroom period. The obligation to supply during the tail period (after the funds for the AMC subsidy are exhausted) may, at company request, be reduced by an amount directly corresponding to the amount sold out of existing headroom. This reduction should be taken in a manner that least disrupts overall vaccine supply, preferably taken out of the last years of the Supply Agreement, although the exact schedule should be determined between UNICEF and the manufacturer prior to entering into a given Supply Agreement. To assure supply continuity over time, manufacturers should not be allowed to provide an annual commitment during this period that exceeds the capacity they have committed to provide as part of the AMC unless agreed to by UNICEF and GAVI. Sales out of headroom will be limited by the extent of country demand.

• Entry into Supply Agreements to align supply and demand: The entire $1.5 billion AMC subsidy will be announced at the launch of the Pneumo‐AMC. At a time to be determined by GAVI and UNICEF (which could be at a to‐be‐determined standard time each year), manufacturers will be asked to submit bids for 10 years of committed supply with an estimated start date not more than 5 years into the future. All manufacturers who have (i) become an AMC registered manufacturer; (ii) produce vaccine that has been pre‐qualified by WHO (iii); completed the AMC eligibility application process with the IAC; and (iv) received approval from the IAC of the vaccine in question as meeting the TPP, are entitled to enter into Supply Agreements according to the terms laid out in the Section below provided that there is defined demand for the vaccine.

The basis for any Supply Agreement is the strategic demand forecast, which will be updated by GAVI semi‐annually in accordance with the country approval process and timeline. The portion of the strategic demand forecast that is not already covered by Supply Agreements will be open for bids on an annual basis from manufacturers that meet the requirements above. Suppliers may forward a bid to UNICEF and discussions may be initiated at the point in time where a product summary file has been accepted for review by WHO. As a general rule, Supply Agreements will be entered into only at the point in time when the vaccine is approved by the IAC. Only in extraordinary situations and based on analyses by GAVI/UNICEF which indicate that in the interest of vaccine security, in the case of limited vaccine supply or similar reasons in which there is justification to deviate from that practice, will additional Supply Agreements be established. If, in a single round, for a given start year, the total supply commitments offered by multiple manufacturers were to exceed estimated demand or total funding available, GAVI/UNICEF would have the ability to negotiate with manufacturers and enter into Supply Agreements. These Supply Agreements would be for specific vaccines in quantities to match the overall forecasted demand but at the same time and in accordance to existing practices, to take into consideration country vaccine preferences and the need to have multiple manufacturers to support the sustainable uninterrupted supply of vaccines. In a situation where Supply Agreements with manufacturers having IAC approved products are insufficient to meet the projected demand 5 years ahead, UNICEF may develop policies that, at its discretion, allow for entry into Provisional Supply Agreements with companies with products deemed by the IAC to be

93

highly likely to become pre‐qualified within a timeframe sufficient to meet impending AMC demand needs. Provisional Supply Agreements could have firm, timed, intermediate milestones and no contractual obligations from the AMC until all milestones had been met.

• Adjustment for inflation: Firms entering into a Supply Agreement at a tail price lower than the tail price cap should be permitted to increase the tail price at the rate of the GDP deflator until the tail price reaches the tail price cap ($3.50). In addition, the IWG recommends that, at the request of a company, the tail price cap be reassessed by the IAC at three year intervals or a cumulative 7% rise in the GDP deflator, whichever comes first. The first review could take place 3 years after signing the offer agreement. This reassessment should take into account current and projected rates of inflation as well as other pertinent, available information such as manufacturing costs and efficiencies. It should also recommend increases in the tail price cap in a manner that fairly shares real increases in costs between manufacturers, GAVI and the countries. The IAC should, at its next meeting, develop the plan for moving forward on this issue, including identifying the appropriate experts to develop and implement the specifics of the suitable process and methodology.

5.1.2 IMPLEMENTATION PHASE

5. Researching and Developing Pneumococcal Vaccine and Investing in Production for GAVI‐eligible Countries

This activity includes the activities of firm(s) within the pharmaceutical industry to continue R&D for the development of pneumococcal vaccine for GAVI‐eligible countries (e.g. Stage 3 clinical trials, testing in HIV‐positive children, heat stability and investment in developing production capacity). The output is that vaccines are developed and production facilities established (incremental scale‐up and/or dedicated facilities for pneumococcal vaccines) to serve GAVI‐eligible country markets.

6. Completing the AMC Eligibility Process and Signing Agreements

Once one (or more) AMC Registered firm(s) have developed a vaccine believed to meet the TPP requirements, the product is forwarded for AMC eligibility assessment. It is submitted for pre‐qualification by WHO. If it meets WHO’s quality and safety standards and public health performance standards, upon pre‐qualification it is submitted to the IAC, which determines whether the vaccine meets the TPP and is eligible for AMC funding. AMC‐registered manufacturers, that meet all of the TPP requirements, will have the opportunity to enter into Supply Agreements with the procurement agency (UNICEF) in order to access funding through the AMC. To maximize transparency, equal treatment and

94

certainty for all manufacturers (and to minimize transaction costs and lead‐in time), most of the key features and terms are predetermined (see Exhibit 6], ensuring limited negotiations at the stage of entering into each final Supply Agreement. There will be specific terms for a specific manufacturer that will be made only at the stage of entry into the actual Supply Agreement (e.g. level of supply commitment, tail price and start date for dedicated capacity). These will be the negotiated elements of the Supply Agreement. The AMC will also guarantee a portion of the anticipated demand from the dedicated capacity through the mechanism of purchasing guarantees.

Exhibit 6: Element of a Supply Agreement (Drawn extensively from the IWG Report) Although there will be some flexibility for GAVI/UNICEF to adapt the elements, the expected key elements of a Supply Agreement include the following:

• Firms should be required to make a supply commitment of a minimum of 10 million doses annually, over 10 years;

• The starting date for dedicated manufacturing capacity to go on‐line should be no more than 5 years after entering into a Supply Agreement. There should also be a requirement for specific notification to UNICEF/GAVI and the IAC when such dedicated capacity comes online and manufacturers would provide biannual updates on progress. The duration of the supply commitment should be 10 years from the start date specified by the firm. The AMC would not allow new entrants after 2020. Funds that are not committed to a firm at this point would be returned to AMC donors or transferred to GAVI for use in future procurement of pneumococcal vaccine;

• After the first anniversary of full supply capacity availability under a Supply Agreement, if overall demand fails to meet projected demand included in the Supply Agreement when it was first entered into (at a company’s request to the IAC and in accordance with UNICEF procedures) supply commitments will be prorated to match overall observed demand, taking into account the actual demand development in the previous three years. No company’s commitment should be reduced below its current level of supply in that given year. Companies would be allowed to sell their vaccine from dedicated supply for which there is no AMC demand in other non‐GAVI markets. GAVI and UNICEF expect to purchase pneumococcal vaccines for GAVI‐eligible countries only in accordance with the AMC arrangements (i.e. subject to the tail price cap). In extenuating circumstances these practices could be adapted by GAVI/UNICEF, with the authorization of the IAC;

• From the third anniversary of full supply capacity availability under a Supply Agreement, if overall demand for pneumococcal vaccine has risen to the level included in the Supply Agreement when it was entered into but demand for a particular product is lower than the supply commitment, at company request to the IAC and in accordance with UNICEF procedures, a company’s supply commitment may be modified downward annually taking into account the actual demand curve for its product in the three previous years;

95

• In both of the situations above, the share of any remaining portion of the AMC contribution could be correspondingly reallocated away from the firm, corresponding to the lower supply commitment;

• In extraordinary circumstances, a firm entering into a Supply Agreement may approach the IAC with a request for an increase in the tail price. Any increase would be subject to close scrutiny, and would require firms to reveal information about manufacturing costs (potentially through a negotiated process of intermediation by a neutral third‐party). The IAC should not agree to increase the tail price for a specific manufacturer except in extraordinary circumstances and any increase should be fully justified with respect to achieving the AMC objectives.

The outputs are a pre‐qualified and then IAC approved vaccine and signed Supply Agreements.

7. Stimulating Demand for Pneumococcal Vaccines in GAVI‐eligible Countries

These activities are outside the scope of the AMC, but critical to the logic of the initiative if the public health outcomes are to be achieved. Activities are to be included in GAVI’s strategic work plan for support for programmes in 72 of the poorest countries (GAVI‐eligible countries). These will be funded by existing GAVI resources and implemented by GAVI and/or partners, such as WHO, UNICEF and PneumoADIP.46 The countries are responsible for making evidence‐based decisions on the introduction of pneumococcal vaccines into their national health programmes. These demand complementary activities include pneumococcal disease burden studies, assessment of the public health and economic impact of pneumococcal vaccines, informing and educating country decision makers about pneumococcal disease and the options for its management and ensuring that GAVI‐eligible countries have the guidelines to allow them to submit applications in a timely fashion. GAVI and/or partners will provide financial support for country co‐payments and the tail price (depending on country need). Prior to UNICEF entering into a Supply Agreement with a

46 Major PneumoADIP activities will cease at the end of 2008. A competitive bidding process is underway to retain a new outsourced entity to conduct support activities around new vaccine introduction, including pneumococcal vaccines, and is planned to be in place by the end of 2008. However, it is assumed that GAVI, through some mechanism, will continue to support the introduction of pneumococcal vaccines in GAVI‐eligible countries.

96

specific manufacturer, the GAVI Board will be asked to approve the provision of funding to support the firm order timing commitment, acknowledge the financial commitments made for specific country approvals (done in rolling two‐year periods) for the AMC period and approve a budget envelope of funding needed for the duration of the Supply Agreement.

The outputs of these activities are transparent, credible, effective demand, reflected in GAVI‐eligible country applications for pneumococcal vaccines and approved provision of funding for countries.

8. Preparing for Introduction of Pneumococcal Vaccines in GAVI‐eligible Countries

In addition to supporting demand creation with GAVI‐eligible countries, GAVI and/or partners (WHO, UNICEF, PneumoADIP) may provide financial and technical support to GAVI‐eligible countries for the introduction of the vaccine (e.g. training of health workers, social mobilization, development of cold chain, addressing transport and other logistic issues). This is also outside the scope of the AMC, but critical to achieving its public health outcomes and credible demand forecasts. The output is the GAVI‐eligible countries’ delivery capacity.

9. Undertaking Financial Transactions

Once GAVI‐eligible countries have submitted applications for vaccines that are approved, the financial transactions are triggered. UNICEF will be responsible for procurement under the AMC including issuing supply agreements in accordance with AMC terms and conditions and individual country product preferences and demand. UNICEF will issue purchase orders to manufacturers based on country approvals received from GAVI and annual shipment plans agreed to with countries. Manufacturers will be paid based on actual delivery of goods.

The World Bank will be responsible for collecting donor funds and disbursing payments (via the GAVI procurement account) and ensuring donor payments consistent with disbursement needs. GAVI will contribute its own funds to the donor payments to meet the funding requirements. Country co‐financing will follow the standard GAVI procedure in accordance to which the country procures a specified number of doses to meet its co‐financing requirement. The outputs of these activities are the payments by both the donors, including GAVI, and the countries receiving the vaccines to UNICEF for procurement of vaccines.

97

The World Bank will be responsible for cash flow management related to donor funds and for the management of currency risk, ensuring that cash contributions are made on a timely basis, as well as ensuring that available cash for the AMC is appropriately invested. Due to the size of the AMC fund, cash flow and investment management will be important functions and will be subject to evaluation periodically with respect to cash flow management.

10. Monitoring AMC Implementation

During the process of AMC implementation, the IAC, DC, GAVI and UNICEF will play a role in monitoring the implementation of the initiative and resolving disputes. Some of the responsibilities of the key stakeholders with respect to monitoring the AMC implementation (including M&E) have been identified as the following:

• GAVI has suggested that it will provide the staff and funding to implement the M&E framework;

• UNICEF and GAVI will be responsible for collecting information required for projecting the timeliness for the availability of the vaccines. Manufacturers will, under the Supply Agreements, be asked to report regularly to UNICEF (as the procurement agency) on the timelines of progress towards prequalification and the intended availability of supply; and

• The IAC will have a role to play in judging the need to invoke mitigation provisions in extraordinary circumstances, including suspension or cancellation of an agreement with a manufacturer or to allow for temporary relief in the case of force majeure events.

5.2 AMC OUTCOMES (EXPECTED RESULTS)

The AMC activities and outputs lead to a range of outcomes – immediate, intermediate and long‐term. This section describes first the outcomes related to the AMC objectives. These should be directly attributable to the AMC activities. Secondly, the section identifies other longer‐term outcomes that cannot be directly attributable to the AMC but to which the AMC contributes.

98

AMC Objectives

Accelerated Development of Pneumococcal Vaccines

Many recent policy efforts such as GAVI’s work and the ADIP efforts have likely increased investments by multinational vaccine manufacturers in R&D into pneumococcal vaccines. The AMC aims to be an additional policy, which will contribute towards increasing these investments, particularly in late stage vaccine trials for currently available vaccines. Together with other recent vaccine policy efforts, the AMC may also help to encourage the development of new vaccines to meet the needs (e.g. serotype composition and vaccine presentation) of GAVI‐eligible countries.

Accelerated Availability of Effective Pneumococcal Vaccines

The guarantee of an initial purchase price for a limited quantity of new vaccines, through binding commitments with manufacturers, is expected to provide the incentive for industry to create, scale up or reallocate plant capacity to produce the pneumococcal vaccines. This expected to bring forward the availability of effective pneumococcal vaccines for GAVI‐eligible countries.

Accelerated Pneumococcal Vaccines Uptake

The binding commitments with manufacturers, including predictable pricing for the vaccines and commitments to supply the vaccines at low, long‐term and sustainable prices after AMC finances are depleted, are expected to accelerate vaccine uptake.

Effectiveness of the AMC Mechanism as an Incentive Pilot Tested

The pneumococcal AMC will serve as a pilot project for increasing access of GAVI‐eligible countries to new vaccines. It will allow stakeholders to assess the intended and unintended impact and feasibility of the AMC mechanism and provide guidance in the design of AMCs for other vaccines. It is expected to demonstrate the effectiveness of the mechanism for increasing access to vaccines. However, what will be required to demonstrate its effectiveness will depend on the level of results needed by the donors to reflect effectiveness. It may be that evidence of accelerated availability of pneumococcal vaccines would be sufficient to prove the effectiveness of the mechanism. It may also be that greater proof of the effectiveness would be required, through evidence of increased procurement

99

and distribution of the pneumococcal vaccines in GAVI‐eligible countries (either during or after the AMC period) or increased immunization coverage rates.

Other Expected Outcomes

Lives Saved and Disease Burden Reduced as a Result of Widespread Use of the Vaccine

The following table provides an overview of the forecasted level of vaccine uptake and deaths averted with the AMC in place. It is worth noting that:

• These numbers represent the impact of pneumococcal vaccine adoption in GAVI‐eligible countries. They represent the impact during and after the AMC, where the AMC is a stimulus for the rapid adoption by countries and the impact continues well after the AMC Supply Agreements are executed; and

• There is no clear counterfactual for what uptake would be in the absence of the AMC. Nevertheless, under a range of reasonable counterfactuals the AMC would likely be considered successful on cost‐benefit grounds even if only part of this forecasted level of vaccine uptake were achieved.

Time Period Subjects Vaccinated (Millions)

Doses Supplied (Millions)

Deaths Averted (7/1000)

2010‐2015 129 386 .9 2010‐2020 398 1193 2.78 2010‐2025 744 2232 5.21 2010‐2030 1105 3314 7.7 Source: AMC Secretariat

The supply of effective pneumococcal vaccines would immunise about 1100 million infants over the period 2010‐2030, preventing an estimated 7.7 million child deaths. In addition, there are likely to be herd immunity benefits that will act as a multiplier, expanding benefits to un‐immunised children and older populations.

Sustainable Market for Pneumococcal Vaccines Created Post AMC

The AMC Supply Agreements provide for a post‐AMC period (tail period) during which the supplier is obligated to continue supplying the vaccine at a contracted tail price. Support from GAVI or any donor may be provided to countries wishing to procure vaccines post AMC.

Documents

AMC MONITORING AND EVALUABILITY STUDY FOR THE AMC FOR PNEU… · advance market commitment for pneumococcal vaccines report of the monitoring and evaluability study november 13, 2008