!!

PARTNERSHIP FOR TRANSPARENCY FUND

Philippines: Second Medicine Monitoring Project

Project Completion Assessment

Bruce Murray and Marie Elysee Murray (Intern)

6 December 2014

!

i!

TABLE OF CONTENTS

ABBREVIATIONS AND ACRONYMS ................................................................................ ii EXECUTIVE SUMMARY .................................................................................................. iii PROJECT COMPLETION ASSESSMENT ....................................................................... 1 I. KEY FACTS ................................................................................................................ 1 II. OBJECTIVES AND PROJECT OVERVIEW .............................................................. 1 III. SCORING THE PROJECT ....................................................................................... 2

A. Approach and Project Design ................................................................................ 3 B. Project Implementation ........................................................................................ 12 C. Outcomes, Impact and Sustainability .................................................................. 22 D. Replicability .......................................................................................................... 35

IV. CONCLUSIONS ...................................................................................................... 36 A. Overall Assessment ............................................................................................. 36 B. Main Lessons and Recommendations ................................................................. 38

ANNEX 1: LOCATION OF REGIONS IN THE PHILIPPINES ......................................... 40 ANNEX 2: PHOTOS OF NAMFREL MEDICINE MONITORING PROJECT .................. 41 ANNEX 3: PROCUREMENT PRICES FOR SELECTED DRUGS ................................. 47 ANNEX 4: BUDGET AND EXPENDITURES FOR MMP2 .............................................. 52 ANNEX 5: A CHECK LIST FOR POSSIBLE COLLUSION ............................................. 53 ANNEX 6: PROCUREMENT RATINGS OF THE DEPARTMENT OF HEALTH RELATIVE TO OTHER GOVERNMENT AGENCIES ..................................................... 54

!

ii!

ABBREVIATIONS AND ACRONYMS ABC Approved Budget for the Contract APP Annual Procurement Plan BAC Bids and Awards Committee CACP Coalition Against Corruption Program CAR Cordillera Administrative Region CHDs Centers for Health Development COA Commission on Audit COBAC Central Office Bids and Awards Committee CPAR Country Procurement Assessment Review CSO Civil Society Organization DBM Department of Budget Management DILG Department of Local Government DAP Disbursement Acceleration Program DOH Department of Health DRPI Drug Reference Price Index EMO Election Monitoring Organization FDA Food and Drug Administration GPRA Government Procurement Reform Act, Republic Act 9184 (Procurement

Law) GPPB Government Procurement Policy Board IDC Integrity Development Committee IMC Integrity Management Committee IRR Implementing Rules and Regulations JCI Junior Chamber International LGU Local Government Unit MOA Memorandum of Agreement MMP Medicine Monitoring Project MMP1 First Medicine Monitoring Project MMP2 Second Medicine Monitoring Project NAMFREL National Citizens' Movement for Free Elections NCPAM National Center for Pharmaceutical Access and Management NCR National Capital Region NGO Non - Government Organization PDAF Priority Development Assistance Fund PCA Project Completion Assessment PCR Project Completion Report PTF Partnership for Transparency Fund RA Republic Act RHs regional hospitals RIRR Revised Implementing Rules and Regulations SEC Securities and Exchange Commission SWS Social Weathers Station UNDP United Nations Development Programme WHO World Health Organisation

!

iii!

EXECUTIVE SUMMARY Consistent with the health sector reform’s emphasis on good governance and the provisions of the Procurement Law, Republic Act No. 9184 (An Act Providing for the Modernization, Standardization and Regulation of the Procurement Activities of the Government and for Other Purposes), the Department of Health (DOH) engages with civil society organizations (CSOs) to promote transparency, integrity and accountability in procurement. In 2010 the Partnership for Transparency Fund (PTF) approved a $33,350 grant to the National Citizens' Movement for Free Elections (NAMFREL) to finance the Second Medicine Monitoring Project (the Project – MMP2). The Project is designed to promote transparency and reduce opportunities for corruption in the procurement, delivery and use of medicines and the procurement of hospital infrastructure and equipment. This was to be achieved by deploying trained volunteers to observe all stages of the procurement process and to monitor the delivery and storage/use of the drugs in hospitals. MMP2 built on the design and lessons learned under MMP1. The Project’s main activities were identifying, mobilizing and training local volunteers, monitoring DOH procurement and the delivery and use of medicines, undertaking a drug price survey, reporting results, learning lessons and disseminating the experience gained under the Project. NAMFREL was an accredited observer on the Bids Award Committees (BACs) in 28 selected regional hospitals (RHs) and Centers for Health Development (CHDs). NAMFREL volunteers monitored the P2.443 billion worth of procurement or 54% of the Annual Procurement Plans (APPs). The overall Project performance was rated as Highly Satisfactory based on: • Approach and Project Design -- Highly Satisfactory. Although it was not well documented in the Approach Paper, the Evaluation Team’s research clearly shows that MMP2 targeted on a relevant problem. Corruption is a problem in the health sector and the pharmaceutical supply chain in many countries and the prices of drugs in the Philippines are, for inexplicable reasons, multiple times higher than the prevailing international prices. There is a supportive legal/regulatory environment for the Project in that CSO monitoring Government procurement is a legal requirement in the Procurement Law. DOH’s procurement system is well developed, with one exception. Although CSOs were invited to attend 100% of BAC meetings, volunteers actually observed only 19% of the meetings DOH-wide in 2010. NAMFREL is a Non-Government Organization (NGO) with a good international reputation and a nationwide reach. NAMFREL has built an increasingly strong, collaborative partnership with DOH over the past decade. The Project proposal included a clear statement of objectives and activities that were feasible for the given resource envelop and benchmarks against which actual implementation and outcomes could be assessed. Significant improvements in the MMP2 design over the MMP1 design included: (i) tripling the number of hospitals covered; and (ii) undertaking more analysis of the prices resulting from the bidding process by comparing prices among hospitals and with over-the-counter retail prices. • Project Implementation – Highly Satisfactory: MMP2 was implemented on time and within budget and the financial management was good. The planned activities set out in the Project proposal were delivered and the expected outputs largely materialized. The volunteers reported that the training and orientation was useful and that they learned about their role in the context of the Procurement Law. The new Memorandum of Agreement (MOA) between DOH and NAMFREL was signed as

!

iv!

expected and proved to be useful in securing high-level support for the Project in DOH and in the hospitals. Feedback received by the Evaluation Team indicates that the concept of NAMFREL volunteers observing the procurement, delivery and use of medicines, hospital infrastructure and equipment was actively welcomed by DOH and the hospitals. Some volunteers built very strong partnerships with their hospitals. Monitoring by both NAMFREL and PTF was good and action was taken to resolve challenges as they arose during implementation. • Outcomes, Impact and Sustainability – Highly Satisfactory: MMP2 achieved good outcomes, plausibly had some indirect impacts on reducing opportunities for corruption. Some of the benefits have been sustained even after PTF funding ended, because of some volunteers’ motivation to continue monitoring bidding events. There is evidence that MMP1 and MMP2 contributed to institutionalizing the procurement monitoring process in DOH, that NAMFREL has built a strong partnership with DOH and that the DOH and the hospitals have recognized NAMFREL as a partner for improving procurement transparency, protecting their public reputations and the recent widespread perception that DOH is now a well-run Government agency that takes its anti-corruption strategies seriously. Despite falling short of the overly optimistic output targets set in the Project proposal, under MMP2 the volunteers did a significant amount of monitoring throughout the procurement cycle from pre-bidding and bid opening through to medicine delivery and storage and in-hospital use. The volunteers monitored P2.4 billion in procurement, 54% of the APPs, P303 million or 39% of the Approved Budget for Contracts (ABCs) for medicine delivery and P198 million or 20% of the ABCs for storage and in-hospital use. MPP2 was more efficient and effective than MPP1. Under MMP2, volunteers monitored three times as many hospitals and six times as much procurement compared to what was achieved under MMP1. NAMFREL’s price survey found that that there are wide, unexplained variations in the prices paid by hospitals for the same medicines. There is evidence that some of the Project benefits are sustainable, although steps must be taken to further strengthen sustainability. Replicability -- Exceptional. The MMP2 has many elements that can, and should, be replicated in future projects – the major goals, elements and stakeholders are in place and the Project design can be replicated not just in the health sector in the Philippines but also in other countries. The MMP2 model has proven to be appropriate and practical and generates good outputs. NAMFREL established a good relationship with DOH, a core group of volunteers has been trained, much of the training material can be used again with suitable updating, the volunteers have gained practical experience monitoring the full procurement cycle for medicines and the procurement of hospital infrastructure, equipment and supplies and undertaking drug price surveys. The lessons learned and recommendations for PTF and NAMFREL are: 1. Lesson and Recommendation for PTF • Consistent engagement with partners in two or more projects over a decade or so promotes the efficient and effective delivery of results and sustainability of benefits: MMP2 was more effective and efficient than MMP1 in monitoring procurement at all stages of the procurement cycle. That reflected the experience gained by NAMFREL, DOH, the hospitals and the volunteers over a period of years. Engagement with the same partners over the course of two or more projects helps to build strong, constructive relationships between a CSO and its government

!

v!

counterpart. For future projects, other things being equal, PTF should give priority to funding follow on projects for NGOs that have demonstrated an ability to achieve results.

2. Lessons and Recommendations for NAMFREL • Improve some details in the procurement monitoring: In addition to providing sufficient flexibility to deal with scheduling conflicts and short notices of meetings by mobilizing teams of four or more volunteers for each hospital, NAMFREL should engage in dialogue with DOH to ensure that: (i) volunteers are invited to attend the bid evaluation meetings undertaken by the Technical Working Groups; and (ii) all documents needed for post-delivery monitoring of medicines and supplies and their inventory and use in hospitals are complete and readily available to the volunteers. These issues should be covered in an updated MOA. • Promote sustainability by addressing the need to pay the travel allowances of volunteers after donor-funded projects end: Monitoring by the volunteers declined after the PTF funding ended because money was not available to pay for their transportation allowances, thus undermining the sustainability of Project benefits. NAMFREL should engage with DOH to see if part of the proceeds of the sale of procurement documents could be used to finance such costs. If so, this issue should be covered in an updated MOA. • Provide more coverage of collusion and bid rigging in the training: Collusion is difficult to detect, although the wide unexplained variations in the prices paid by hospitals for the same drugs suggest that it is probably occurring. NAMFREL should provide more coverage of collusion and bid rigging in its training courses. • Explore whether advances in mobile technology could usefully deployed for future projects: Advances in mobile technology and related platforms may make it possible for NAMFREL to more efficiently collect and analyze data from the volunteers and for the volunteers to submit their reports. Such systems might also provide a mechanism for stronger follow-up with volunteers to ensure their knowledge gained during the training is being used and that problems can be identified and resolved. This type of regular two-way communications, especially with volunteers in the regions further from Manila, would also strengthen NAMFREL’s relationship with their volunteers and encourage the volunteers’ loyalty and participate in future projects. • Discuss with DOH institutionalizing a DOH-wide monitoring system for the number of BACs that are actually attended by volunteers: Discuss with DOH the possibility of developing a system so that senior managers are aware of the number of procurement meetings actually monitored by CSO observers. • Assess in a future project whether the Drug Reference Price Index (DRPI) estimated by DOH is an appropriate benchmark: The National Center of Pharmaceutical Access and Management (NCPAM) has done excellent work to establish a comprehensive database to monitor the prices paid by all hospitals for about 600 medicines and to set DRPIs to give guidance to BACs. More analysis should be undertaken to ensure that the levels of the DRPIs are appropriate for all drugs relative to international prices and are well below the retail price for the drugs.

!!

PROJECT COMPLETION ASSESSMENT

I. KEY FACTS Project Name: Second Medicine Monitoring Project CSO Name: National Citizen’s Movement for Free Elections Grant Amount: USD $33,350 Date of Submission of Project Concept: June 2010 Date of Submission of the Final Proposal to PTF: 11 August 2010 Date of PTF Approval of the Project: 11 August 2010 Period of Implementation: September 2010 to October 2011 Date of Submission of NAMFREL’s Project Completion Report to PTF: July 2012 NAMFREL Project Coordinator: Edgar E. Camenting Project Completion Assessment Authors: Bruce Murray and Marie Elysee Murray Date of Project Completion Assessment: 6 December 2014

II. OBJECTIVES AND PROJECT OVERVIEW 1. Corruption is a serious problem in the Philippines, is wide spread and affects all sectors. The Philippines country ranked 102nd out of 146 countries on Transparency International’s Corruption Perception Index was in 2004 and 134rd out of 178 counties in 2010. 1 Citizens generally believe that some progress has been made in addressing corruption since 2010 and the Philippines’ country ranking improved to 94th out of 177 countries by 2013. Despite this improvement, there remains a widely held view that corruption is a persistent, deep-seated problem in the Philippines and results in Government goods and services costing more than they should, bribes sometimes being paid to get access to Government services and Government revenues being lower than they should be because taxes are not paid. There is a large body of evidence that demonstrates that the poor and vulnerable are most adversely affected by corruption. 2. Consistent with the health sector reform’s emphasis on good governance and the provisions of the Procurement Law (Republic Act No. 9184, An Act Providing for the Modernization, Standardization and Regulation of the Procurement Activities of the Government and for Other Purposes), the Department of Health (DOH) engages with Civil Society Organizations (CSOs) to promote transparency, integrity and accountability through procurement monitoring. For over a decade, beginning in 2004, the National Citizens' Movement for Free Elections (NAMFREL) has partnered with DOH to help improve transparency and reduce opportunities for corruption by monitoring procurement. Through the Coalition Against Corruption Program (CACP), DOH officially designated NAMFREL as an observer on the Bid Award Committees (BACs) for all of its regional hospitals (RHs) and Centers for Health Development (CHDs). 3. The Partnership for Transparency Fund (PTF) has supported NAMFREL work in the health sector since 2008, providing two grants totaling $56,646 to finance the first and second Medicine Monitoring Projects (MMP1 and MMP2 respectively) (see Figure 1). The 11 August 2010 MMP2 grant agreement committed PTF to providing $33,350,

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!1 The higher the country’s ranking, the worse is the perception of corruption in a country. http://www.transparency.org/research/cpi

!

2!

approximately half of MMP2’s total cost of $67,350.2 NAMFREL provided the remaining half of the budget, mostly in the form of counterpart support, valued at the equivalent of P1,528,792. To ensure smooth implementation, NAMFREL and DOH signed an updated formal Commitment of Support and Cooperation Memorandum of Agreement (MOA) on 9 August 2010.

Figure 1: NAMFREL Projects Financed by PTF Project Government

Agency Year of

Approval Amount of PTF

grant ($) 1. Medicine Monitoring Project, Phase 1 DOH 2008 23,296 2. Medicine Monitoring Project, Phase 2 DOH 2010 33,350 Total 56,646 Source: NAMFREL and PTF Grant Agreements

4. NAMFREL’s detailed Project Completion Report (PCR) was submitted to PTF in July 2012. The Evaluation Team that prepared this Project Completion Assessment (PCA) undertook its fieldwork in February and March of 2014. Discussions were held in Manila with NAMFREL staff, two DOH assistant secretaries and their staff and officials from DOH’s National Center for Pharmaceutical Access and Management (NCPAM). The Evaluation Team also interviewed NAMFREL volunteers and hospital officials and members of the BAC committees in the National Capital Region (NCR), Cabanatuan in Nueva Ecija in Region 3 and Baguio in the Cordillera Autonomous Region (CAR) (see maps in Annex 1). Time constraints did not permit the Evaluation Team to visit other hospitals covered by MMP2. Pictures of some of the people interviewed by the Evaluation Team are shown in Annex 2. Background documents provided by PTF and NAMFREL were reviewed and questions were formulated to structure the interviews. The fieldwork went smoothly and the cooperation of NAMFREL in arranging the meetings and for the DOH and NCPAM officers, hospital officials and the volunteers for sharing their time is gratefully acknowledged. Overall, the Evaluation Team validated the factual material included in NAMFREL’s PCR, which was comprehensive and provided a good base on which the evaluation could build. A wrap up meeting was held with NAMFREL on 7 March 2014, at which the Evaluation Team shared its preliminary findings and assessment. The preliminary findings were also shared with PTF. 5. The draft PCA was forwarded to PTF Philippines on 26 September 2014 and, after considering PTF’s comments, was forwarded to NAMFREL on 7 October 2014 for review and comment and to correct factual errors. NAMFREL provided its comments on 5 December 2014 expressing broad agreement with this report. In considering PTF Philippines’ and NAMFREL’s comments all factual corrections were made. However, consistent with the fact that this is an independent evaluation, if there were differences of opinion or interpretation of agreed facts between the Evaluator Team and PTF, the evaluator’s opinions were reflected in the report. III. SCORING THE PROJECT

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!2 Including the estimated value of services provided in kind.

!

3!

6. The scoring system used to evaluate the Project follows PTF’s Guidelines.3 Each of the following dimensions of evaluation were scored on a 5-point scale4: (i) Approach and Project Design; (ii) Project Implementation; (iii) Outcomes, Impact and Sustainability; and (iv) Replicability. The scores for the four dimensions were added together using the weights5 specified by PTF to arrive at the overall Project assessment. The overall score of this Project is consistent with a Highly Satisfactory performance rating, with Highly Satisfactory ratings for Approach and Project Design; Project Implementation; and Outcomes, Impact and Sustainability and an Exceptional rating for Replicability. The details supporting these ratings are given in the following sections.

A. Approach and Project Design 7. MMP2’s objectives were to help improve the delivery of health services and to counter irregular practices by:

(i) ensuring transparency of RH and CHD purchases by monitoring the full cycle of the procurement process (i.e., pre-bidding; bid opening; bid evaluation; bid award); (ii) preventing fictitious and incomplete deliveries by monitoring and observing the actual delivery of awarded contracts and inventories in hospital pharmacies and medicine store rooms; (iii) ensuring proper and timely distribution of essential medicines to the intended hospital beneficiaries by monitoring its distribution from hospital pharmacies to the various hospital departments; and, (iv) assessing the reasonableness of the prices paid for pharmaceuticals by comparing prices for the same medicines across hospitals and benchmarking the prices paid against the retail prices in local pharmacies.

8. The MMP2 design drew on the successful experience gained under MMP1 in monitoring the procurement and delivery of medicines, infrastructure and equipment. PTF’s evaluation concluded that MMP1 was generally successful and that its main accomplishments were demonstrating: (i) the feasibility of using local NAMFREL chapters to engage local communities and mobilize volunteers to monitor procurement; and (ii) that the volunteers’ work can generate significant findings related to the way hospitals manage their budgets, deficiencies in procurement practices, issues related to the pricing practices of pharmaceutical companies and shortcomings in stock keeping and warehousing practices. PTF’s evaluation also found that MMP1 faced some challenges: (i) the reluctance of some of the hospitals to accept observers and, more generally, the rather passive attitude of DOH towards the project; (ii) motivating the volunteer observers on a sustained basis and to find the financial means to compensate them for their transport and other costs; and (iii) less than expected monitoring because of scheduling conflicts which prevented some volunteers from attending BAC meetings and short notice of the time of meetings)6.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!3 PTF Impact Evaluation Scoring Framework Revised Concept Note. January 2012 4 1 = Unsatisfactory Performance; 2 = Partially Satisfactory Performance; 3 = Satisfactory Performance; 4 = Highly Satisfactory Performance; 5 = Exceptional Performance. 5 Approach and Project Design (15%); Project Implementation (20%); Outcomes, Impact and Sustainability (45%); and Replicability (20%). 6 PTF. Philippines. Medicine Monitoring Project. Project Completion Assessment. 2010

!

4!

9. MMP2 built on the “full cycle” monitoring of pharmaceutical, infrastructure and equipment procurement undertaken under MMP1. Full cycle monitoring involves the areas of budgeting and pre-bid stages of procurement, delivery, and inventory of drugs and medicine. Similar to MMP1, MMP2 was also designed to help improve the warehousing and storage room management systems in RHs and CHDs. As was recommended in PTF’s MMP1 evaluation, MMP2 usefully extended the scope of MMP1 to include assessing the reasonableness of the prices that hospitals paid for drugs by benchmarking the prices resulting from the bidding process against the retail price for the drugs in local drug stores. 10. The MMP2 approach was based on strong partnership with DOH to reduce opportunities for corruption by increasing transparency through monitoring by volunteers. This collaborative approach is consistent with the approach recommended in a UNDP report7 on tackling corruption. That report concluded that while CSOs have a role to play in combatting corruption in most situations CSOs should “avoid becoming investigative bodies, whistle-blowers, or forces of anti-corruption vigilantes. Political leaders are unlikely to welcome this.” Rather, “CSOs can contribute best by concentrating on providing information, expertise, and sustaining a sense of strength in numbers.” 11. The MMP2 Project Proposal set out clear steps that were to be taken to fulfill the Project objectives. According to the NAMFREL’s Project Proposal, the following key activities were to be undertaken:

(i) Consultation with hospitals and regional health offices to elicit their full support for effective implementation of the project. (ii) Recruitment and training of NAMFREL volunteers on the Government Procurement Reform Act (GPRA) RA 9184 and its implementing rules and regulations (IRRs) to enhance competency in observing the bidding. (iii) Quarterly meetings with the volunteers to discuss reports, problems, and concerns encountered. (iv) Engage actively in monitoring public bidding in RHs and CHDs for both hospital and regional health offices to ensure that the process is in accordance with the provisions of the GPRA RA 9184. (v) Perform actual and post monitoring on the delivery and inventory of medicines and ensure that these were delivered and allocated properly to hospital beneficiaries; (vi) Develop a database to aid comparative bid pricing specifically for medicines as primary referenced by the DOH RHs and CHDs in determining an acceptable bid price in public bidding. (vii) Improve the existing monitoring tools to capture relevant information observed in generating reports and manuals. (viii) Hold project evaluation meetings to review the project objectives and assess whether it was achieved and to identify areas where further improvement is needed. (ix) Present the Project’s accomplishments, observations and findings on system flaws and best practices of each RHs and CHDs.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!7 UNDP. Asia Pacific Human Development Report. Tackling Corruption, Transforming Lives. Accelerating Human Development in Asia and the Pacific. 2008

!

5!

12. MMP2 was implemented in 21 RHs and 7 CHDs throughout the country in the following regions: the NCR, Northern Luzon (Regions 1, 2, 3 and the CAR), Southern Luzon (Regions 4 and 5), Visayas (Regions 6, 7 and 8) and Mindanao (Regions 9, 10, 11 and 13) (see maps in Annex 1). The hospitals covered under MMP2 were selected based on active participation of the local NAMFREL chapters and its volunteers and the 2008 DOH Audit report of the Commission on Audit (COA), which identified procurement discrepancies. 13. DOH has 71 RHs and 16 CHDs. Monitoring procurement in 21 out of 71 RHs (30%) and 7 out of 16 CHDs (44%) represented a significant coverage. MMP2’s coverage exceeded that of MMP1 -- 8 RHs and 3 CHDs. The Project scope did not include provincial hospitals or hospitals operated by Local Government Units (LGUs). Given the resources that PTF allocated to support MMP2, a narrow scope focused on selected DOH RHs and CHDs was a pragmatic decision and consistent with the financing available. 14. The PTF Guidelines require that the corruption problem that a project is designed to address be defined as precisely as possible in the Project proposal. There was no description of the corruption problem in the MMP2 Project proposal, either as a general problem or with numbers to show the broad dimensions of the problem. That, perhaps, is not surprising given that: (i) PTF did not comment on this issue when reviewing the draft Project proposal; (ii) the partnership between NAMFREL, DOH and PTF that was built up during MMP1; (iii) the widely held perception that corruption was pervasive in all sectors in the Philippines, including the health sector, under the previous administration; and (iv) the belief that citizen monitoring can improve the transparency of the procurement process and reduce opportunities for corruption. PTF has funded many successful projects in the Philippines that are built on the latter premise. 15. IMF analysis8 shows that in many countries corruption has a significant, negative effect on health indicators such as infant and child mortality and lowers the immunization rate for children and discourages the use of public clinics. Transparency International estimated the scale of global corruption in the pharmaceutical sector as losing 10% to 25% of public procurement spending to corruption, and in some countries up to two thirds of scarce medicine supplies in hospitals are lost through corruption and fraud9. Research undertaken by the Evaluation Team provides clear evidence of the prevalence of corruption in the health sector, including corruption related to pharmaceuticals, on a worldwide basis and evidence that this is a problem in the Philippines:

• A 2006 Transparency International report documented the problem of corruption in the health sector worldwide 10 : This comprehensive report described the scale of corruption in the health sector, corruption in hospitals, informal payments for health care, corruption in the pharmaceutical sector and corruption related to HIV/AIDS. The report examined corruption in the health sector on a worldwide basis and was supported by 45 country studies, one of

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!8 UNDP. Fighting Corruption in the Health Sector. Methods, Tools and Good Practices. Jillian Clare Kohler, PhD. October 2010 9 WHO. A framework for good governance in the pharmaceutical sector. Working Draft. Dr. Eloy Anello. October 2008 10 Transparency International. Global Corruption Report 2006

!

6!

which was in the Philippines 11 . The report identified four major forms of corruption: (i) embezzlement and theft from the health budget and/or user-fee revenue; (ii) corruption in procurement; (iii) corruption in payment systems; and (iv) corruption in the pharmaceutical supply chain. Among other things the report concluded that medicines and medical supplies might be stolen for personal use, use in private practice or resale. According to Transparency International corruption (e.g., collusion, bribes and kickbacks) in the pharmaceutical supply chain and procurement can result in overpayment for medicines, poor quality and products being diverted or stolen at various points in the supply and distribution chain. The report concluded that anti-corruption measures must be tailored to fit the conditions prevailing in a country’s health system. Report recommendations that are particularly supportive of MMP2’s design include: ! Undertaking preventative measures “including procurement guidelines;

codes of conduct for operators in the health sector, both institutional and individual; and transparency and monitoring procedures”.

! Publishing up-to-date information on the Internet on health budgets and performance at the national, local and health delivery center levels.

! Undertaking independent audits of government departments, hospitals, health insurance entities and other agencies handling health service funds.

! Ensuring that “information about tender processes, including offers to tender, terms and conditions, the evaluation process and final decisions, is publicly available on the Internet.”

! Pharmaceutical, biotech and medical equipment companies should ‘adopt the Business Principles for Countering Bribery, through which a company commits to refraining from bribery in its operations and implementing a comprehensive anti-corruption programme”.12

! Introducing “avenues for public oversight, which improve accountability and transparency. These should oversee procurement and drug selection at facility level and health delivery at community and local health board level.”

! Making information on public policies, practices and expenditures “open to public and legislative scrutiny, while all stages of budget formulation, execution and reporting should be fully accessible to civil society.”

! Making an Integrity Pact a binding agreement so that both bidders and contracting agencies not to offer or accept bribes in public contracting, applicable to major procurement in the health sector.13

! Debarring companies that engage in corrupt practices from participating in tender processes for a specified period of time.

16. The Project approach and design is broadly consistent with the guidance provided by the World Health Organisation (WHO) in its assessment instrument for Measuring Transparency in the Public Pharmaceutical Sector14 and Framework for Good

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!11 Local-level corruption hits health service delivery in the Philippines. Omar Azfar and Tugrul Gurgur. Pages 37/39 12 For more on the Business Principles for Countering Bribery and its supporting guidance document and suite for implementation and monitoring tools, see www.transparency.org/ building_coalitions/ private_sector/business_principles.html 13 For Transparency International’s Integrity Pact, see www.transparency.org/integrity_pact/index.html 14 WHO. Measuring Transparency in the Public Pharmaceutical Sector. Assessment Instrument. 2009

!

7!

Governance in the Pharmaceutical Sector.15 The WHO framework identifies a large number of stages where corruption can occur in the pharmaceutical industry: (i) research and development of medicines; (ii) conducting clinical trials; (iii) filing patents; (iii) manufacturing; (iv) registration; (v) price fixing; (vi) licensing of professionals and establishments; (vii) selection of essential medicines; (viii) procurement; (ix) distribution; (x) inspection of establishments; (xi) prescriptions; (xii) dispensing; (xiii) pharma-covigilance; and (xiv) medicine promotion. The WHO framework specifies criteria to assess the likelihood of corruption in each of these areas. The Project focuses on three of these areas – procurement, distribution and dispensing. Given the funds available and the skills of the volunteers, it was appropriate for the Project to focus on a subset rather than trying to cover all of the areas where corruption can occur in the pharmaceutical industry. Explicit provision is included in the WHO guidelines for CSOs to play various roles such as providing institutional moral leadership, raising their voices on anticorruption matters, as whistle-blowers when necessary and as independent monitors and assessors of the enforcement process of anti-corruption initiatives. The WHO framework stresses that the capacity of governments to listen and respond to CSOs must be built up to facilitate collaborative anti-corruption efforts.

• The Project approach and design is broadly consistent with the recommendations in a 2010 UNDP16 report on fighting corruption in the health sector: The UNDP study examined the role of the providers, the government as a regulator through the procurement of medical supplies, distribution and storage of drugs, and the payer. Problems identified included absenteeism, theft of medical supplies, informal payments, fraud, weak regulatory procedures, subjective procurement procedures, diversion of supplies for private gain and embezzlement of health care funds. Typical problems associated with the procurement of drugs included manipulating specifications to favor one supplier; payoffs to procurement officials; overpayment for medicines; procurement of unnecessary products; collusion and manipulation of prices; incomplete deliveries; inadequate audit and control systems; and circumventing the WHO Certification Scheme on the Quality of Pharmaceutical Products Moving in International Commerce. Examples of anti-corruption initiatives suggested included the posting of supply prices to increase transparency and to help prevent collusion; price outlier analysis; comparison of in-country prices to the global median for identical products17; public posting of supplies received by hospitals; strengthening the supply chain management system; regular external and internal audits; community oversight and monitoring; and citizen scorecards.

• Drug prices in the Philippines are excessive when compared to international norms: In 2009 Health Action International18 published a study on the prices of publicly procured medicine in the Philippines. Using the standardized WHO/HAI (2008) methodology, the study examined 50 essential

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!15 WHO. A framework for good governance in the pharmaceutical sector. Working Draft. Dr. Eloy Anello. October 2008 16 UNDP. Fighting Corruption in the Health Sector. Methods, Tools and Good Practices. Jillian Clare Kohler, PhD. October 2010 17 Researchers have benchmarked 90 countries and concluded that countries should aim to be in the 25th to 50th percentile, i.e., below the median prices. 18 The results of the study were presented to the Third International Conference for Improving the Use of Medicines in Antalya, Turkey in November 2011. See Health Action International. Medicines Price Components in the Philippines. Douglas Ball and Klara Tisocki. 2009. http://www.haiweb.org/medicineprices/surveys/200807PHC/sdocs/PriceComponentsReportPhilippines.pdf

!

8!

medicines procured in five DOH hospitals, six provincial hospitals and five municipal hospitals. Although some generics were procured at close to the international reference price, overall the report concluded that the price of publicly procured drugs was excessively high in the Philippines and that there is extreme variability in the prices paid among hospitals for the same drugs that is not related to the volume of medicine procured. The study found that, on average, the prices of originator brands and generic equivalents were 16 and 3 times higher, respectively, than the prices available on the international market. The prices paid by public procurement of medicines in the 16 Filipino hospitals studied were typically between 13 and 40 times the international reference price. The study also found that prices were generally lower for DOH hospitals than for provincial hospitals and that the highest prices were paid by municipal hospitals. The Health Action International study concluded that there was an urgent need to improve the efficiency of the public procurement process for medicines as the high prices paid were indicators of inefficiency, incompetence and corruption. The report suggested improving transparency and good governance in the procurement process by stricter implementation and better monitoring.

• Testing of Generics: The Health Action International study also concluded that there was a need to address issues related to the quality and inter-changeability of generic medicines and to ensure the quality assurance of generics. The study concluded that the Philippine Food and Drug Administration (FDA) should be responsible for this work (e.g., inspections; establishing bioequivalence; post marketing surveillance). Presently BACs have the power to require that controlled random experiments be done in the Philippines to demonstrate that drugs meet the bid specifications – decisions on such matters vary from hospital to hospital. While it must be clearly established that drugs are safe and suitable for the purposes intended, the Evaluation Team questions whether such decisions should be made at the local level rather than at a centralized level in the FDA. One supplier of generics reported to the Evaluation Team that their company was disqualified because the drug trials had taken place in India rather than the Philippines. That company felt that decision was part of collusive practices between the municipal hospital BAC and the traditional local suppliers to disqualify a lower priced bidder. It is beyond the competence of the Evaluation Team to provide a definitive recommendation in this area. DOH, through the FDA, should consider whether there should be a centralized, national certification process for drugs or whether decision making in this area should continue to be done at the local level by hospitals.

17. A supportive policy, legal and regulatory environment contributed to the Project’s relevance and increased the likelihood of its success:

• The commitment of the Aquino Administration to good governance and to root out corruption: President Aquino campaigned on a platform of good governance, high ethical standards in government and strong anti-corruption measures.19 A Roadmap for Implementing Good Governance Measures, which was developed with input from CSO networks, business associations and donor partners, has three objectives: (i) to curb corruption; (ii) to improve the delivery of

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!19 The Philippines was one of the first countries to sign the September 2011 Open Government Declaration that promotes government transparency, access to information and involvement of CSOs to reduce corruption.

!

9!

public services; and (iii) to enhance the business and economic environment. A Cabinet Cluster on Good Governance and Anti-Corruption was established with a focus on institutionalizing “open, transparent, accountable and participatory governance.” The Project provided an opportunity to capitalize on this favorable climate for reform, and build on the gains and lessons learned from MMP1. Although CSOs have been involved in monitoring government procurement in the Philippines for over a decade, the Evaluation Team was advised by both DOH and NAMFREL that the demand for CSO observers had increased sharply after the Aquino administration took office in 2010. Prior to the work begun by NAMFREL in 2004 there was no citizen monitoring of procurement in DOH. Under MMP1 DOH’s cooperation could be classified as ‘passive tolerance’. Under MMP2 the Evaluation Team classified DOH’s cooperation with NAMFREL as ‘active engagement’. All DOH and hospital officials interviewed by the Evaluation Team warmly welcomed the work of the volunteers and viewed it as something that improved the transparency and credibility of the procurement process and helped to protect the reputations of DOH and the hospitals.

• Supportive Legal Environment: Section 13 of Article V of the Procurement Law,20 states that “To enhance the transparency of the process, the BAC shall, in all stages of the procurement process, invite, in addition to the representative of the Commission on Audit, at least two (2) observers to sit in its proceedings, one (1) from a duly recognized private group in a sector or discipline relevant to the procurement at hand, and the other from a non-government organization: Provided, however, that they do not have any direct or indirect interest in the contract to be bid out.” The related Revised Implementing Rules and Regulations (RIRR) state that: (i) observers, duly authorized by the BAC, can monitor the procurement proceedings on-line; (ii) observers cannot vote during BAC meetings; (iii) observers will come from organizations registered with the Securities and Exchange Commission (SEC) and will meet various criteria21; (iv) observers will be informed at least two days before the stages of procurement to which observers shall be invited22: pre-bid conference; opening of bids; post qualification; and contract award; and special meetings of the BAC; and (v) defines the role of the observers, including providing for their access to key documents and reporting on whether or not the BAC complied with the applicable provisions.23 Periodically the Government and its three largest donors prepare Philippine Country Procurement Assessment Reviews (CPARs) that assess the procurement system and its compliance with the provisions of RA 918424.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!20 Republic Act (RA) 9184 of 2003: An Act Providing for the Modernization, Standardization and Regulation of the Procurement Activities of the Government and for Other Purposes. 21 (i) knowledge, experience or expertise in procurement or in the subject matter of the contract to be bid; (ii) absence of direct or indirect interest in the contract to be bid out; and, (iii) any other criteria that may be determined by the BAC. 22 While the observers must be invited in writing at least three calendar days before the BAC meeting, their absence does not nullify the proceedings. 23 Subject to signing confidentiality agreements, the observers are to have access to the minutes of BAC meetings, abstracts of bids, post qualification summary reports, APP and related PPMP and opened proposals. Their role is to observe and report on the compliance with the RIRR and they may submit a report to the BAC Chair, copied to the Government Procurement Policy Board (GPPB) and the Office of the Ombudsman. If no report is submitted, it is understood that the BAC followed the proper procedures. 24!Asian Development Bank, Government of the Philippines, Japan International Cooperation Agency, World Bank Philippines Country Procurement Assessment Report (2012).!

!

10!

• Supportive procurement system in DOH: DOH has a detailed four-volume procurement manual25 that draws on the 2006 Generic Procurement Manuals published by the Government Procurement Policy Board (GPPB). In addition to providing detailed policies and procedures, DOH’s procurement guidelines make it clear that NGO accredited observers must be invited to procurement meetings. The most recent Philippine CPAR assessed 17 government agencies, including DOH, using 17 indicators organized under four pillars (i) compliance with legislative and regulatory framework; (ii) agency institutional framework and management capacity; (iii) procurement operations and management capacity; and (iv) integrity of the agency procurement system26, 27. The CPAR found that DOH’s system was generally better than the systems of most other Government agencies in terms of compliance with the legislative and regulatory framework, the agency’s institutional framework and management practices dimensions of procurement operations and market practices (e.g., capacity of agency personnel; management of procurement and contract management records; and contract management procedures) and the integrity and transparency of the agency procurement system. Two of the indicators used to assess integrity and transparency that are directly related to MMP2 are: (i) whether CSOs and professional organizations are invited to public bidding activities; and (ii) the percentage of public bidding activities actually attended by CSOs and professional organizations. Like most other agencies, DOH invited CSOs and professional organizations to observe its bidding activities as required under the Procurement Law28. However, only 19% of DOH’s procurement activities were actually attended by independent observers. This was lower than the 35% average for all agencies. Given the provisions of RA 9184 and DOH’s procurement manual, the fact that four out of five DOH procurement meetings were not actually attended by independent observers is a matter of concern that needs to be addressed. The problem of CSOs not actually attending procurement meetings goes far beyond the Project. It is pervasive in DOH and is a problem throughout the Government. The Evaluation Team is not sure why this figure is so low since both hospital officials and NAMFREL volunteers both reported independently that procurement meetings were regularly attended during the course of MMP2. It may be that much of the data for the Country Procurement Review was collected in 2010, the year between MMP1 and MMP2. NAMFREL is the only NGO that attends DOH procurement meetings. NAMFREL’s core business is monitoring elections and during portions of election years, like 2010, NAMFREL is fully occupied with planning and monitoring elections.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!25 DOH. DOH Customized Procurement Manuals (Volumes 1 to 4). 2010. The four volumes cover: (i) Guidelines on the Establishment of Procurement Systems and Organizations; (ii) Manual of Procedures for the Procurement of Goods and Services; (iii) Manual of Procedures for the Procurement of Infrastructure Projects; and (iv) Manual of Procedures for the Procurement of Consulting Services. 26 Asian Development Bank, Government of the Philippines, Japan International Cooperation Agency, World Bank Philippines Country Procurement Assessment Report (2012). See Annex 4 in that report. 27 The Agency Procurement Compliance and Performance Indicators are designed to strengthen the Government’s capacity to monitor the implementation of RA9184 and are patterned after the OECD’s Development Assistance Committee’s Methodology for the Assessment of Procurement System Base Line indicators and Compliance and Performance Indicators tools. 28 DOH scored 100% on this criteria. Many other agencies also scored 100% and the average score for all agencies was 94%.

!

11!

18. Strength of the Project Proponent: NAMFREL29 is a well-respected Filipino NGO that was established in 1983 as a non-partisan election watchdog. It developed a reputation as a credible, unbiased election monitor in the 1980s. More than 140 benefactors and 125 organizations support NAMFREL. A National Council governs NAMFREL and an Executive Committee implements the policies and plans of the organization and supervises the activities of the committees and staff under it. NAMFREL has a nation-wide reach with 80 provincial and 17 city/municipal chapters. At the local level, a chairperson represents NAMFREL in 1,609 cities and municipalities throughout the country. Over 250,000 volunteers from different religious, civic, business, professional, labor, youth, educational and non-government organizations undertake the monitoring work. NAMFREL has a permanent secretariat, headed by a secretary general. Although NAMFREL’s primary work relates to monitoring elections, between elections it mobilizes volunteers as part of its Good Governance Program to monitor Government procurement as part of its anti-corruption work. 19. Other factors that strengthened the Project’s relevance included:

• Well-tested model: The relevance of the Project was strengthened by the fact that it built on past experience and successful partnerships. MMP2 successfully built on MMP1 that developed simple, easy-to-use tools and that evolved and were improved as the lessons learned in MMP1 were reflected in the design of MMP2.

• Non-confrontational approach: NAMFREL’s approach reflects a belief that both the Government and citizens have roles to play to promote good governance and that the best results are achieved if they cooperate constructively and are non-confrontational. To build rapport and trust with DOH, NAMFREL’s approach focuses on system reform rather than “naming and shaming” DOH and hospital officials and DOH and the hospitals as institutions. NAMFREL’s approach shows that NGOs can raise issues in a difficult area without the officials feeling threatened.

20. Overall, the Project’s approach and design targeted a relevant problem and identified an applicable cause-and-effect design. 21. Measurable indicators to track Project achievements: The Project Proposal clearly sets out timelines and deliverables against which actual implementation can be assessed. The MMP2 Project Proposal also sets out quantified targets for three key result areas: (i) more transparent and fair procurement process: 100% compliance of hospital BACs on the provisions of the Procurement Law and its IRRs; (ii) more efficient and cost-effective delivery of drugs and medicines: 75%-100% delivery reports of drugs and medicines; and (iii) timely allocation of priority drugs and medicines to hospital’s beneficiaries: 50%-100% inventory report of priority drugs and medicines. Other expected outputs that were identified in the Project proposal included: (i) an increase in the number of trained volunteers on the provisions of the Procurement Law and its IRRs; (ii) improved monitoring tools; (iii) documentation of Project results and notes of experiences (minutes of consultations/meetings, project evaluation and recommendations to DOH); and (iv) analysis of drug prices. The Evaluation Team used these benchmarks as a framework within which to assess implementation and outcomes. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!29 See http://www.namfrel.com.ph/v2/home/index1.htm

!

12!

22. Cells phones are widely used in the Philippines. Mobile technology (e.g., mobile phones; smartphones; tablets) is changing the ways that NGOs and development professionals can conduct surveys, disseminate information to program participants and collect beneficiary feedback. For future projects, NAMFREL may wish to explore whether advances in mobile technology could be used to better monitor the activities of its volunteers and ease the preparation of comprehensive reports. One reference to this emerging technology is Pact’s Mobile Technology Handbook 30 . That handbook discusses how mobile technology can improve data collection, platforms such as Magpi, Mobenzi, Ushahidi/CrowdMap and FrontlineSMS and outlines ways of creating an organizational strategy around mobile technology. For example, Magpi has developed an inexpensive system to collect data using phones and tablets31. NAMFREL could create electronic forms on Magpi or some competing platform that volunteers could fill by SMS or through an app. Magpi stores, analyzes, and presents the data in several formats.

Overall Assessment of the Project Approach and Design 23. The Evaluation Team’s rating of the Project’s Approach and Design was Highly Satisfactory (4). Although it was not well documented in the Approach Paper, the Evaluation Team’s research clearly shows that MMP2 targeted a relevant problem. Corruption is a problem in the health sector and the pharmaceutical supply chain in many countries and the prices of drugs in the Philippines are, for inexplicable reasons, multiple times higher than the prevailing international prices. There is a supportive legal/regulatory environment for the Project in that CSO monitoring Government procurement is a legal requirement in the Procurement Law. DOH’s procurement system is well developed, with one exception. Although CSOs were invited to attend 100% of BAC meetings, volunteers actually observed only 19% of the meetings DOH-wide in 2010. NAMFREL is an NGO with a good international reputation and has a nationwide reach. NAMFREL has built an increasingly strong, collaborative partnership with DOH over the past decade. The Project proposal included a clear statement of objectives and activities that were feasible for the given resource envelop and benchmarks against which actual implementation and outcomes could be assessed. Significant improvements in the MMP2 design over that of MMP1 included: (i) tripling the number of hospitals covered; and (ii) undertaking more analysis of the prices resulting from the bidding process by comparing prices among hospitals and with over-the-counter retail prices.

B. Project Implementation 24. MMP2 was implemented as expected. The plan was to implement the Project over a 12-month period beginning in September 2010. The Project was implemented on time and was largely completed in October 2011. The NAMFREL Secretariat efficiently undertook the Project planning, management, coordination, monitoring and training activities and did the necessary analytical work and report preparation. The two Project Coordinators financed under the PTF grant played lead roles in undertaking this work.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!30 http://www.pactworld.org/blog/pact’s-mobile-technology-handbook-debuts 31 See Magpi Mobile Data Collection QUICK GUIDE and www.magpi.com.

!

13!

Actual achievements relative to the implementation steps set out in the Project proposal are discussed below.

Consultation with hospitals and regional health offices/Signing of the Memorandum of Agreement with DOH 25. NAMFREL and DOH signed a Memorandum of Agreement (MOA) in 2004, when NAMFREL began to monitor DOH procurement. By 2010 that MOA was out of date. There was a new administration in power and none of the people who had signed the 2004 MOA were still involved with DOH or NAMFREL. The new, updated MOA was signed by the DOH Secretary and the NAMFREL Chairperson and witnessed by a DOH Assistant Secretary/Chairperson of the Integrity Development Committee (IDC) and the NAMFREL Director General on 9 August 2010. The MOA outlined the roles and responsibilities of both parties. The importance of the MOA was stressed by both NAMFREL and DOH as it signified DOH’s official, high level endorsement of NAMFREL’s role in monitoring procurement and gave NAMFREL and its volunteers legitimacy when the hospitals were approached. Without active DOH support from the highest levels, MMP2 was unlikely to succeed. Given these factors, PTF was correct in requiring the new MOA to be signed as a pre-condition to signing the MMP2 Project Grant Agreement 32 . After the MOA was signed, DOH informed the hospitals that NAMFREL was an accredited NGO and that they should invite NAMFREL to monitor all phases of procurement. NAMFREL selected and trained the volunteers. The complete procurement cycle was to be monitored including bidding (NAMFREL was to observe all phases of the bidding and award process from pre-qualification through bid opening to post qualification and bid award), drug delivery and storage and use within the hospital. NAMFREL was also to collect data on comparative prices from commercial drug stores and compare the prices paid by the hospitals as a result of the bidding process. 26. The 28 DOH hospitals were chosen based on the 2004 and 2008 Commission on Audit (COA) reports on irregularities in the procurement process of certain DOH hospitals 33 and the commitment from the local NAMFREL chapters for active participation in the Project. Nine of the 11 hospitals covered under MMP1 were also covered under MMP2 (see Figure 2). NAMFREL held consultation meetings with hospital officials in November/December 2010 to seek the support and cooperation for the Project from the hospital directors, chairpersons of the hospital IDCs and BACs, and the supply and the pharmacy officers. During the meetings NAMFREL provided copies of the MMP2 proposal and MOA, reminded the officials that relevant documents should be made available online and stressed that NAMFREL’s approach involved constructive engagement to support the DOH procurement process rather than confrontation.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!32 The grant agreement was signed on 11 August 2010. 33 See Commission On Audit 2004 Report. Department of Health. Drugs And Medicines Inventory. Pages 59-66.

!

14!

Figure 2: Hospitals Included in the MMP2 Design A. National Capital Region C. Southern Luzon 1. Amang Rodriguez Memorial Hospital 1. Batangas Regional Hospital 2. Dr. Jose Fabella Memorial Hospital1/ 2. Bicol Medical Center 3. Dr. Jose Reyes Memorial Medical Center 3. CHD for Bicol Region 4. National Center for Mental Health1/ 4. CHD for Mimaropa1/ 5. Philippine Childrens’ Medical Center D. Visayas 6. Rizal Medical Center 1. Corazon Locsin Montelibano Memorial

Regional Hospital1/ 7. Research Institute for Tropical Medicines1/ 2. Governor Celestino Gallares Memorial

Hospital1/ 8. CHD for Metro Manila 3. Eastern Visayas Regional Medical Center B. Northern Luzon 4. CHD for Western Visayas 1. Baguio General Hospital and Medical Center1/

E. Mindanao

2. Cagayan Valley Medical Center 1. Caraga Regional Hospital 3. Dr. Paulino J. Garcia Memorial Research Medical Center1/

2. Margosatubig Regional Hospital1/

4. Mariano Marcos Memorial Hospital Medical Center

3. Northern Mindanao Medical Center

5. Veterans Regional Hospital 4. Southern Philippines Medical Center 6. CHD for Central Luzon 5. CHD for Northern Mindanao

6. CHD for Southern Mindanao Notes: 1 = Included in MMP1. Two CHDs for the Cagayan Valley and Cabrazone were covered under MMP1 but were dropped under MMP2. Source: NAMFREL-DOH Medicine Monitoring Project 2. Final Report. July 2012

Recruitment and training of NAMFREL volunteers 27. To support the capacity of the local NAMFREL chapters, assistance was mobilized from other local CSOs, NGOs, the private sector and local colleges and universities34. The other local organizations that supported MMP2 were:

• Environmental Legal Assistance Center for the Governor Celestino Gallares Memorial Hospital in Tagbilaran City, Bohol.

• Saint Paul University Philippines for the Cagayan Valley Medical Center in Tuguegarao City, Cagayan.

• Babas Foundation Incorporated for the Southern Philippines Medical Center and the CHD for Southern Mindanao in Davao City, Davao Del Sur.

• Concerned Negrenses for Good Governance for the Corazon Locsin Montelibano Memorial Regional Hospital in Bacolod City, Negros Occidental.

• EMS Components Assembly Incorporated for the Research Institute for Tropical Medicines in Alabang, Muntinlupa City.

• In areas where the local NAMFREL chapters did not wish to participate in MMP2, (e.g., the cities of Mandaluyong, Marikina and Quezon) NAMFREL sought the support of individual volunteers.

28. The local NAMFREL team leader played the lead role in was recruiting and organizing the training of volunteers, and deploying them. Although problems were encountered in a few areas, engaging the local communities in this manner was

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!34 Agreements were entered into with local CSOs.

!

15!

generally successful and this approach helped to ensure that sufficient volunteers were available for MMP2 in most areas35. 29. NAMFREL organized a comprehensive orientation/training session 36 to familiarize the volunteers with the Project, their role and the provisions of the Procurement Law and its Revised Implementing Rules and Regulations (RIRR). The training was designed to provide the volunteers the knowledge on the Procurement Law and its recently amended RIRR. The seminar served as a refresher course for experienced volunteers and to train and orient new volunteers. 32 volunteers, including 18 provincial team leaders and 14 field volunteers, attended the training. During the training, the team leaders were individually provided with copies of the Handbook on Philippine Government Procurement, the Procurement Law and its RIRRs, case studies and training materials. The training was important because some of the volunteers had limited knowledge of the Government procurement in general, the Procurement Law and its RIRRs or their role as volunteer monitors of DOH’s procurement cycle. A training-the-trainers approach was adopted. Two volunteers from each region were invited to attend the training in Manila with their expenses paid by the Project. When the trained volunteers returned home, they trained other volunteers. 30. The training was designed to provide the volunteers with the knowledge and skills to determine whether the BACs conducted procurement activities in accordance with the provisions of the Procurement Law and to identify the steps to be taken if irregularities were observed. The design of the training course drew on international and domestic experience and updated the training material prepared for MMP1. The training lasted for three days with the third day devoted to case studies/simulations/role playing and mock bidding. Staff from the National Center for Pharmaceutical Access and Management (NCPAM)37 provided presentations as part of the training sessions to share their experience and expertise on the drug pricing and distribution and provided insights. All volunteers interviewed by the Evaluation Team stated that the training was practical and useful and helped them to understand their roles and to equip them for their job. The NAMFREL Secretariat advised that ideally training would be provided every six months to train more volunteers and as refresher training but that could not be done because of a limited budget. 31. NAMFREL did not provide training for DOH or hospital staff. However, concerned DOH staff and BAC members undergo the training consistently and DOH felt that they all understand the procurement rules and regulations, something that was confirmed by the most recent Philippine Country Procurement Assessment Report. DOH advised that it learned lessons from MMP1 that have now been integrated into the procurement processes. For example, there is now a code of conduct clause in the bid documents that prohibits things like gifts and transactions with individuals involved in the BAC, which is codified in the DOH procurement manual. BAC Members and suppliers have all received training on the Code of Conduct.

Monitoring on Public Bidding, Delivery and Inventory of essential drugs and medicines

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!35 Some problems in this area were experienced in MMP1. 36 At the New Horizon Hotel in October 2010. 37 http://www.ncpam.doh.gov.ph

!

16!

32. RHs and CHDs invited NAMFREL observers to BAC meetings38. In some cases the invitations were sent to the NAMFREL Secretariat, which forwarded them to the local team leaders so volunteers would be deployed to observe the procurement activities. In other cases the hospitals sent the invitations directly to the local team leaders with a copy to the NAMFREL Secretariat. The volunteers generally submitted the required reports to the NAMFREL Secretariat recording their observations and whether the procurement activities followed the provisions of the Procurement Law. 33. Similar to MMP1, NAMFREL volunteers encountered some difficulties of not being able to attend BAC meetings and medicine delivery and inventory because of schedule conflicts, particularly for volunteers that were employed, or the notice of meeting39 was received late. NAMFREL advised the Evaluation Team that failure to attend meetings was roughly equally divided between these two reasons. All volunteers interviewed by the Evaluation Team reported that they received a transport allowance under the Project to attend meetings that facilitated their ability to attend meetings. 34. To make up for the unattended deliveries, the volunteers got copies of pertinent documents (e.g., Delivery Receipts; Inspection and Acceptance Reports). According to NAMFREL’s Project Completion Report (PCR) those documents were assessed by the NAMFREL Secretariat to determine if the deliveries were consistent with in the contract and tallied with the inventory of items. This approach on relying on signed documents to validate the receipt of medicines was, in the view of the Evaluation Team, a pragmatic approach since most drugs are delivered by courier services and the schedules are not always known in advance, thus making physical observation of deliveries challenging. Also, the medicine deliveries are typically staggered over a year to avoid expiry dates problems and limiting the storage space requirements. 35. At most of the hospitals visited by the Evaluation Team, both hospital officials and the volunteers advised that most of the BAC meetings were attended during the life of the Project, although there has been a decline in attendance since the Project ended. Monitoring of deliveries, storage and use was less frequent. Based on the findings of the Evaluation Team, the best monitoring results were achieved when: (i) there was close contact between the local team leader and the BAC secretary and notices of meetings were communicated by phone, text or E-mail with the formal written notice hand delivered or available when the volunteer arrived at the meeting; and (ii) the local team had 3 to 5 volunteers as that allowed the volunteers to cover for each other if one experienced scheduling conflicts. NAMFREL issued formal IDs to the volunteers, which they wore to all meetings. That helped to confirm their official status. 36. During MMP2, NAMFREL volunteers monitored the procurement of P2.443 billion worth of items, equivalent to 54% of the P4.517 billion APPs in the hospitals that were monitored. The Project design envisioned monitoring procurement at 28 hospitals, 21 RHs and 7 CHDs (see Figure 2). In practice, somewhat fewer hospitals were covered than was planned. In 2010 procurement was monitored in 20 (71%) of the planned 28

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!38 At DOH headquarters there are four BACs [the Central Office Bids and Awards Committees (COPACs)]. According to DOH, NAMFREL has been invited to monitor the COPACs but has not attended. 39 Although the Procurement Law requires three days advance notice of meetings, that is not always complied with and sometimes invitations were issued at short notice.

!

17!

hospitals covering 41% of the ABC. In 2011 the coverage of the monitoring improved to 2340 (82%) of the targeted 28 hospitals covering 70% of the ABC. Reasons for the lack of monitoring at some hospitals included: (i) an inability to mobilize volunteers in some places; (ii) a delay of some volunteers to committing to MMP2 in 2010 although they participated in 2011; (iii) scheduling conflicts as volunteers tried to balance the time needed to attend BAC meetings, which normally lasted the whole day as many different contracts were considered during the same BAC meeting, with their other commitments, particularly for work; (iv) most hospitals only invited the volunteers to attend the pre-bid conference and opening of bids despite the Procurement Law’s requirement that observers should be invited to every stage of the procurement cycle41; (v) short advance notice for some meetings42; and (vi) a limited number of volunteers in some areas to cover for other volunteers who had scheduling conflicts. The NAMFREL Secretariat followed up to try to resolve these problems during implementation. To partially compensate for the lack of monitoring at some hospitals, the NAMFREL Secretariat requested and reviewed some key documents43 to verify the procurement followed the provisions in the Procurement Law. The details of the outcomes associated with the monitoring are discussed under Outcomes in Section C below. 37. The coverage of the monitoring was strongest for the BAC meetings and, similar to the pattern encountered for MMP1, was lower for the medicine delivery and storage/use/warehousing phases. A contributing factor for the decline in monitoring the delivery of medicines related to the system used to deliver medicines to hospitals and to use the medicines in the hospitals. Typically the pharmaceutical companies deliver by commercial delivery/courier companies rather than in bulk medicines. Also, rather than delivering one year’s supply of medicine at one time the deliveries are staggered depending on volumes, whether the drugs are fast moving or slow moving, the shelf life of the drug and whether the pharmaceutical companies are required to replace unused drugs before their expiry date is reached44. Also, the dispatch of medicines from the hospital pharmacy and storage area to the various hospital departments happens on a real time basis throughout the day and cannot realistically be physically monitored by the volunteers. All of these factors made it difficult to forecast exactly when medicines will be delivered to hospitals or sent to the hospital departments and to arrange for the volunteers to physically monitor the deliveries or use in the hospitals. This issue was discussed in NAMFREL’s consultation meetings with the team leaders. It was agreed that the volunteers would shift to post-delivery monitoring from physical monitoring. This approach required securing copies of pertinent documents from the hospital supply and pharmacy offices to validate the implementation of procurement contracts (e.g., signed delivery receipts; inspection acceptance reports; certificates of payment; requisition

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!40 The hospitals not monitored in 2011 included: (i) Veterans Regional Hospital, (ii) Eastern Visayas Regional Medical Center, (iii) Southern Philippines Medical Center; and (iv) CHD for Southern Mindanao. 41 NAMFREL’s PCR stated that the pre-bid conference and the opening of bids are the only time that BACs convene and that the other stages of procurement (e.g., the post-qualification evaluation) is done by the BAC’s Technical Working Group, takes place over more than one day and often convenes at short notice as doctors balance the need to attend to patients and the need to attend Technical Working Groups. 42 Section 13.3 of the RA 9184 provides that “observers should be informed at least three (3) days before the public bidding activity.” 43 These documents included copies of the: (i) Annual Procurement Plan; (ii) newspaper advertisements; (iii) minutes of the pre-bid conference, submission and opening of bids; (iv) post-qualification summary report, and (v) notices of award. 44 A standard clause is included in drug supply contracts whereby pharmaceutical companies are required to replace medicines in inventory that are nearing their expiry date.

!

18!

issuance slips; monthly inventory issuance report). Analysis of these documents complimented the physical monitoring and inspections of inventories and drug expiry dates. This approach seems reasonable to the Evaluation Team as a pragmatic way of addressing these problems encountered during implementation. However, some volunteers could not obtain the necessary documents to do post-delivery monitoring. Also, some volunteers felt that it was not appropriate for them to monitor delivery and use for those cases that they did not observe the bid opening, a problem that was often the case in 2010 when MMP2 was just beginning. The Evaluation Team does not believe that there is a problem of post-delivery monitoring if the volunteers did not observe the earlier phases of the procurement cycle. 38. The volunteers also inspected medicine warehouses and storage in the RHs and CHDs to assess whether pharmaceuticals were stored in clean, dry properly temperature controlled and secure facilities. The use of photos usefully complemented physical inspection reports that were submitted to the NAMFREL Secretariat. Problems were reported to DOH and the IDC.

Quarterly meeting with NAMFREL team leaders/submission of Progress Reports 39. NAMFREL used a number of methods to monitor Project implementation and to help resolve problems encountered. The local team leaders submitted reports to the NAMFREL Secretariat that, in turn, compiled aggregate data based on the information included in the monitoring reports. The Secretariat and Project Coordinators maintained contact, monitored developments and provided advice to the local team leaders via text, telephone and E-mails. The Project Coordinators visited each region once during the one-year implementation period to meet with the volunteers and hospital officers to monitor implementation and to help resolve problems. There were two quarterly meetings45 with the team leaders to discuss challenges and resolve issues encountered by the volunteers. 40. PTF monitored the Project based on brief monthly progress reports submitted by NAMFREL, informal phone/text/E-mail messages and more substantial progress reports that summarized the activities undertaken, the progress made, plans to resolve problems encountered and financial matters and a review of NAMFREL’s comprehensive PCR. The Project files show that the PTF visited NAMFREL’s offices on 5 April 2011 to formally review the of the Project status, problems encountered and propose remedial actions. PTF’s monitoring mission concluded that activities and timeline of deliverables were in line with MMP2’s stated objectives. Given that the Project was implemented in a one-year time span, and that implementation was relatively smooth, the level of PTF’s monitoring was appropriate. NAMFREL felt that PTF’s monitoring was appropriate, helpful and not overly onerous. PTF released funds in a timely manner, after NAMFREL submitted the necessary documentation. PTF satisfactorily administer the Project.

Developing comparative prices for essential drugs and medicines to aid DOH hospitals and Chad’s in determining acceptable bid prices resulting from public bidding. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!45 Held on 19 March and 2 July 2011 respectively.

!

19!

41. NAMFREL’s research found that in the Philippines pharmaceutical products were more expensive than in neighboring countries like Thailand, Malaysia and Indonesia. The high cost of medicine limits the access of some people to necessary drugs, particularly the poor suffering from treatable diseases. NAMFREL focused on drugs used to treat the top causes of death in the Philippines46. NAMFREL’s price monitoring was designed to support Napalm’s review and updating of the essential drug list. The volunteers visited local drug stores to collect information on the retail, over-the counter prices to determine their affordability and availability in their location. The volunteers forwarded the results of their price survey to the NAMFREL Secretariat, which compiled and analyzed the data. The resulting analysis was reported in the MMP2 PCR (see Section C below on Outcomes). 42. In the Project proposal it was expected that MMP2’s work on comparative prices work would provide an analytical benchmark to help DOH and the hospitals to determine whether the ABC allocations for drugs were appropriate and whether the prices resulting from the public biddings were reasonable. Hospitals should never pay more for drugs than the over-the-counter price given the volumes of drugs purchased by hospitals and the need for drug stores to mark up their prices to generate the funds to pay for their rents, overheads, staff and profit. Given these factors, the Evaluation Team believes that the over-the-counter retail drug price is too high to be a relevant comparator when judging the reasonableness of the prices resulting from the bidding process. The comparative prices were expected to be posted in the NAMFREL’s website and updated every three months. This did not happen and was an over-optimistic expectation. 43. Although NCPAM has undertaken a considerable amount of work to develop reference prices for BACs during the past two years, which is described in more detail in Section C below, that work appears to be unrelated to, and not influenced by, the price comparison work financed under MMP2. NCPAM officers interviewed by the Evaluation Team were not aware of MMP2’s work in this area. There was scope for NAMFREL to more actively share its work with comparative pricing with NCPAM.

Project evaluation to review whether the Project objectives were achieved and to identify areas where further improvements were needed 44. NAMFREL produced a comprehensive, evidence based PCR that was submitted to PTF in July 2012. NAMFREL’s assessment provided adequate evidence to conclude that MMP2 broadly achieved its objectives. NAMFREL’s self-evaluation identified Project achievements as well as areas where improvements were needed. The Evaluation Team’s positive assessment of the Project is given in Section A of Chapter II below.

Presentation of accomplishments, observations and findings to DOH and to the public 45. MMP1 and MMP2 are relatively well known projects, both domestically and in some international circles. NAMFREL’s dissemination activities included making international and domestic presentations and providing information to publicize !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!46 Those diseases include heart disease, stroke, cancer, chronic obstructive pulmonary, diabetes, dengue fever, maternal death, kidney failure and perinatal disease. See Parallel Universes /http:emeritus.blogspot.com/2007/07/Philippines-top-ten-causes-of-mortality.html

!

20!

the MMP projects. Examples include: (i) being invited to make a presentation on MMP by the Coalition Against Corruption; (ii) making a presentation in Colombo; (iii) presenting the Project at a PTF conference in India; and (iv) presenting the Project at a PTF meeting at the Asian Institute of Management in 2010 to share experience among NGOs. Earlier NAMFREL had presented the MMP1 results in Hong Kong47 during the Opendoors 2009, a regional forum on procurement monitoring as a tool for social accountability48. 46. NAMFREL co-hosted two international workshop/conferences in November 2014. The first was the Asian Citizen Election Monitoring Organizations (EMOs) workshop and the second was the 9th Declaration of Principles for International Election Observation. NAMFREL advised the Evaluation Team that the MMP program was well received and elicited a lot of interest from foreign CSO and EMO participants. NAMFREL presented MMP as one of the activities EMOs can undertake to complement their election work. 47. Many hits came up when the Evaluation Team undertook an Internet search for “NAMFREL Medicine Monitoring Project.” Some were on the NAMFREL website49, some were on the PTF website and some were elsewhere (e.g., Open Contracting; ANSA-EAP; i-site.ph; Stocktaking of Social Accountability Initiatives, Tools, and Approaches Used By Civil Society Organizations To Monitor Public Service Delivery in the Philippines. La Salle Institute of Governance. Che Aquino. 2012; Coalition Against Corruption. Transparent Accountable Governance (TAG) project funded by USAID. Makati Business Club). 48. Based on the feedback that the Evaluation Team received there was scope to improve the depth and dissemination of the MMP2 PCR in DOH, particularly through the IDC. Some of the DOH and NCPAM officials and hospital officers met by the Evaluation Team could not recall seeing the report or being briefed on the specific Project outcomes and achievements. DOH cannot be expected to act on the MMP2 findings and recommendations unless the Project results are actively disseminated by NAMFREL in DOH and NCPAM through multiple channels and on multiple occasions.

Financial management of the Project 49. NAMFREL provided a $33,350 grant to finance MMP2. The grant was fully disbursed in three installments: (i) $15,000 upon signing of the grant agreement; (ii) $10,000 six months after the start of the Project, subject to: (a) a certified statement showing the itemized use of the first tranche; and (b) satisfactory progress in implementing MMP2; and (iii) $8,350 upon submission of a satisfactory PCR providing NAMFREL’s self-evaluation of the project. PTF disbursed all of the funds to NAMFREL after the necessary conditions for disbursement were fulfilled. The use of three tranches was appropriate to balance NAMFREL’s need for cash flow and PTF’s need to manage

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!47 See http://www.namfrel.com.ph/v2/activities/good_governance.php During MMP2’s implementation updates were posted on NAMFREL’s webpage. 48 About 50 participants from civil society, government practitioners and donors’ agencies from the East Asia-Pacific region attended. According to NAMFREL’s MMP2 proposal, several participants showed interest in duplicating the MMP1 concept for medicine monitoring project in their country. 49 See http://www.namfrel.com.ph/v2/activities/good_governance.php

!

21!

the Project financing in a way that tied the release of funds to satisfactory Project performance. 50. A well-known firm professional firm audits NAMFREL’s annual corporate accounts that include MMP2 revenues and expenditures. Consistent with the practice of most NGOs, NAMFREL’s small project accounts are not independently audited. Based on the financial data included in the PCR, MMP2 was implemented on budget with both the approved budget and the actual expenditures were slightly over P3 million, including the imputed value of services provided in kind (e.g., provision of office space for the NAMFREL Secretariat; the time of the volunteers). Cash expenditures until the end of October 2011 were P1.511 million, very close to the budgeted amount of P1.500 million. Counterpart contributions in kind accounted for about half of the total Project cost. Although NAMFREL incurred some expenditures after October 2011, most of the expenditures took place during the peak of MMP2’s implementation, i.e., before the end of October 2011. Most of the PTF funds were used to finance NAMFREL’s two project coordinators, training the volunteers, travel and accommodation and per diems for volunteers when they attended BAC or hospital IDC meetings (see Annex 4). NAMFREL provided funds to the volunteers and local CSOs on a reimbursement basis. Overall financial management and cost control for MMP2 were good.

Other Evidence of Successful Implementation 51. Other factors that support the conclusion that the Project was relevant and well implemented includes:

• NAMFREL is DOH’s main NGO partner: Senior DOH officials and hospital officials advised the Evaluation Team that while other NGOs have been invited to monitor procurement, NAMFREL is the only NGO that attends BAC meetings50. NAMFREL is DOH’s main NGO partner in monitoring all types of procurement for the full procurement cycle.

• Support from top DOH executives and hospital administrators: Two DOH assistant secretaries and all hospital administrators, BAC chairs and BAC secretaries interviewed by the Evaluation Team welcomed NAMFREL’s involvement and viewed it as useful and important. NAMFREL’s approach emphasizes working constructively with DOH officers and hospital officials, which enhances the acceptance of this approach to social accountability. As a result, there are officials in DOH and the hospitals who support, champion, listen to, and act on NAMFREL’s recommendations, an attitude that cascades down to the rank-and-file.

• Sustained engagement: Sustained engagement is needed to make significant progress in changing the behavior of government officials and bidders and improving transparency to help create an environment that reduces opportunities for corruption. At the time of the MMP2 evaluation, NAMFREL had been engaged with DOH for a decade. The relevance of the Project was strengthened by the fact that it built on past experience and a successful partnership. Supporting efforts of CSOs to contribute to openness and transparency of procurement has been a major focus of PTF’s program in the Philippines. PTF rewarded NAMFREL’s good performance under MMP1 by providing the financing

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!50 One other NGO is invited but does not send observers to BACs.

!

22!

necessary for MMP2. PTF is NAMFREL’s major financial contributor for medicine monitoring.

• Member of DOH’s IDC: NAMFREL is a member of DOH’s IDC that provides a platform to discuss governance matters at the highest levels. DOH officers confirmed that NAMFREL attends the meetings regularly and that its inputs are valued. The local NAMFREL coordinators are members of the hospital IDCs.

Overall Assessment of the Project Implementation 52. Based on its analysis, the Evaluation Team rated Project implementation as Highly Satisfactory (4). MMP2 was implemented on time and within budget and the financial management was good. The planned activities set out in the Project proposal were delivered and the expected outputs largely materialized. The volunteers reported that the training and orientation were useful and that they learned about their role in the context of the Procurement Law. The new Memorandum of Agreement (MOA) between DOH and NAMFREL was signed as expected and proved to be useful in securing high-level support for the Project in DOH and in the hospitals. Feedback received by the Evaluation Tem indicates that the concept of NAMFREL volunteers observing the procurement, delivery and use of medicines, hospital infrastructure and equipment was actively welcomed by DOH and the hospitals. Some volunteers built very strong partnerships with their hospitals. Monitoring by both NAMFREL and PTF was good and action was taken to resolve challenges as they arose during implementation.

C. Outcomes, Impact and Sustainability

Outcomes

Procurement Monitoring 53. Under MMP2 NAMFREL’s procurement monitoring increased to cover far more procurement than under MMP1. During MMP2 procurement was monitored for contracts valued at P2.448 billion, equivalent to 54% of the APPs for the hospitals covered by the Project (see Figure 3). This was a significant increase, nearly 6 times, over the P412.6 million of procurement monitored under MMP1. Part of this reflects the 3 fold increase in the number of hospitals where monitoring actually took place (see Figure 3).51 However, the other half of the increase indicates that with the experience gained and a stronger partnership with DOH and the hospitals, efficiency and effectiveness improved significantly under MMP2 compared to MMP1. Medicines accounted for 41% of the value of the procurement monitored under MMP2 and non-drug related procurement accounted for 59%52. The proportion of the APP monitored improved significantly from 2010 (41%) to 2011 (71%). This reflects the fact that in 2010 procurement monitoring only began during the fourth quarter after the volunteers were mobilized, oriented and trained. These ratios of contracts monitored far exceed the 19% of civil society and/or professional organizations' attendance in public bidding activities reported in the most recent Philippine Country Procurement Review DOH-wide in 2010. These figures demonstrate that MMP2 was effective in achieving the desired outputs and outcomes.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!51 Under MMP1 monitoring took place in 8 hospitals. Under MMP2 monitoring was undertaken at 23 hospitals. 52 Medicines accounted for 51% of the procurement monitored under MMP1.

!

23!

Figure 3: Procurement Monitoring of Pharmaceuticals, Infrastructure, Equipment and Supplies 2010-2011 (Peso million)

Procurement Monitoring Region RHs/CHDs

Covered1/ APP Drugs Non

Drugs Total %

National Capital Region 8 out of 8 1,639 213 480 693 42 Northern Luzon 5 out of 6 928 382 421 803 87 Southern Luzon 4 out of 4 737 73 266 339 46 Visayas 2 out of 4 606 62 88 150 25 Mindanao 4 out of 6 590 271 192 463 78 Total 23 out of 28 4,500 1,001 1,447 2,448 54 Notes: 1 = These figures are for 2011. In 2010, the monitoring covered 20 of 28 hospitals. Source: NAMFREL. MMP2 Project Completion Report 54. MMP2’s effectiveness in achieving the desired procurement monitoring outcomes varied by region. Coverage was very good in Northern Luzon (87%) and Mindanao (78%), modest in Southern Luzon (46%) and the National Capital Region (42%) and poor in the Visayas (25%) (see Figure 3). Volunteers could not be mobilized to cover two or the four hospitals in the Visayas. The Evaluation Team originally planned to visit Iloilo in the Visayas as part of its fieldwork but changed its plans when NAMFREL advised that no monitoring took place at that hospital53. Monitoring at the Baguio General Hospital was much more effective under MMP2 than under MMP1. That reflected the strong team of volunteers assembled in conjunction with a local church-affiliated CSO and a turn over in the head of the Baguio General Hospital. The volunteers reported that the incumbent is more receptive to citizen monitoring than was his predecessor.

Delivery Monitoring 55. Similar to MMP1, MMP2 experience demonstrated that monitoring medicine deliveries was more challenging than monitoring procurement. The factors that contributed to this outcome are summarized in Section B above. Deliveries were monitored in 24 of the 28 hospitals. The value of medicine deliveries monitored totaled P303 million, 62% of the value of the contracts awarded and 39% of the ABCs (see Figure 4). This was a substantial increase, about 5 fold, over the P58 million of deliveries monitored under MMP1, equivalent to 32% of the contract ABC of P180 million. While the increase in the number of hospitals covered would explain a 3-fold increase, the remainder reflects better efficiency and effectiveness under MMP2 than under MMP1. Figure 4: Delivery Monitoring of Pharmaceuticals, 2010-2010 (Peso million) Region RHs/CHDs

Covered ABC Value of

Contracts Awarded

Value of Contracts Monitored

% of ABC

% of Value of

Contracts Awarded

National Capital Region 8 out of 8 222.20 162.08 114.17 51 70 Northern Luzon 4 out of 6 175.41 134.40 71.44 41 53 Southern Luzon 4 out of 4 76.51 35.52 21.13 28 59 Visayas 2 out of 4 60.26 25.98 16.20 27 62 Mindanao 6 out of 6 246.88 128.17 79.57 32 62 Total 24 out of 28 781.26 486.15 302.51 39 62 Source: NAMFREL. MMP2 Project Completion Report

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!53 The Evaluation Team decided not to visit hospitals in the parts of the Visayas that were overloaded with coping with the effects of the devastation caused by Typhoon Yolanda.

!

24!

56. Nationally, the value of the contracts for which delivery was monitored represented 39% of the ABC. There was some variation by region, ranging from lows of 27% in the Visayas and 28% in Southern Luzon to highs of 41% in Northern Luzon and 51% in the National Capital Region. Monitored delivery as a percentage of the value contracts awarded averaged 62% nationally, ranging between a high of 70% in the National Capital Region to a low of 53% in Southern Luzon (see Figure 4).

Use and Distribution Monitoring 57. As was the case for MMP1, monitoring medicine storage, inventories and use within the hospitals was more challenging for the volunteers than was monitoring procurement and deliveries. Medicine inventories and use were monitored in 22 of the 28 hospitals. In total, the value of medicine inventories monitored totaled P198 million, 41% of the value of the contracts awarded and 20% of the ABCs (see Figure 5). This was a 6-fold increase over the P32 million of medicine inventories monitored under MMP1, equivalent to 18% of the contract ABC of P180 million. The increase in the number of hospitals covered under MMP2 explains a 3-fold increase and the other 3-fold increase reflects better efficiency and effectiveness under MMP2 than under MMP1. Despite this improvement, the coverage of inventory monitoring as a proportion for both the ABCs and the value of awarded contracts was modest. While these proportions are low in all regions, the 6% and 14% of ABCs in Northern and Southern Luzon and 13% of the value of awarded contracts in Southern Luzon are clearly unacceptable outcomes (see Figure 5). These figures indicate that NAMFREL, working in partnership with DOH and the hospitals, should make special efforts in the design, training and supervision of future projects to improve the coverage of medicine storage, inventories and the dispensing from the hospital pharmacies to the various hospital departments. Figure 5: Inventory Monitoring of Pharmaceuticals, 2010-2010 (Peso million) Region RHs/CHDs

Covered ABC Value of

Contracts Awarded

Value of Dispensed Medicine

Monitored

% of ABC

% of the Value of Awarded Contracts

National Capital Region 8 out of 8 220.49 162.08 78.29 36 48 Northern Luzon 3 out of 6 356.71 134.40 48.81 14 36 Southern Luzon 2 out of 4 76.51 35.52 4.46 6 13 Visayas 2 out of 4 60.26 25.98 13.71 23 53 Mindanao 6 out of 6 271.06 128.17 53.22 20 41 Total 22 out of 28 985.03 486.15 198.49 20 41 Source: NAMFREL. MMP2 Project Completion Report 58. Nationally, the value of the contracts for which delivery was monitored represented 39% of the ABC. There was some variant by region, ranging from lows of 27% in the Visayas and 28% in Southern Luzon to highs of 41% in Northern Luzon and 51% in the National Capital Region (see Figure 5).

Price monitoring 59. In addition to assessing local availability, the price monitoring work was designed to help determine if the products offered by suppliers to government hospitals through the competitive bidding process was reasonably priced. The MMP2 PCR summarized the results of the drug price surveys. Table 4.1-A in that report summarized the prices paid for medicines that are used to combat the top killer diseases identified by WHO and

!

25!

DOH for two randomly chosen hospitals – the Baguio General Hospital and Medical Center and the Eastern Visayas Regional Medical Center. NAMFREL’s price analysis found that for the nine drugs for which data was available:

• For 8 of the 9 drugs, the public bidding resulted in prices that were below the retail price. The one exception was Ciprofloxacin. Both hospitals paid more for that medicine than the over the counter price.

• On average the prices paid in the competitive bidding was 56% of the retail price (51% in the Baguio General Hospital and 60% in the Eastern Visayas Regional Medical Center).

60. The Evaluation Team is not convinced that the retail price is the correct benchmark to use in assessing whether or not the prices resulting from the bidding process are reasonable because:

• The Health Action International study found that, on average, the prices of originator brands and generic equivalents were 16 and 3 times higher, respectively, than the prices available on the international market – prices paid by public procurement of medicines were typically between 13 and 40 times the international reference price. Although the multiples were generally lower for DOH hospitals, this suggests that international prices compiled by WHO would be a better benchmark.

• Hospitals purchase large volumes of medicines and drug stores must mark up their prices to generate the funds to pay for their rents, overheads, staff and profit. Accounting for these factors should result in bid prices that are significantly lower than the retail price.

61. Consideration should be given to examining international drug prices to compare benchmarks against which to compare the prices paid in the Philippines for similar drugs and to provide guidance to the BACs. Management Sciences for Health has published the International Drug Price Indicator Guide54 annually since 1986. WHO supported the 2013 edition. The Guide is designed to be a reference tool for those involved in drug procurement by making price information comparative available so that medicines are procured at the lowest possible price. 62. Table 4.1-B in NAMFREL’s PCR compared the average price per dose paid by hospitals in nine regions throughout the country for two drugs: (i) phenytoin (50 mg/ml of 2 mg); and (ii) meropenem (1 g). The analysis found that there was a wide variation in price paid by the hospitals for both drugs – the ratio between the lowest and higher price paid was about 2.6 times. This variance was largely unexplained and was not related to the volume purchased or the distance that the medicine had to be transported. These large differences raise suspicions about collusion, bribery and kickbacks in some hospitals. 63. The 2008 Universally Accessible Cheaper and Quality Medicines Act (Republic Act 9502) gave DOH the power and tools to monitor and regulate drug prices to protect !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!54 http://erc.msh.org/mainpage.cfm?file=1.0.htm&module=DMP&language=English. The Guide contains comparative prices from pharmaceutical suppliers, international development organizations and government agencies. Previous editions were supported by the UK Department for International Development (DFID) and the Medicines Transparency Alliance (2007; 2008), the Bill and Melinda Gates Foundation (2000; 2001; 2002; 2003; 2004; 2005; 2006), the Norwegian Agency for Development Cooperation (1999), the World Bank (1998) and the International Network for Rational Use of Drugs, with funds from the Danish International Development Agency (1996).

!

26!

consumers and public procurement from excessive price mark-ups. During the last two years NCPAM has undertaken a lot of excellent work collecting and analyzing the bid prices to develop a reference guide for medicine prices that will guide the work of BACs. Data were collected, coded and analyzed for over 600 drugs. The resulting extensive database of the purchase prices for essential drugs in all DOH RHs and CHDs. 64. The NCPAM website 55 now publically discloses the data gathered by the electronic monitoring of drug prices. The last full reports available on the webpage were published in 2012, although there is a 2013 electronic spreadsheet available on the website with 123,430 records for the purchases of many drugs by many hospitals throughout the Philippines56. The most recent 2012 report includes line items for 689 drugs showing the generic name, the drug strength/form, the range from the highest to lowest bid and the drug reference price. The drug-monitoring database is based on the prevailing public tender prices in the previous year as reflected in the Purchase Orders submitted by DOH hospitals, the Central Office Bids and Awards Committees (COBACs) and the Philippine International Trading Corporation Pharma Ltd. 65. Based on this work, NCPAM has an established Drug Reference Price Index (DRPI), based on the median price, to guide national and local BACs in the procurement of pharmaceuticals. The prices in the winning bids for all forms of procurement (e.g., public bidding; direct contracting; shopping; negotiated procurement; emergency procurement and consignment) are now not to exceed the DRPI prices. This approach should eliminate clear failures of the bidding process to obtain reasonable prices and the most suspicious procurement decisions. 66. DOH and NCPAM know that some hospitals consistently pay more than other hospitals for the same drugs and in the past has commended some hospitals for having the lowest prices for drugs and medicine57. NCPAM’s data shows that the prices of drugs procured are highly variable with no relationship between price and volume or distance. This is a very useful database as the information can be analyzed to identify clear outliers, raise red flags and identify hospitals that consistently achieve low prices and hospitals that consistently pay unexplained high prices. It provides a strong analytical tool that DOH can use to hold BACs accountable for their procurement decisions. 67. The 2013 data on the NCPAM website was analyzed by the Evaluation Team for two drugs: (i) simvastim 100 mg tablets; and (ii) phenytoin 100 mg tablets (see Figures 8 and 9). Those graphs confirm the very wide range in the prices paid by hospitals for the same drug. For simvastin the median price was higher than the over-the-counter price reported in the MMP2 price survey and for phenytoin both figures were nearly the same. Similar patterns are shown for the graphs for most of the other drugs shown in Annex 3. These comparisons indicate that NCPAM should consider doing more analysis to compare the median price that is used to set the DRPI to the over-the-counter price. If

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!55 http://www.ncpam.doh.gov.ph 56 The plan is to update the prices annually based on the Purchase Orders. 57 For example, the Dr. Paulino Garcia Memorial Research and Medical Centre in Cabanatuan, which received its ISO accreditation on 11 February 2014, the first hospital in the Philippine to do so, was commended by DOH for having low medicine prices. Because DOH’s 2008 drug reference guide was out of date, this hospital consulted with other hospitals when it was concerned about the bid prices. !

!

27!

similar patterns occur for most other medicines, it would appear that a DRPI set at the median level is too high. Clearly hospitals should never pay more than the retail price for any drug.

Figure 8: Analysis of the Contract Prices for Simvastin

Note: For simvastin, the analysis was based on 20 mg tablets. There were 182 awarded bid prices in the database, which are represented by points from lowest to highest. Three very high prices (Php 442.5; Php 450 and Php 1,475) were excluded from the analysis as outliers. The median price of the remaining 179 contracts for 20 mg tablets of simvastin was P16.35, ranging from a low of P1 to a high of P60 (a ratio of 60 times). The retail price, based on the MMP2 survey was P8. Source: Derived from NCPAM’s online database by the Evaluation Team

Figure 9: Analysis of the Contract Prices for Phenytoin

Note: For phenytoin, the analysis was based on 100 mg capsules. There were 112 awarded bid prices in the database, which are represented by points from lowest to highest. One observation with a very high price of Php 3,218.57 was excluded as an outlier from the analysis. The median price of the remaining 111 contracts for 100 mg capsules of phenytoin was Php 33.9, the retail price based on the MMP2 survey was P33, the lowest price was P12 and the highest price was P59.96, a multiple of 5 times. Source: Derived from NCPAM’s online database by the Evaluation Team

68. Hospitals explained that sometimes bidders were from related companies and sometimes technical specifications of the end users (doctors), including decisions that

0

10

20

30

40

50

60

70 Pr

ice

per 2

0 m

g ta

blet

(Php

)

Median Price

Retail Price

0

10

20

30

40

50

60

70

Pric

e pe

r 100

mg

tabl

et (P

hp)

Retail Price

Median Price

!

28!

drug trials needed to be undertaken in the Philippines, resulted in the low bidders being disqualified. Such technical decisions sometimes varied from hospital to hospital. Volunteers do not have the expertise to determine whether technical requirements that excluded some bidders are legitimate or bogus. In some other countries decisions related to certifying that drugs are fit to use of the intended purpose are made nationally by agencies like DOH’s Food and Drug Administration (FDA). 69. The results of NAMFREL’s price survey and the excellent work undertaken by NCPAM documenting the unexplained wide variation in prices for the same drugs across hospitals resulting from the bidding process raise suspicions that there is corruption, collusion and kickbacks in some medicine procurement decisions. Feedback from key informants was that collusion is difficult to identify and very difficult to prove. Because collusion happens outside the formal meetings it is difficult for NAMFREL monitors to detect. Some NAMFREL staff and volunteers felt that more training in this area would be beneficial. 70. Bid rigging or collusion is, according to OECD, when the bidders agree to eliminate competition so that prices are higher and the government pays more58. WHO and Transparency International reports document that collusion takes place in the pharmaceutical sector globally. The World Bank estimates that in some industries or geographic areas where collusive bidding is more prevalent, bid prices are often 30% or more above the government estimate59. 71. NAMFREL’s Guide to Medicine Monitoring in Public Hospitals60 covers: (i) an overview of the MMP1; (ii) the DOH IDC; (iii) the DOH Organizational Structure; (iv) the DOH Procurement System; (v) the DOH Delivery System; (vi) the DOH Inventory Management System; (vii) organizing a team of volunteers; (viii) responsibilities of volunteers; (ix) how to use monitoring report forms; and (x) red flags of corruption in procurement, delivery and inventory. This was published in 2007 and needs to be updated. The section on red flags covers:

• Corruption in procurement including (a) collusion between bidders and project/procurement officials; (b) local cartels or collusion between bidders; (c) excluding qualified bidders; and (d) leaking bid information; (e) advertisement; (f) pre-bid conference; and (g) receipt and opening of bids.

• Corruption in delivery: (a) connivance between the receiving officer and supplier in issuing an approved schedule for delivery despite incomplete documentation; (b) connivance between the receiving officer and supplier in recording a complete delivery although some items were missing; (c) connivance between the receiving officer and supplier in antedating late delivery in return for a payment from the supplier; (d) suppliers attempting to bribe the inspection team; and (e) the inspection team being offered gifts or meals.

• Corruption in inventory: (a) unrestricted access to storage area and access to storage areas by unauthorized personnel; (b) no segregation between receipt of

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!58!Source: OECD. Detecting Bid Rigging in Public Procurement!59 Source: World Bank. Most Common Red Flags of Fraud and Corruption in Procurement in Bank Financed Projects 60 NAMFREL, together with the Transparent Accountable Governance (TAG) Project and the Coalition Against Corruption, with funding support from the Asia Foundation and the United States Agency for International Development published a comprehensive Guide to Medicine Monitoring in Public Hospitals.

!

29!

inventory, recording of inventory account and identification of obsolete or surplus material and their sale/disposal; (c) systematic pattern of improperly labeled inventory; and (d) lack of regular physical inventories.

72. NAMFREL should add additional material in its training courses to cover collusion to supplement the red flags in its Guide to Medicine Monitoring in Public Hospitals drawing on OECD’s Checklist for Detecting Collusion (see Figure 10) and the more detailed checklist for collusion in Annex 5.

Figure 10: OECD Checklist for Detecting Collusion in Public Procurement 1. Look for markets that are more susceptible to bid rigging (e.g., small number of bidders; standardized or simple products; little or no entry). 2. Look for opportunities that the bidders have to communicate with each other during the bidding process, especially prior to the opening of the tenders. 3. Look for indications that the bidders have communicated with each other (e.g., a bidder having knowledge of another bidder’s pricing or is not expecting to be the low bidder; a bidder refers to “industry” or “standard” practices or prices; one bidder picks up or submits bidding material for another firm; a bidder says something that indicates that certain non-public information, or an answer to a question, was learned by talking to another bidder). 4. Look for any relationships among the bidders after the successful bid is announced (e.g., bidders split the extra profit that is earned through bid rigging, either directly or through lucrative sub-contracts or joint bids). 5. Look for suspicious bidding patterns (e.g., patterns of winning bids over the course of many bids – winning bids are rotated among bidders or by geographical area or size of contract; a bidder never wins but keeps bidding; a bidder wins whenever it bids, even if it bids rarely; a bidder may show a pattern of submitting relatively high bids for some tender offers and relatively low bids for other, similar tender offers; pricing may be unusual -- all bids may be unexpectedly high, or discounts or rebates may be unexpectedly small; bids differ from previous, similar procurements for no apparent reason; pricing may not make sense when you consider transportation costs). 6. Look for unusual behavior (e.g., the winning bidder does not accept the contract or withdraws before the award is made; bids are submitted without normal detail or required documentation, or without the necessary information; the number of bidders is unexpectedly small, with some normal bidders not participating or withdrawing their bids). 7. Look for similarities in the documents submitted by different bidders. Bid-riggers sometimes have a single person that prepare all the bids or by a group of people working together. In such cases competing bid documents may be submitted on the same type of paper, with the same postmarks, have the same misspellings, have the same handwriting, use the same wording, contain the same alterations or changes and include the same miscalculations or the same amounts. Source: OECD. Detecting Bid Rigging in Public Procurement

Impact 73. The evidence indicates MMP1, MMP2 and the changing attitudes of DOH, especially during the present administration, have had a clear impact in institutionalizing the process of CSOs monitoring of DOH’s procurement. Before NAMFREL began to be involved in 2004, no NGO monitored DOH’s procurement. The attitude of some DOH officials and hospital officers toward the volunteer monitoring during MMP1 was characterized as passive tolerance because of the legal requirements of the Procurement Law. The attitudes of DOH officials and hospital officers changed for the better under MMP2 and the Evaluation Team would describe it as active engagement. All people interviewed by the Evaluation Team welcomed NAMFREL and the volunteers as active partners who were viewed as helping to improve the transparency of the procurement process and protecting DOH’s reputation with the public.

!

30!

74. DOH and the hospitals reported that NAMFREL and its volunteers were their key NGO partner for procurement monitoring and regularly attended BACs meetings. Another NGO was invited to BACs but DOH reported that that NGO rarely, if ever, attended. There was strong support in DOH and all hospitals visited by the Evaluation Team for NAMFREL monitoring. Two DOH assistant secretaries explicitly made that point to the Evaluation Team. NAMFREL was viewed as a trusted partner and has been invited to be a member of the IDC. NAMFREL regularly attends the monthly IDC meetings and, according to DOH, makes good contributions. 75. The MMP1 PCR identified a considerable number of procurement problems that were reported to DOH, which included: (i) instances when contracts were awarded that exceeded the ABC; (ii) hospitals not issuing Notices to Proceed; (iii) hospitals accepting medicines with less than the required maximum shelf life; (iv) hospitals purchasing medicines beyond the total quantity awarded; and (v) suppliers offering different bid prices for the same medicine in different hospitals.61 76. Procurement problems reported in the MMP2 PCR included: (i) volunteers not being invited to the Technical Working Group to observe the post-qualification work despite the provision in the Procurement Law states that observers should be invited to observe the entire procurement cycle; (ii) two BACs awarded procurement contracts that were more than the approved budget; (iii) the opening of bid proposals by some hospital BACs did not start on time; (iv) there were some difficulties in obtaining relevant documents to monitor deliveries and some were not properly completed; and (v) some inventory related documents were not properly completed and signed. Most of these issues also arose during MMP1. 77. Based on the feedback received by the Evaluation Team and the information in the MMP1 and MMP2 PCRs, there appear to have been fewer problems related to warehousing and storage reported under MMP2 than was the case during MMP1. The MMP1 PCR reported a number of procurement problems: (i) most of the public hospitals had poor storage system and warehouse facilities (e.g., lack of temperature control; water leaks; rodents and pests; lack of shelving and storage space; medicines stored in hallways and insecure areas; lack of refrigerated storage facilities); and (ii) medicines were not properly organized and labeled that made it difficult for the pharmacist to implement the first-in-first-out method of dispensing and distributing medicines. The pharmacies visited by the Evaluation Team were generally well organized, the medicines were labeled and neatly shelved and some temperature-controlled facilities were available. RITM had particularly impressive cold storage facilities. While there continue to be problems with the facilities in some hospitals for storing medicines, there appear to be fewer problems than were reported under MMP1. DOH and the hospitals have taken some actions to address the storage problems reported under MMP1. 78. The Evaluation Team could not develop direct evidence that MMP1 and MMP2 contributed to a reduction in the level of corruption in a way that was measurable and attributable to the Projects. It is plausible to assert the increased transparency

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!61 Many of those problems occurred in the Baguio General Hospital during MMP1, in addition to other hospitals. The volunteers reported that the application of the procurement rules and regulations improved after the new leadership in the hospital assumed office.

!

31!

associated with NGOs monitoring the full procurement cycle contributes to reducing the opportunities for corruption. There is some indirect evidence to support this hypothesis although the Evaluation Team acknowledges that many other factors are also at play, particularly the 2010 change in administration. At most, one could conclude that MMP1 and MMP2 contributed to the findings of recent surveys that found that DOH had one of the best ratings among all Government agencies in terms of combatting corruption. 79. The Social Weather Stations’ (SWS) 2013 Survey of Enterprises on Corruption62 found that 56% of respondents reported seeing ‘a lot’ of corruption in the public sector. Although this was an increase63 over the 43% reported in 2012, the 56% recorded in 2013 was the second lowest figure since 2000. The general perception was that the Government was making progress in combatting corruption -- 73% of respondents believed that the Government efforts to reduce corruption had been effective, marginally less than the 78% recorded in 2012. The Government’s progress in reducing corruption was also reflected in other surveys. The Philippines received better rating in both the 2014 International Finance Corporation’s Doing Business Survey and Transparency International’s Corruption Perceptions Index compared to previous years. 80. Respondents to the 2013 SWS survey64 were asked to rate 26 government institutions for their sincerity in fighting corruption65. The Office of the President for the Aquino administration was perceived as being much more sincere in its efforts to fight corruption than was the case for the previous administration and retained its Excellent rating, the only Government agency to receive such a rating. DOH was one of four agencies that received a Very Good rating for its sincerity rating in fighting corruption, the same rating that it received in 2012. Although the perception that DOH was sincere in trying to combat corruption began to improve from the Modest rating that it received in 2006, the rating increased significantly in 2012 81. A July 2014 survey undertaken by the Makati Business Club asked business executives to rate the performance of 62 Government offices, agencies and services over the previous year by indicating whether or not the respondent was, or was not, satisfied with the agency’s performance. DOH ranked seventh among the 62 agencies, a significant improvement over DOH’s fourteenth place ranking in 2012. This is consistent with the findings of the Philippine Country Procurement Review that DOH’s procurement system compares favorably to the procurement systems of other government agencies (see Annex 6).

Sustainability 82. There is some evidence that the Project benefits were sustainable. Some NAMFREL volunteers have continued to monitor procurement in 2012, 2013 and 2014

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!62 The SWS survey covered the opinion of top executives in 951 companies. 63 These survey results may have been affected by a very high level scandal that was exposed in 2013 involving many senators and congressmen and fake NGOs. 64 The SWS survey was supported by the Australian Aid - The Asia Foundation Partnership in partnership with the National Competitiveness Council and the Makati Business Club's Integrity Initiative programme. 65 The ratings of agencies' sincerity in fighting corruption were grouped into Excellent (+70 and above), Very Good (net +50 to +69), Good (+30 to +49), Moderate (+10 to +29), Neutral (-9 to +9), Poor (-10 to -29), Bad (-30 to -49), Very Bad (-50 to -69), and Execrable (-70 and below). Changes were considered “notable” when the rating moved to a different grade.

!

32!

after the Project ended in 2011. However, some hospitals reported that the frequency of NAMFREL attendance at the BACs has declined. The evaluation identified a general issue related to the volunteers that needs to be considered for future projects – funding their travel allowances after grant funded projects end. Many of the volunteers live on modest incomes and do not have additional personal funds to spend on the cost of travel to attend procurement meetings. Lack of funds sometimes limited the attendance of volunteers, an issue that was raised by both NAMFREL and the volunteers. Although no problems were reported during the implementation of MMP2 when funding from the PTF grant was available, this became a problem after the Project ended since NAMFREL did not have the funds to continue paying for the volunteers’ travel allowances. Finding a solution to this problem is essential for the benefits of the Project and future similar projects to be sustainable. 83. Section 15 of the Procurement Law provides, subject to the availability of funds, for honoraria to be paid to BAC members, the BAC secretariat and the Technical Working Group members but the law and the Implementing Rules and Regulations (IRRs) are silent on whether honoraria should be paid to the official observers. The Department of Budget Management (DBM) issued guidelines for paying honoraria from funds that are generated through the proceeds of the sale of bid documents, fees for the suppliers/contractors to register, charges for copies of minutes of bid opening, BAC resolutions, and other BAC documents, protest fees, liquidated damages and bid/performance security forfeitures66. 84. The Philippine Country Procurement Review 67 included recommendations to strengthen the role of NGOs/CSOs in procurement monitoring grouped under five headings: (i) sustainability of CSO participation in public procurement monitoring; (ii) absence of a guide in observing public biddings; (iii) availability, quality, and other related concerns of training; (iv) submission and utilization of observers’ reports; and, (v) the legitimacy of “Special Purpose” CSOs involved in public procurement activities, especially at the local level. Most of these recommendations were reflected in the design of MMP2 [e.g., identifying and training volunteers; mobilizing a generally adequate number of volunteers to attend meetings (although there is still room to improve in this area for some hospitals); preparation of procedures, checklists and manuals; and developing procedures to certify the legitimacy of the volunteers through the DOH, NAMFREL MOA and IDs and setting out how their reports are to be submitted and considered]. 85. There is one recommendation in the Country Procurement Review related to the payment of the transportation allowance for volunteers that DOH and NAMFREL should seriously consider. The suggestion was to allocate a percentage of earnings from the sale of bid documents to CSO training. Pending the implementation of this recommendation at the national level by GPPB, DOH could consider implementing it on a pilot basis. That would ensure that funds are available for periodic refresher training and the training for new volunteers. This approach could be extended to allow for a portion of the proceeds of the sale of bid documents to be used to pay for the transport

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!66 DBM Budget Circular 2004-5A, issued on 7 October 2005 and updated through DBM Budget Circular 2007-3, issued on 29 November 2007. 67 See Annex 5. Philippines Country Procurement Assessment Report 2012. Government of the Philippines, Asian Development Bank, Japan International Cooperation Agency and the World Bank. 2013

!

33!

allowances of the NAMFREL volunteers. The latter would be a welcome extension to the current practice in some hospitals of using the proceeds of the sale of bid documents to fund the honorariums of government officials and to pay for the snacks and meals of everyone who attends BAC meetings. 86. NAMFREL is a member of DOH’s Integrity Management Committee, the successor of the IDCs. NAMFREL continues to attend the monthly meetings and DOH stated that NAMFREL makes helpful interventions. However, there is scope for NAMFREL to more effectively use this high level forum to disseminate Project findings, resolve issues that require decisions to be made at the highest levels in DOH [e.g., using the proceeds of the sale of procurement documents to finance the travel costs of volunteers to attend procurement meetings and for future training; finding ways to address collusion and over pricing; ensuring that volunteers are invited to attend all meetings in the procurement cycle (bid evaluation in addition to pre-bid conferences and bid opening) and have access to all necessary documents for post-delivery monitoring; DOH institutes a tracking system to provide feedback to the highest levels on the number of BACs that were actually attended by observers rather than invitations being issued]. The volunteer representatives on hospital IDCs should also make better use of their presence on those committees.

Overall Assessment of Outcomes, Impact and Sustainability 87. The key dimension of the evaluation, to which PTF assigns a 45% weight in the total scoring, is whether the project outcomes had the desired impact on reducing corruption and whether those impacts are expected to be sustainable. The Project Proposal specified benchmarks and specific targets to measure the results achieved by MMP2. Figure 11 sets out the specific targets and actual achievements. Although MMP2 did not meet the benchmarks, the Evaluation Team feels that, with the benefit of hindsight, the targets were overly optimistic, particularly given the results actually achieved under MMP1. Despite the outcomes of MMP1, the MMP2 targets remained the same as for MMP1. MMP2 achievements are impressive when measured against the 2010 benchmark in the Philippine Country Procurement Review as only 19% of DOH’s BAC meetings were monitored and the fact that coverage was about 6 times higher than was achieved under MMP1 for all categories of monitoring. PTF provided grants of $26,296 and $33,350 for MMP1 and MMP2 respectively. There was a marked improvement in effectiveness and efficiency of MMP2 compared to MMP1 using the amount monitored per $1 of PTF grant. Based on an assumed exchange rate of P45 per dollar, the amount of monitoring per dollar of PTF contribution was: (i) procurement: $1,631 for MMP2 vs. $349 for MMP1, a 4.7 times improvement; (ii) delivery: $202 for MMP2 vs. $152 for MMP1 a 1.3 times improvement; and, (iii) inventory: $132 for MMP2 vs. $27 for MMP1 a 4.9 times improvement.

!

34!

Figure 11: Actual Results Compared to the Targets Set Out in the MMP2 Project Proposal Key Result Areas Target Actual Achievement Comments

1. More transparent and fair procurement process.

100% compliance of hospital the provisions of RA 9184, i.e., 100% of BACs monitored by volunteers.

Contracts accounting for 54% of the APPs were monitored (41% in 2010 and 71% in 2011).

There was a shortfall in achieving the 100% target that, with the benefit of hindsight, the Evaluation Team feels was overly optimistic. MMP2 procurement monitoring achievements were impressive compared to the 19% figure achieved overall in 2010 by DOH and the 5.9 times increase in the amount monitored under MMP2 than under MMP1.

2. More efficient and cost-effective delivery of drugs and medicines.

Volunteers monitor 75%-100% of the delivery of drugs and medicines.

Volunteers monitored medicine deliveries equal to 62% of contracts awarded and 39% of the ABC.

There was a shortfall in achieving the 75% to 100% target that, with the benefit of hindsight, the Evaluation Team feels was overly optimistic. MMP2 delivery monitoring was impressive when compared to MMP1 achievements as it was 5.2 times higher.

3. Timely allocation of priority drugs and medicines to hospital beneficiaries.

Volunteers monitor 50%-100% of the inventory reports of priority drugs and medicines.

Volunteers monitored medicine inventories equal to 41% of contracts awarded and 20% of the ABC.

There was a shortfall in achieving the 50% to 100% target that, with the benefit of hindsight, the Evaluation Team feels was overly optimistic. MMP2 delivery monitoring was impressive when compared to MMP1 achievements as it was 6.1 times higher.

Sources: MMP2 Project Proposal, date in NAMFREL’s MMP2 Project Completion Report and the Evaluation Team.

88. Two other expected outputs of the Project were: (i) an increase trained volunteers; and (ii) improved monitoring tools to capture the data necessary to produce comprehensive reports. Based on the interviews with the volunteers and DOH officials and a review of the comprehensive PCR, the Evaluation Team concluded that these outputs were delivered. 89. The final output promised in the MMP2 Project proposal was to undertake a comparative price analysis to produce something similar to a price catalog, where the bid price and the prevailing market price of selected essential drugs and medicines are compared. The plan was to post the comparative prices on NAMFREL’s website and for NAMFREL to update the data every three months. In reviewing the draft Project proposal, PTF commented that these objectives might be overly optimistic, which turned out to be the case. Although the volunteers undertook comparative price surveys and NAMFREL reported on some of the results in the MMP2 PCR, this data had not had, at the time of evaluation, an impact on helping the BAC’s to determine whether or not the bid prices were reasonable. Also, the price data was not posted on NAMFREL’s website nor was NAMFREL able to update it quarterly. Nevertheless, the price survey did document some important problems that need to be addressed: (i) the wide variation in the prices paid by different hospitals for the same drugs; and (ii) the fact that in a few cases some hospitals paid more than the over-the-counter retail price. NAMFREL’s work complements the comprehensive work NCPAM has undertaken during the past two years. Although this evaluation raises questions about the appropriateness of the benchmark set for the DRPI, further work is needed to verify that tentative conclusion. 90. Based on the foregoing, the Evaluation Team rated MMP2 as Highly Satisfactory (4) in terms of achieving outcomes, impacts and sustainability. MMP2 achieved good outcomes, plausibly had some indirect impacts on reducing opportunities for corruption and some of the benefits have been sustained after PTF funding ended. There is evidence that MMP1 and MMP2 contributed to institutionalizing the procurement monitoring process in DOH, that NAMFREL has built a strong partnership with DOH and that NAMFREL’s monitoring is recognized by DOH and the hospitals as something that improves procurement transparency, helps to protect their public

!

35!

reputations and has contributed to the recent widespread perception that DOH is now a well-run Government agency that takes its anti-corruption strategies seriously. Despite falling short of the overly optimistic output targets set in the Project proposal, under MMP2 the volunteers did a significant amount of monitoring throughout the procurement cycle from pre-bidding and bid opening through to medicine delivery and storage and in-hospital use. The volunteers monitored P2.4 billion in procurement, 54% of the APPs, P303 million or 39% of the ABCs for medicine delivery and P198 million or 20% of the ABCs for storage and in-hospital use. MPP2 was more efficient and effective than MPP1. Under MMP2 volunteers monitored three times as many hospitals and six times as much procurement compared to what was achieved under MMP1. NAMFREL’s price survey found that that there are wide, unexplained variations in the prices paid by hospitals for the same medicines. There is evidence that some of the Project benefits are sustainable, although steps must be taken to further strengthen sustainability.

D. Replicability 91. Based on the findings of the Evaluation Team, Replicability was rated as Exceptional or 5. The MMP2 has many elements that can, and should, be replicated in future projects – the major goals, elements and stakeholders are in place and the Project design can be replicated in health sector, both in the Philippines and in other countries. The MMP2 model has proven to be appropriate and practical and generates good outputs. NAMFREL established a good relationship with DOH, a core group of volunteers has been trained, much of the training material can be used again with suitable updating, the volunteers have gained practical experience monitoring the full procurement cycle for medicines and the procurement of hospital infrastructure, equipment and supplies and undertaking drug price surveys. 92. NAMFREL intends to propose a third phase of the Medicine Monitoring Project (MMP3). The three main activities of MMP3 include:

(i) Continuing procurement monitoring in DOH hospitals following the model developed under MPP1 and MMP2. As well as RHs and CHDs, MMP3 is expected to cover the four DOH Central Office Bids and Awards Committees (COBACs).

(ii) NAMFREL plans to expand the coverage of medicine procurement monitoring to selected Local Government Unit (LGU) hospitals/centers to be identified with the help of the Department of Interior and Local Government (DILG) and DOH. CSO procurement monitoring does not presently cover LGU hospitals. The LGU hospitals provide health services and access to medicines in most rural or impoverished areas. A sizeable amount of LGUs funds and funds from the Priority Development Assistance Fund (PDAF)/Disbursement Acceleration Program (DAP) are reportedly allocated by politicians for pharmaceutical and health services. While the Commission on Audit (COA) attempts to audit related expenditures, most are not transparent and not monitored effectively. The Evaluation Team notes that there has been a major scandal in the Philippines in the use of PDAF/DAP that resulted in corruption charges and allegations against many senators and congressmen involving fake NGOs and that there have been press reports of corruption and interference by the mayors in the operations of LGU hospitals. Thus, expanding the MMP model to cover some LGU hospitals would be desirable.

(iii) NAMFREL would engage and support DOH/NCPAM in developing a price matrix and monitoring system for the top 50 or so essential drugs based on the

!

36!

Philippine Drug Formulary. This would be beneficial for consumers and would be a useful tool for BACs to help determine proper pricing and to help prevent price fixing, collusion and manipulation of the pricing of drugs.

93. NAMFREL recognizes that to effectively pursue these objectives, the following requisites must be addressed:

(i) Review and renewal of the NAMFREL/DOH MOA. This review is ongoing and is an item in the regular meetings of the DOH-IMC (Integrity Management Committee, the successor of the IDC) on which NAMFREL sits as a CSO representative. NAMFREL also sits on two IMC Sub-Committees (i.e., Financial, Procurement and Asset Management; External Stakeholders). Comments/inputs are being awaited from the Office of the DOH Secretary and the Office of the Deputy Executive Secretary for Legal Affairs for NAMFREL’s expanded role and partnership. A final version of the MOA is expected in 2014.

(ii) Meeting with DILG officials with the support of DOH officers to revive talks for a NAMFREL partnership and offer medicine/health infrastructure procurement monitoring of selected LGU hospitals. NAMFREL’s objective is to have a MOA signed in 2014.

(iii) New and recurrent training on the Procurement Law, the related RIRRs and its provisions and specific applications for procurement volunteers. Given the nuances of the law encountered as it is applied and an increase demand for new observers as replacements or for new assignments to cover the COBACs, NAMFREL puts on periodic training (the most recent training was conducted on 27/28 June 27 2014). The two-day training seminar was organized in partnership with the DOH, the Bishops Businessmen’s Conference for Human Development and the Junior Chamber International (JCI)-Manila. There were 21 participants in the training from NAMFREL NCR chapters, JCI-Manila and the University of Santo Thomas College of Business Administration and Accountancy. This pool of participants will be screened for proficiency in understanding the Procurement Law and absence of conflicts of interest and invited to be procurement volunteers for the four new COBACs. NAMFREL updated and improved its instruction and training materials for the recent training. NAMFREL is in the process of developing a manual to be reproduced and used as resource material for future training. NAMFREL plans to produce the new manual in 2014 as an update of the exiting manual. If necessary funding can be secured, NAMFREL plans to print 1,000 copies of the manual.

(iv) To improve the capability and capacity of its two MMP project coordinators, NAMFREL will finance the cost of their attending short courses and seminars on project management, evaluation and monitoring; technical writing and health project service with an institution that conducts these courses or with the DOH if DOH has such in-house course.

94. This advanced planning and training are strong indicators that NAMFREL is committed to continuing to work in the medicine procurement area and a strong indicator of the Replicability of the Project design. The relatively wide dissemination of information on MMP1 and MMP2 in international forums and Internet publications may promote Project Replicability in the health sectors in other countries. IV. CONCLUSIONS

A. Overall Assessment

!

37!

Following PTF’s rating and scoring methodology, and using the weights suggested by PTF, the Evaluation Team concluded that MMP2 was very successful and rated it as Highly Satisfactory (see Figure 12). The Evaluation Team rated the Project as Highly Satisfactory for three of four of PTF’s prescribed dimensions of evaluation -- Approach and Project Design; Project Implementation and Outcomes, Impact and Sustainability. The Evaluation Team rated the fourth dimension of evaluation, Replicability, as Exceptional.

Figure 12: Overall Assessment of MMP2 Category Value Raw Score Weighted

Score Approach and Project Design

15% 4 0.60

Project Implementation 20% 4 0.80 Outcomes, Impact and Sustainability

45% 4 1.80

Replicability 20% 5 1.00 Overall Score 100% 4.20 Source: Evaluation Team

95. The Evaluation Team’s very favorable rating of MMP2 is consistent with an independent assessment of the Project that was undertaken for the United Kingdom’s Department for International Development (DFID) 68 . That assessment examined MMP2’s: (i) results statement; (ii) context and theory of change; (iii) approaches, methods and tools; (iv) implementation experience; and (v) long-term Impact on people’s lives. All of these factors were assessed positively. When commenting on the value for money, that study concluded that P1.5 million were spent on monitoring P1.4 billion of procurement (i.e., a ratio of approximately 0.1%). The DFID assessment stated “While no data was available on the level of corruption, typically drug procurement has a significant level of corruption, most likely well in excess of 5% of the total value of drugs purchased. It is evident that this project had a very high rate of return…well in excess of 100% and maybe 10 or 20 times greater.” That DFID assessment concluded that NAMFREL provided strong evidence, assessed as being excellent, that results were achieved. MMP2 was awarded perfect scores for all four groups of factors that were assessed in terms of the strength of evidence: (i) appropriateness (i.e., the methods that were justifiable given the nature of the intervention and purpose of the assessment); (ii) triangulation (i.e., conclusions about the intervention’s effects were judged by using a mix of methods, data sources, and perspectives); (iii) contribution (i.e., how change happened can be shown and it can be explained how the Project contributed to the change); and (iv) transparency (i.e., the assessment was open about the data sources and methods used, the results achieved, and the strengths and limitations of the evidence). Consistent with the approach adopted for this evaluation, DFID’s assessment

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!68 Most Significant Result Analysis: NAMFREL, Philippines. MMP2 was one of 10 PTF projects that DFID selected for assessment and one of four PTF-supported projects that were show cased on 31 March 2014 in a DFID seminar on Most Significant Results. PTF was the only organization out of 20 NGOs to have 4 projects selected for show casing. Fifteen NGOs had one project selected by DFID, 4 NGOs had two selected and 18 NGOs did not have any projects selected.

!

38!

did not attempt to gain feedback from the final beneficiaries (i.e., the patients) views on the effects of the Project or to identify who has been affected and how.

B. Main Lessons and Recommendations 1. Lesson and Recommendation for PTF • Consistent engagement with partners in two or more projects over a decade or so promotes the efficient and effective delivery of results and sustainability of benefits: MMP2 was more effective and efficient than MMP1 in monitoring procurement at all stages of the procurement cycle. That reflected the experience gained by NAMFREL, DOH, the hospitals and the volunteers over a period of years. Engagement with the same partners over the course of two or more projects helps to build strong, constructive relationships between a CSO and its government counterpart. In selecting future projects, other things being equal, PTF should give priority to funding follow on projects for NGOs that have demonstrated an ability to achieve results.

2. Lessons and Recommendations for NAMFREL • Improving some details in the procurement monitoring: In addition to providing sufficient flexibility to deal with scheduling conflicts and short notices of meetings by mobilizing teams of four or more volunteers for each hospital, NAMFREL should engage in dialogue with DOH to ensure that: (i) volunteers are invited to attend the bid evaluation meetings undertaken by the Technical Working Groups; and (ii) all documents needed for post-delivery monitoring of medicines and supplies and their inventory and use in hospitals are complete and readily available to the volunteers. These issues should be covered in an updated MOA. • Promote sustainability by addressing the need to pay the travel allowances of volunteers after donor-funded projects end: Monitoring by the volunteers declined after the PTF funding ended because money was not available to pay for their transportation allowances, thus undermining the sustainability of Project benefits. NAMFREL should engage with DOH to see if part of the proceeds of the sale of procurement documents could be used to finance such costs. If so, this issue should be covered in an updated MOA. • Provide more coverage of collusion and bid rigging in the training: Collusion is difficult to detect, although the wide unexplained variations in the prices paid by hospitals for the same drugs suggest that it is probably occurring. NAMFREL should provide more coverage of collusion and bid rigging in its training courses. • Explore whether advances in mobile technology could usefully deployed for future projects: Advances in mobile technology and related platforms may make it possible for NAMFREL to more efficiently collect and analyze data from the volunteers and for the volunteers to submit their reports. • Discuss with DOH institutionalizing a DOH-wide monitoring system for the number of BACs that are actually attended by volunteers: Discuss with DOH the possibility of developing a system so that senior managers are aware of the number of procurement meetings actually monitored by CSO observers.

!

39!

• Assess in a future project whether the DRPI estimated by DOH is an appropriate benchmark: NCPAM has done excellent work to establish a comprehensive database to monitor the prices paid by all hospitals for about 600 medicines and to set DRPIs to give guidance to BACs. More analysis should be undertaken to ensure that the levels of the DRPIs are appropriate for all drugs relative to international prices and are well below the retail price for the drugs.

!

40!

ANNEX 1: LOCATION OF REGIONS IN THE PHILIPPINES

Baguio and the Cordillera Autonomous Region

Region 3 That Includes Nueva Ecija and Cabanatuan

Regions in the Philippines

!

41!

ANNEX 2: PHOTOS OF NAMFREL MEDICINE MONITORING PROJECT

The Evaluation Team meeting with DOH Assistant Secretary

Paulyn Jean B. Rosell-Ubial and NCPAM Officers

Meeting with DOH Assistant Secretary Gerardo V. Bayugo and

DOH and NCPAM Officers

MMP2 Evaluation Kick-off meeting with the NAMFREL Secretary General,

one of the NAMFREL Council Members and one of the Project Coordinators

!

42!

National Capital Region volunteers and a Project Coordinator at

the NAMFREL Secretariat

MMP2 Volunteer ID Card

!

43!

Meeting at the Research Institute for Tropical Medicine in Alabang,

Metro Manila

Secure drug storage facilities at the Research Institute for Tropical Medicine

Meeting with the BAC Secretariat staff at the Children’s Medical Hospital In Quezon City

!

44!

Meeting with the NAMFREL Volunteers in

Cabanatuan City, Nueva Ecija

Meeting with the Medical Director and officers at the Cabanatuan General Hospital

The Pharmacist and her staff at Cabanatuan General Hospital

medicine storage facility

!

45!

NAMFREL volunteers in Baguio

BAC secretariat and pharmacy staff at the Baguio General Hospital

!

46!

Inspecting the medicine storage facilities at the Baguio General Hospital

!

47!

ANNEX 3: PROCUREMENT PRICES FOR SELECTED DRUGS Exhibit 3.A: Procurement Prices for Captopril

V. Note: The analysis was based on 25 mg tablets. There were 570 awarded bid prices in the database, which are represented by points from lowest to highest. Outliers were removed for analysis (Las Nieves Municipal Hospital - Php 110; Litz Pharmacy - Php 115; Loreto Municipal Hospital Pharmacy - Php 115; Masawa Bay Pharma And Medical Supplies - Php 122; Mckline Enterprises, J.C. Aquino Ave., Butuan City -Php 150; Mom's Option Pharmacy & General Merchandise - Php 180, 180 and 200; MS Quality Drug (Php 220); Nasipit District Hospital - Php 1200; New Family Drug - Php 3000). The median price of the data set is Php 3.15, and the retail price based on the MMP2 survey is Php 8.0. The prices in the graph ranged from Php 0.25 to 55. Source: Derived from NCPAM’s online database by the Evaluation Team Exhibit 3.B: Procurement Prices for Celecoxib

!Note: Note: The analysis was based on 200 mg capsules. There were 1092 awarded bid prices in the database, which are represented by points from lowest to highest. The median price of the data set is Php 44.4, and the retail price based on the MMP2 survey is Php 18. The prices ranged from Php 0.01 to 1092. Source: Derived from NCPAM’s online database by the Evaluation Team ! !

0

10

20

30

40

50

60

Pric

e pe

r 25

mg

tabl

et (P

hp)

Median

Retail

0

50

100

150

200

250

300

Pric

e pe

r 200

mg

caps

ule

(Php

)

Median

Retail

!

48!

Exhibit 3.C: Procurement Prices for Ciprofloxacin

!Note: The analysis was based on 250 mg tablets. There were 85 awarded bid prices in the database, which are represented by points from lowest to highest. Outliers were removed for analysis (Rodamel Drugstore Branch II - Php 915.95 and 942; Rodamel Drugstore Branch III - Php 1091, 1559,05, 1750 and 1800). The median price of the data set is Php 35, and the retail price based on the MMP2 survey is Php 16.25. The prices ranged from Php 2.0 to 72.55. Source: Derived from NCPAM’s online database by the Evaluation Team Exhibit 3.D: Procurement Prices for Clarithromycin

!Note: The analysis was based on 500 mg tablets. There were 101 awarded bid prices in the database, which are represented by points from lowest to highest. The median price of the data set is Php 113.95, and the retail price based on the MMP2 survey is Php 58.25. The prices ranged from Php 14.5 to 910. Source: Derived from NCPAM’s online database by the Evaluation Team ! !

0

10

20

30

40

50

60

70

80 Pr

ice

per 2

50 m

g ta

blet

(Php

)

Median

Retail

0

100

200

300

400

500

600

700

800

900

1000

Pric

e pe

r 500

mg

tabl

et (P

hp)

Median

Retail

!

49!

Exhibit 3.E: Procurement Prices for Digoxin

!Note: The analysis was based on 250 mcg/ml ampules. There were 48 awarded bid prices in the database, which are represented by points from lowest to highest. The median price of the data set is Php 193.7, and the retail price based on the MMP2 survey is Php 178.5. The prices ranged from Php 109 to 540. Source: Derived from NCPAM’s online database by the Evaluation Team Exhibit 3.F: Procurement Prices for Felodipine

!Note: The analysis was based on 10 mg capsules. There were 291 awarded bid prices in the database, which are represented by points from lowest to highest. The median price of the data set is Php 25. The prices ranged from Php 13.75 to 93.25. No retail price was available for comparison. Source: Derived from NCPAM’s online database by the Evaluation Team ! !

0

100

200

300

400

500

600 Pr

ice

per 2

ml a

mpu

le, 2

50m

cg/m

l (P

hp) Median

Retail

0

10

20

30

40

50

60

70

80

90

100

Pric

e pe

r 10

mg

caps

ule

(Php

)

Median

!

50!

Exhibit 3.G: Procurement Prices for Metformin

!Note: The analysis was based on 250 mg/ 5ml suspensions. There were 1543 bid prices in the database, which are represented by points from lowest to highest. Outliers were removed for analysis (Cherry Mar Drug – Mardrid Branch – Php 327.13; Choleen Liz Pharmacy – Php 332.99, 335 and 377.89; CKAT Pharmacy – Php 437.01 and 524.26; Clavarian Pharmacy – Php 560; D.O. Plaza Memorial Hospital – Php 564; DAFS Pharmacy – Php 600; Danric Drugstore – Php 665.9; Deking’s Pharmacy – Php 665.9 and 738.56; Direct Access Pharma – Php 738.56; Diwata Pharmacy – Php 941; Dkings Pharmacy I – Php 1120.94 and 2257.3). The median price of the data set is Php 7.64. The prices ranged from Php 0.01 to 1537. No retail price was available for comparison. Source: Derived from NCPAM’s online database by the Evaluation Team Exhibit 3.H: Procurement Prices for Paracetamol

!Note: The analysis was based on 250 mg/ 5ml suspensions. There were 2643 bid prices in the database, which are represented by points from lowest to highest. The median price of the data set is Php 96. The prices ranged from Php 0.01 to 300. No retail price was available for comparison. Source: Derived from NCPAM’s online database by the Evaluation Team

0

20

40

60

80

100

120 Pr

ice

per 5

00 m

g ta

blet

Median

0

50

100

150

200

250

300

350

Pric

e pe

r 60

ml b

ottle

, 250

mg/

5 m

l su

spen

sion

(Php

)

Median

!

51!

!

Exhibit 3.I: Procurement Prices for Spironolactone

!Note: The analysis was based on 25 mg tablets. There were 291 awarded bid prices in the database, which are represented by points from lowest to highest. Outliers were removed for analysis (Palawan Adventist Hospital - Php 1534.3; PE's Drugstore - Php 2501.07, 3000 and 5190.22). The median price of the data set is Php 23, and the retail price based on the MMP2 survey is Php 26. The prices ranged from Php 11 to 70. Source: Derived from NCPAM’s online database by the Evaluation Team !

0

10

20

30

40

50

60

70

80

90

100

Pric

e pe

r 25

mg

tabl

et (P

hp)

Median

Retail

!

52!

ANNEX 4: BUDGET AND EXPENDITURES FOR MMP2 (In Pesos)

Annex 4: Budget and Expenditures for MMP2 Item Approved Budget Actual Expenditures

September 2010 to October 20111/

Amount % Amount % Overhead and In Kind Contributions Project management costs (e.g., salaries for two Project Coordinators; honorariums for speakers during training; per diems for volunteers during consultation meetings)

312,000 10.3 554,445 18.2

In kind value of the provision of office space2/ 216,600 7.1 216,600 7.1 In kind estimate of the value of the volunteer’s time2/

1,312,192 43.3 1,312,192 43.2

Sub-total Overhead and In Kind Contributions 1,840,792 60.8 2,083,237 68.5 Direct Project Expenses Travel/Transportation (e.g., travel for volunteers to attend training; volunteers attending BAC/IDC meetings; NAMFREL Secretariat staff meetings with hospitals and volunteers in local areas; project evaluation)

734,700 24.3 403,083 13.3

Meals and Accommodation (e.g., travel for volunteers to attend training; volunteers attending BAC/IDC meetings; NAMFREL Secretariat staff meetings with hospitals and volunteers in local areas; two quarterly team meeting; project evaluation)

346,900 11.4 428,064 14.1

Printing, Communications and Others 106,400 3.5 125,504 4.1 Sub-total Direct Project Expenses 1,188,000 39.2 956,651 31.5 Grand Total 3,028,792 100.0 3,039,888 100.0 Notes: 1 = Neither NAMFREL nor Evaluation Team had access to figures for MMP2 costs incurred after 31 October 2011. While there were some expenses, the large majority of the MMP2 costs were incurred between 1 September 2010 and 31 October 2011. 2 = The value of the in-kind contributions for the provisions of office space and the value of volunteers’ time was not re-estimated in the Project Completion Report so the same estimate was used as was in shown in the Project Proposal. Sources: MMP2 Project Proposal and NAMFREL’s Project Completion Report

!

53!

ANNEX 5: A CHECK LIST FOR POSSIBLE COLLUSION

Source: State of Texas Attorney General

!

54!

ANNEX 6: PROCUREMENT RATINGS OF THE DEPARTMENT OF HEALTH RELATIVE TO OTHER GOVERNMENT

AGENCIES

Annex 6: Procurement Ratings of DOH Relative to Other Government Agencies1/ Pillar I: Compliance with Legislative and Regulatory Framework

Indicator 1: Competitive Bidding as Default Procurement Method

Average Score

DOH Score2/

A. % of public bidding contracts in terms of value of total procurement 73% 51% B. % of public bidding contracts in terms of volume of total procurement 16% 41% Indicator 2: Alternative Methods of Procurement A. % of alternative modes of contracts in terms of value of total procurement 27% 49% B. % of Shopping contracts in terms of value of total procurement 10% 1% C. % of Negotiated Procurement in terms of value of total procurement 15% 37% D. % of Direct Contracting in terms of value of total procurement 4% 11% E. % of Repeat Order contracts in terms of value of total procurement 1% 0.03% F. % of Limited Source contracts in terms of value of total procurement na na Indicator 3: Competitiveness of the Bidding Process A. Average number of bidders who acquired bidding documents 7 6 B. Average number of bidders who submitted bids 5 4 C. Average number of bidders who passed bid evaluation 3 1 Pillar II: Agency Institutional Framework and Management Capacity Indicator 4: Presence of Procurement Organizations A. Creation and operation of Bids and Awards Committee(s) or BAC(s) 100%

Compliant Compliant

B. Creation and operation of a BAC Secretariat or Procurement Unit 100% Compliant

Compliant

Indicator 5: Procurement Planning and Implementation A. APP is prepared for all types of procurement 82%

Compliant Compliant

Indicator 6: Use of Government Electronic Procurement System A. Agency registered with Phil-GEPS 100%

Compliant Compliant

B. % of bid opportunities posted at Phil-GEPs 100% 100%

C. % of contract award information posted at Phil-GEPs 53% 100%

D. % of contract awards procured through alternative methods posted in PhilGEPS 35% 58%

Indicator 7: System for Disseminating and Monitoring Procurement Information A. Presence of website that provides minimum, up-to-date procurement information easily accessible at no cost

94% Compliant

Compliant

B. Preparation of Procurement Monitoring Reports and submission to GPPB 41% Compliant C. Posting of Procurement Monitoring Report in agency website 18% Compliant Notes: 1 = Based on a summary of 2010 the Agency Procurement Compliance and Performance Indicators 2 = Average score of 17 agencies Source: Derived from Annex 5 in Philippines Country Procurement Assessment Report 2012. Government of the Philippines, Asian Development Bank, Japan International Cooperation Agency and the World Bank. 2013

!

55!

Annex 6: Procurement Ratings of DOH Relative to Other Agencies1/ (continued) Pillar III: Procurement Operations and Market Practices Indicator 8: Efficiency of Procurement Processes

Average Score

DOH Score2/

A. % of total value of procurement against total value of approved APPs 50% 81% B. % of failed biddings and total number of procurement activities conducted 13% 13% Indicator 9: Compliance with Procurement Timeframes A. Average number of days to procure goods 101 117 B. Average number of days to procure infrastructure projects with ABC of P50 million and below

121 162

C. Average number of days to procure infrastructure projects with ABC above P50 million 126 56 D. Average number of days to procure consulting services 154 143 Indicator 10: Timely Payment of Procurement Contracts A. Payments are released upon submission of complete and required documents as provided for in the contract

35 30

Indicator 11: Capacity Building for Government Personnel and Private Sector Participants

A. There is a system within the agency to evaluate the performance of procurement personnel

59% Compliant

Compliant

B. % of participation of procurement staff in annual procurement training 73% trained

100%

C. Agency has activities to inform and update bidders on public procurement 41% of pilot

agencies

Compliant

Indicator 12: Management of Procurement and Contract Management Records A. The BAC Secretariat has a system for keeping and maintaining procurement records 76% FC;

18% SC; 6% PC

Fully Compliant

B. Implementing Unit has and is implementing a system for keeping and maintaining contract management records

59% FC; 29% SC; 12% NC

Fully Compliant

Indicator 13: Contract Management Procedures A. Agency has well defined and written procedures for quality control, acceptance and inspection of goods, works and services

82% Compliant

Compliant

B. Supervision of civil works is carried out by qualified construction supervisors (applicable for works only)

85% Compliant

Compliant

C. Agency implements CPES for its works projects and uses results to check contractors qualifications (applicable for works only)

62% Compliant

Compliant

Notes: 1= Based on a summary of 2010 the Agency Procurement Compliance and Performance Indicators 2 = Average score of 17 agencies Source: Derived from Annex 5 in Philippines Country Procurement Assessment Report 2012. Government of the Philippines, Asian Development Bank, Japan International Cooperation Agency and the World Bank. 2013

!

56!

Annex 6: Procurement Ratings of DOH Relative to Other Agencies1/ (continued) Pillar IV: Integrity and Transparency of the Agency Procurement System Indicator 14: Civil Society Participation in Public Bidding Average

Score DOH

Score2/

A. Civil society organizations or professional associations are invited for every public bidding activity

94% Compliant

Compliant

B. % Percentage of civil society and/or professional organizations' attendance in public bidding activities

35% 19%

Indicator 15. Internal and External Audit of Procurement Activities A. Creation and operation of internal audit unit as prescribed by DBM (Circular Letter No. 2008-5, April14, 2008)

94% Compliant

Compliant

B. Conduct of regular audit of procurement processes and transactions by internal audit unit

69% Compliant

Compliant

C. Internal audit recommendations on procurement related matters are implemented within 6 months of the submission of the auditor's report

69% Compliant

Compliant

D. Agency Action on Prior Year's Audit Recommendations (APYAR) on procurement related transactions

80% Compliant

Compliant

Indicator 16. Capacity to Handle Procurement Related Complaints A. The BAC resolved Motion for Reconsiderations, Protests and Complaints within seven (7) calendar days as per Section 55 of the IRR

87% Compliant,

8 CDs Average

Compliant Within 7 CDs

B. All decisions on MRs/complaints/protest are submitted to GPPB, and dispositive portion are publicly posted in the agency and GPPB websites

21% Compliant

na

Indicator 17. Anti-Corruption Programs Related to Procurement A. Agency has specific anti-corruption program related to integrity development (e.g. IDAP or IDR)

88% Compliant IADP

B. Agency has specific policies and procedures in place for detection and prevention of corruption associated with procurement.

63% Compliant

Notes: 1 = Based on a summary of 2010 the Agency Procurement Compliance and Performance Indicators 2 = Average score of 17 agencies Source: Derived from Annex 5 in Philippines Country Procurement Assessment Report 2012. Government of the Philippines, Asian Development Bank, Japan International Cooperation Agency and the World Bank. 2013