Rapid Assessment of Quality and Field Performance of HIV Rapid Diagnostic Tests in Three Countries A Report to the Bill & Melinda Gates Foundation Shirley Villadiego, MD, MPH

Jeanette Lim, MPH, RN

Arthur Lee, BA

Roger Peck, BSc

April 13, 2012 MAIL ING ADDRESS PO Box 900922 Seattle, WA 98109 USA ADDRESS 2201 Westlake Avenue Suite 200 Seattle, WA, USA TEL: 206.285.3500 FAX: 206.285.6619 www.path.org

Rapid Assessment of Quality and Field Performance of HIV RDT in Three

Countries in Sub-Saharan Africa

Table of Contents 1. Executive Summary ................................................................................................................. 2

2. Introduction ............................................................................................................................. 3

3. Methodology ............................................................................................................................ 4

3.1 Literature search .................................................................................................................... 4

3.2 Global stakeholder interviews ............................................................................................... 4

3.3 Field assessment of HIV RDT performance and quality ...................................................... 5

4. Results ..................................................................................................................................... 5

4.1 Literature search ............................................................................................................... 5

4.2 Global stakeholders interviews ........................................................................................ 6

4.2.1 Factors influencing quality and field performance of HIV RDT ................................... 7

4.2.2 Perception of self testing ................................................................................................ 8

4.3 Field assessment of HIV RDT performance and quality ................................................. 8

4.3.1 National policies ............................................................................................................. 8

4.3.2 National algorithm .......................................................................................................... 9

4.3.3 Who conducts HIV testing ........................................................................................... 10

4.3.4 Training and supervision .............................................................................................. 10

4.3.5 Perception of current tests ............................................................................................ 11

4.3.6 Procurement, supply, storage, and distribution of RDTs ............................................. 11

4.3.6 Quality assurance and quality control .......................................................................... 13

4.3.7 User errors .................................................................................................................... 15

4.3.8 Staffing and capacity .................................................................................................... 16

4.3.9 Perception of home-based and self testing ................................................................... 16

5. Opportunities for Improvements in HIV testing .................................................................... 17

6. Conclusions and Recommendations ...................................................................................... 18

7. References ............................................................................................................................. 20

2

1. Executive Summary Efforts to scale up prevention and treatment programs for HIV have occurred in the past several years in areas with high endemicity such as sub-Saharan Africa. However, these efforts of scaling up prevention strategies have been hampered by numerous field challenges.

The rapid expansion of rapid diagnostic tests (RDTs) has outpaced quality assurance (QA) programs to monitor performance, raising concerns about accuracy. Considering the impact of misrepresented results, there is a clear and urgent need for establishing strong QA systems, especially in resource-constrained countries where the use of HIV RDTs is recommended for diagnosis of infection.

PATH, with funds from the Bill & Melinda Gates Foundation, conducted a rapid assessment to collect relevant information regarding the landscape of field performance of HIV RDTs and to evaluate the perception of self testing as an option for increasing access to testing in under-served populations. The assessment was divided into three components:

1. A literature search to identify knowledge gaps regarding field performance of HIV RDTs.

2. Interviews with global stakeholders from representative organizations working on HIV RDT quality.

3. Field assessment of HIV RDT performance and quality with national and regional directors of HIV control, HIV coordinators, ministry of health staff, users, and clinic-level supervisors.

According to the literature, there appears to be limited published work on the evaluation of field performance of HIV RDTs. Published work has focused mainly on post-manufacturing test performance, specifically individual test or tester issues, and not necessarily on issues affecting manufacture, procurement, supply, and test quality.

Field evaluations have reported lower specificity due to false positive results with weak positive bands. Another study found that RDTs performed very well in lab evaluations in South Africa conducted by skilled lab technicians (100% sensitivity) and slightly less well in field testing by nurses and counselors (92.5-97.3% sensitivity). Authors attributed this to user errors including lack of on-site supervision, non-adherence to manufacturer instructions, and lack of quality control (QC) management. In addition, other studies reported that challenges included heavy workload, perceived lack of training, stress, stock-outs, staff shortages, language barriers, loss-to-follow-up, and inadequate referral and support services.

Global stakeholders had varying opinions about which issues are most significant, often reflecting their areas of expertise or emphasis. The magnitude and extent to which these issues affect quality and field performance is still a significant gap in our knowledge, calling for additional evaluation. They identified three areas that can influence the quality and field performance of HIV RDTs: human factors, technology-related factors, and health-systems factors.

In regards to self testing, global stakeholders expressed opinions that varied widely from whole-heartedly endorsing self testing to having great reservations about promoting self testing.

Field assessments were conducted in Kenya, Malawi, and Uganda. All three countries reported having national HIV policies. National guidelines regarding HIV testing include detection,

3

prevention, testing, QA, and treatment. There is one HIV testing algorithm for each country which is outlined in each country’s national guidelines. Considerations for selection of tests for the algorithm include sensitivity, specificity, or a ratio of both (for tie-breaker); testing time; shelf life; storage temperature; packing; labeling; avidity; number of runs per kit; and cost.

Countries identified various issues that affect field performance and quality of HIV RDTs that are defined in depth in this document. There was consistency in the responses between global stakeholders and in-country stakeholders regarding issues affecting the quality and performance of HIV RDTs.

Based on the findings of this rapid assessment, we are able to define three different domains that directly or indirectly influence the quality and field performance of HIV RDTs—human, technology, and health systems. These domains are closely linked, and an imbalance in one may affect the other two. Improvements in HIV testing will necessarily require a closer look at a multitude of issues to arrive at a comprehensive solution that can address key challenges.

With the impetus of addressing the three main domains that affect quality and field performance of HIV RDTs, we detail specific recommendations for improving HIV RDT field performance and HIV self testing.

2. Introduction Efforts to scale up prevention and treatment programs for HIV have occurred in the past several years in areas with high endemicity such as sub-Saharan Africa. Although an estimated four million people are infected and receiving antiretroviral therapy (ART), many more will need counseling and testing to achieve the goal of universal treatment and ultimately stop transmission of the disease. Testing programs utilizing HIV RDTs have revolutionized the field by increasing access to fast testing, moving testing from laboratories into community settings.

However, these efforts of scaling up prevention strategies have been hampered by numerous field challenges. The rapid expansion of RDTs has outpaced quality assurance (QA) programs to monitor performance, raising concerns about accuracy.1 Previous evidence suggests that RDTs may underperform in the field, failing to detect a substantial number of infections.2

Independently of the setting or the person who conducts the test, the accuracy and reliability of the diagnostics must be maintained for the success of HIV/AIDS programs. To date, more than 86 million people have received counseling and testing for HIV. Even a small error can result in a large number of misdiagnosed cases. For example, a 1% error rate in testing 1 million people could result in misdiagnosing 10,000 cases.

There have also been reports of challenges related to test quality, test selection, testing algorithms, quality of new lots, and post-market surveillance.

3

PATH, with funds from the Bill & Melinda Gates Foundation, conducted an assessment to collect relevant information regarding the landscape of field performance of HIV RDTs and to evaluate the perception of self testing as an option for increasing access to testing in underserved populations.

Considering the impact of misrepresented results, there is a clear and urgent need for establishing strong QA systems, especially in resource-constrained countries where the use of HIV RDTs is recommended for diagnosis of infection.

4

3. Methodology The primary purpose of the assessment was to collect relevant information regarding field performance of HIV RDTs. The assessment was divided into three components:

1. A literature search to identify knowledge gaps regarding field performance of HIV RDTs.

2. Interviews with global stakeholders from representative organizations working on HIV RDT quality.

3. Field assessment of HIV RDT performance and quality with national and regional directors of HIV control, HIV coordinators, ministry of health staff, users, and clinic-level supervisors.

Interviews with global stakeholders were conducted October through December 2011, and field assessment was conducted in January and February 2012. The specific objectives of the assessment were:

• To identify root causes of HIV RDT failure in the field.

• To elucidate approaches to improve test performance in countries with a high HIV burden.

• To evaluate the perception of self testing for HIV for vulnerable and under-served populations

3.1 Literature search We initially conducted a search on PubMed using the following keywords: HIV, Africa, quality assurance, quality control, manufacturing, rapid diagnostic test, performance, procurement, supply chain, training, failure(s), self testing, home based testing, implementation, barriers, and challenges.

The search produced a list of 609 articles. An initial reviewer read the abstracts from this list and culled 142 articles. A preponderance of articles was related to stigma and attitudes about testing and attitudes about HIV. The initial reviewer selected a few articles on these topics but focused on articles addressing test failures, manufacturing issues, biases, RDTs that show high potential in HIV diagnosis, laboratory-based issues (e.g., quality, infrastructure, reagents), and RDT quality issues.

The 142 abstracts were forwarded to secondary reviewers. From this list, 22 articles were extracted. Ten additional articles referenced in interviews were reviewed and included in the list of literature relevant to the project. See Appendix I for the list of articles.

In addition to articles, we also accessed reports, presentations, and white papers referred to us by global and in-country interviewees.

3.2 Global stakeholder interviews One of our initial activities was to interview global-level stakeholders involved in HIV control. See Appendix II for the list of interviewees. Stakeholders were identified through convenience snowball sampling. Starting with initial contacts, we then asked interviewees to provide further contacts to interview. We repeated this until the referrals became redundant. Stakeholders

5

interviewed varied from academic research to policymakers and implementers and include organizations such as the World Health Organization (WHO), Clinton Health Access Initiative, Médicins sans Frontièrs, the London School of Tropical Medicine, US Centers for Disease and Prevention (CDC), Pangaea, and independent consultants. Interviews with global stakeholders also used a semi-structured qualitative approach, and were conducted by phone. Based on our knowledge from literature, other related work, and hypotheses developed, we created separate interview guides for global stakeholders, in-country interviewees, and interviewees involved in procurement and supply. See Appendix III for semi-structured interview guides.

3.3 Field assessment of HIV RDT performance and quality For the field assessment, PATH employed a mixed-method qualitative approach to gather data on:

1. Local procurement, logistics, and supply and distribution systems for HIV RDT and necessary consumables.

2. Documentation of implementation of local external training and QA programs including results of external quality assurance (EQA) when available.

3. Observational data collection and group and individual dialogue with representative user groups on adherence to national RDT guidelines and applicability of guidelines in the field.

4. Consultation with clinic/hospital-level supervisors and national and sub-national reference laboratory staff to determine key areas of difficulty and opportunities for rapid improvement including technology, systems (e.g., human resources, supply, and QA programs), and/or policy solutions.

The scope of work was limited to RDT performance in Africa. Countries that participated were selected based on burden of HIV and rapid test performance. Kenya and South Africa were chosen as examples of countries with a high burden of HIV and high rapid test performance, while Uganda and Malawi were chosen as examples of countries with a high burden of HIV and low rapid test performance. Note: South Africa was originally included, but had to withdraw because the time required for ethics approval there went beyond the estimated timeline of this project.

We received a non-research determination from its Research Ethics Committee to conduct this assessment. We also obtained ethics approval from the Kenya Medical Research Institute. Malawi and Uganda did not require any local ethics approvals.

4. Results 4.1 Literature search From the literature review we conducted, there appears to be limited published work on the evaluation of field performance of HIV RDTs. Published work has focused mainly on post-manufacturing test performance, and specifically individual test or tester issues and not necessarily on issues affecting manufacture, procurement, supply, and test quality.

6

The literature suggests that RDTs provide reliable results in Africa. QA is reported not to be conducted regularly, but QA is increasing, especially in countries that are supported by external resources.4

A study reported in Uganda with thousands of samples (personal communication, 2012) and another reported by a Kenyan interviewee (personal communication, 2012) compared RDT performance with ELISA, Western Blot or PCR. Both studies reported a high concordance of RDT performance with other methods (more than 99%). We have not been able to obtain the reports or publications for these studies, and assume they evaluated RDT performance in laboratory-controlled settings. These studies suggest that RDT quality is high, and lower test performance may be attributable to user errors.

A field evaluation in Uganda by Kagulire et al. (2011) pointed to lowered specificity due to false positive results with weak positive bands.5 Shott et al. also reports lowered specificity when weak positives are included in analysis.6 Moodley et al. (2008) found that RDTs performed very well in South Africa laboratory evaluations by skilled lab technicians (100% sensitivity) and slightly less well in field testing by nurses and counselors (92.5% to 97.3% sensitivity), which authors attributed to user errors including lack of on-site supervision, non-adherence to manufacturer instructions, and lack of quality control (QC) management.7

Parekh et al. (2010) outline strategies for implementing QA from manufacture to results dissemination. They point out that the rapid expansion of RDT use has outpaced implementation of QA programs.

In addition, Medley and Kennedy conducted semi-structured qualitative interviews with voluntary counseling and testing (VCT) counselors. Challenges reported included heavy workload, perceived lack of training, stress, stock-outs, staff shortages, language barriers, loss to follow-up, and inadequate referral and support services.

3

A WHO literature review (2011) examines key issues of self testing among health workers in sub-Saharan Africa. This paper also proposes a continuum of approaches for expanding self testing and accompanying services.

4.2 Global stakeholders interviews Information gathered from global stakeholders served to focus and augment our country-level information gathering.

Respondents had varying opinions about which issues are most significant, often reflecting their areas of expertise or emphasis. The magnitude and extent to which these issues affect quality and field performance is still a significant gap in our knowledge calling for additional evaluation.

Box 1. Factors that Influence Quality and Field Performance of

HIV RDT According to Global Stakeholders

Human Factors: • User errors such as: time to read

the test, use of proper reagents, poor sample collection and loading, subjectivity in reading.

• Poor training and certification of operators.

• Poor supervision.

Technology-related factors: • Stability of test. • Temperature and storage

requirements. • Cross reactivity. • Sensitivity, specificity. • Standardization of

manufacturing.

Health Systems Factors: • Budget allocation for Q/A

activities. • High demand for services. • Lack of human resources. • Poor reinforcement of policies. • Procurement and selection.

7

Global stakeholders identified three areas that can influence the quality and field performance of HIV RDT.

4.2.1 Factors influencing quality and field performance of HIV RDT Human factors Human error was cited by interviewees as the potential cause of up to 80% of HIV RDT failure. Potential for error was pointed to at every step of testing from sample collection to dissemination of results. Almost all interviewees mentioned incubation time as having a high potential for error i.e., reading results too quickly or waiting too long. Other examples included substituting incorrect buffer solutions in the case of stock-outs.

Lack of certified training and noncompliance to test protocols were also frequently cited as challenges, which in turn are influenced by system factors such as overburdened staff and high staff turnover as well as lack of policies and guidelines and enforcement thereof.

Technology-related factors Many interviewees expressed confidence in the robust technology of current tests when conducted following standard operating procedures, yet also pointed out many unanswered questions. While extensive testing and validation is done on the storage of tests at specific temperatures and conditions, manufacturers are not required to demonstrate stability of tests in temperature cycling conditions or temperature variability over time, which is consistent with actual device storage in many settings within Africa. Manufacturers also comply with temperature requirements at controlled room temperature, which is 20°C to 25°C. However, ambient temperature in sub-Saharan Africa frequently and routinely goes above the recommended storage temperature for the tests (Jorgensen et al., 2006,8

The impact of cross-reactivity among different test populations is another area that is beginning to be evaluated. This has been observed in some cases where the initial test was positive, then all other factors for failure were ruled out and the confirmatory test was negative. One hypothesis is that some proteins cross-react and affect test results (Kosack, C., oral communication 2011).

PATH unpublished data, 2012).

The manufacturing process and standards can impact performance of HIV RDT in the field. Public sector purchasing of HIV rapid test kits is often done with donor money requiring QA (US Food and Drug Administration, United States Agency for International Development, WHO, or other stringent regulatory authority must be attached to the product. It is unclear to what extent variations in manufacturing QA affect HIV RDT field performance and failure.

Health system factors QA is a low priority due to limited budgets and lack of political will. There are indications that this is especially true for private sector procurement where QA is not necessarily a prerequisite for purchase.

Health system factors that are well known in low-resource settings were described as contributing to, if not the direct cause of, human errors. An overburdened staff exposed to high turnover and under-established quality management typically resulted in the practice of non-compliant test protocols.

Additionally, procurement challenges in conjunction with poor planning lead to use of readily available and incorrect substitute supplies. One example presented by interviewees is that upon encountering stock-outs of buffer from one manufacturer’s kit, users substituted a buffer from

8

test kits by a different manufacturer. In some anecdotal cases, water has been used to replace a buffer.

4.2.2 Perception of self testing We also explored the concept of HIV self testing. The expressed opinions by global stakeholders on self testing vary widely from whole-heartedly endorsing self testing to having great reservations about promoting self testing. Below is a list of current concerns/opinions from interviewees on this topic:

Increased accessibility Those who expressed positive opinions about self testing emphasized the increased accessibility with self testing, trust in the robustness of HIV RDT test technology, as well as in self testers to follow the test protocol–potentially more carefully than an overburdened lab staff who conducts many of these on a daily basis.

Target population Another aspect of self testing that generates differing opinions is what population should be the target for self testing. Some advocate that the main reason and attraction for self testing would be underserved, difficult-to-reach populations who do not trust or access local facilities in high-prevalence settings. Some propose that increased accessibility for these groups via self testing may be a motivation to seek these and other services. Conversely, other interviewees felt that the hard-to-reach populations may already be outside the reach of public services and receiving a positive HIV result may actually drive them further away from the very services they need to access. Other interviewees added that the upper middle-class population is the most appropriate target population for self testing due to their ability to access follow-up services. Other opinions are that self testing would be popular for couples (both male and female) and a feasible option for repeat testing (e.g., in high-prevalence areas where people should be tested more than once).

Adaptability of current tests Global interviewees gave differing opinions regarding the adaptability of existing tests. Since the data showing the significant failure of existing HIV RDTs is conducted by trained staff, some interviewees felt this indicates that HIV self testing would be much more problematic in the hands of lay users: “If trained staff cannot do it, how can lay people?” Others express more confidence in self testers as being capable of and likely to conduct the test more carefully than overburdened lab workers. Many feel that self tests need to be an improvement over current tests in the following ways: be more self-contained, involve fewer steps, have fewer components, and overall minimize potential for human error at each step. Some express relative satisfaction with tests that are currently available.

4.3 Field assessment of HIV RDT performance and quality 4.3.1 National policies National HIV policies exist in all three countries visited. National guidelines regarding HIV testing include detection, prevention, testing, QA, and treatment. These guidelines are available in hard copy as well as on the internet.9,10,11,12

Although policies have been in place for quite a few years in all countries, we observed some discrepancies between policy and practice. The most obvious discrepancy between policy and practice was in supervision and monitoring the quality of testing procedures.

9

4.3.2 National algorithm There is one HIV testing algorithm for each country which is outlined in each country’s national guidelines for HIV testing. Considerations for selection of tests for the algorithm include sensitivity, specificity, a ratio of both (for tie breakers), testing time, shelf life, storage temperature, packing, labeling, avidity, number of runs per kit, and cost.

Serial testing is conducted in all three countries and it consists of the use of three standard tests for screening, confirmation, and a tie breaker in the case of discordance. Table 1 shows current systems in use in each of the three countries.

Table 1. Current algorithms in use Kenya Malawi Uganda Algorithm established in

2006 2011 (following the Bioline recall)

2008 2002

Screening Determine® Determine® Determine® Determine® Confirmation Bioline Uni-Gold™ Uni-Gold™ Stakpak Tie breaker Uni-Gold™ ELISA Bioline Uni-Gold™

While some countries have a list of approved HIV tests (e.g., Kenya has list of more than 80 WHO-approved tests), for the algorithm there are no alternative tests approved for use in any country. Stakeholders in Kenya and Uganda stated that the tests and manufacturers must be approved by WHO and the US Food and Drug Administration for procurement. Therefore, only tests that are part of the national algorithm are used. Interviewees felt that this is good since it decreases confusion; is cost-effective; and makes central procurement, training, and control easier. However, in Kenya at time of the interviews, this was problematic with the 2011 manufacturer recall of SD Bioline. In this case, testing sites were instructed to use the tie-breaker test for the confirmatory test and a long ELISA test for the tie breaker. Testing sites indicated indeterminate supply shortages of the confirmatory test.

Interviewees reported that the algorithm is supposed to be evaluated every two to three years. At the time of the interviews, all three countries were in the process of evaluating tests to consider them for use in a revised algorithm.

The Uganda National Lab is currently evaluating 13 test kits (including 3 saliva tests). However, they reported they will not have enough funding to do as much field evaluation as is desirable.

In Kenya, four designated testing centers (National Public Health Lab Services, University of Nairobi, Kenya Medical Research Institute, and Coast Provincial General Hospital) conduct assessments of sensitivity, specificity, positivity predictive value, and negative predictive value of tests being considered for the algorithm. Ultimately, the testing algorithm is determined by the National AIDS and STI Control Programme in conjunction with a group of governmental and nongovernmental stakeholders. A number of HIV rapid tests, including four “ultra-rapid tests” that give results in less than five minutes, are currently being evaluated for a new algorithm. These ultra-rapid tests are of great interest due to potential shorter total testing time (which would enable health care workers to conduct all three tests in the case of discordant results).

Malawi is testing approximately ten WHO prequalified test kits. The results will be available in 2012.

10

4.3.3 Who conducts HIV testing Two main groups of staff conduct HIV testing: Staff in lab facilities and staff in non-lab facilities. In labs, staff may be dedicated to conducting HIV testing or may rotate conducting different tests. Outside lab facilities, staff conduct HIV testing in specific clinics offering HIV testing (e.g., VCT, prevention of mother to child transmission clinics, or HIV counseling and testing [HCT]). Besides these specific clinics, provider-initiated counseling and testing and HCT may be offered in clinics and wards in health facilities, as well as through community outreach efforts.

All lab staff have specialized laboratory training and certifications (from one to more than four years). Outside the lab, nurses, counselors, social workers, nurse-midwives, and physicians conduct HIV testing. In addition, we also learned that non-health care workers conduct HIV testing. This includes trained community workers, volunteers, expert patients, and other lay persons.

4.3.4 Training and supervision The national guidelines for each country outline training requirements for facilities and personnel who conduct HIV training. Trainings are offered by a variety of agencies including many nongovernmental organizations (NGOs) in each country, and training is mandatory for all test users. Certification or certificates of accomplishment are awarded upon completion of courses which vary from two to four weeks in length. Trainings generally include counseling and hands-on experience conducting testing. We did not learn of any requirements for renewal or maintenance of training certification.

Non-lab staff who conduct testing reported learning via various trainings offered by NGOs, through externally funded projects. Many reported they sought training on their own rather than as a requirement for their job.

Funding for training was identified as a major challenge for conducting initial training and for ensuring maintenance of skills. For example, the training in Malawi used to be four weeks, and it has now been reduced to three weeks due to a reduction in training funds.

In addition, interviewers identified the shortage of well-trained health workers as another challenge in standardizing training formats. The necessity for a rapid response has given rise to the practice of task shifting. Task shifting involves extending the scope of practice of existing cadres of health workers to allow for the rational redistribution of tasks among the health workforce in order to make better use of the health workforce and ease bottlenecks in the service delivery system.13 Having less-trained personnel conducting HIV RDTs has been useful in reaching more people. However, training needs for lay workers require a more stringent training schedule and countries are not always able to accommodate this need due to financial constraints.

The situation with supervision is even more challenging than for training. Funds for supervision are severely constrained in all countries visited. We heard that supervision occurs more in facilities closest to urban centers with remote areas having less, under-qualified, or even no supervision at all for years.

Funds for refresher training and supervision were identified as key for improving HIV testing quality in all countries.

11

4.3.5 Perception of current tests Overall, interviewees, both lab and non-lab staff, reported that conducting the current HIV RDTs was easy and that the training was straightforward. However, they reported some common challenges experienced when conducting the tests:

• Incubation period: Many interviewees stated that the incubation period of test kits is too long and a shorter test time (<5 minutes) is desirable.

• Buffer: Users reported running out of buffer even though the supplied amount is supposed to contain extra for the batch.

• Interpretation: Interpretation of results can cause doubt when color lines are not strong.

• Uncertainty with discrepant results: Many interviewees expressed discomfort and doubt about test quality with regard to discrepant results (e.g., positive, then negative, then positive results received in the algorithm). Almost all interviewees reported stories of clients who tested positive and then were tested again in another setting or time and found to be negative. These stories are circulated frequently and cast doubt for testers and clients alike regarding the quality of the tests. In Kenya this was compounded by the SD Bioline recall—which was in fact due to manufacturing irregularities and initiated by the manufacturer—which augmented fears that the tests could give incorrect results. This highlights the challenges in ongoing training and communication with users, which can strongly influence opinions and behavior.

• Test stability and quality: Some interviewees indicated they questioned how much temperature fluctuation influences test stability and quality. They cited storage temperature ranges as 2°C to 30°C, and actual storage temperatures were likely beyond that range. However, overall interviewees expressed confidence in the robustness of the test.

4.3.6 Procurement, supply, storage, and distribution of RDTs Quantification and procurement Procurement of RDTs is conducted in all three countries by international organizations, but driven by the ministries of health. The Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund) is the largest funder for procurement, providing funds for the ministries of health to buy tests. We also heard that some limited procurement is also independently conducted by NGOs.

Box 2. Procurement Agencies by Country

Kenya: Procurement is conducted by Supply Chain Management System (SCMS) (90%) and Japan International Cooperation Agency (10%). Quantification is conducted by same entities. The Global Fund provides resources for procurement.

Malawi: Procurement and distribution is conducted by the United Nations Children’s Fund. Quantification is conducted by the Ministry of Health and the Global Fund provides resources for procurement.

Uganda: Procurement and quantification for public facilities is by National Medical Stores. Joint Medical Stores procures and quantifies for private sector. Technical assistance for quantification is currently provided by NGOs with funds from CDC and the Global Fund.

12

In some countries, such as Malawi, procurement has been affected by delays in the release of funds. The time for approval from the Global Fund is long. Negotiation between ministries of health and the Global Fund can last up to nine months.

All countries reported issues with quantification of needs for HIV RDTs. In Kenya and Uganda, the number of tests procured was not consistently reported and did not match the numbers of tests reported as being done. A 2008 report on supply and consumption of rapid HIV test kits in Kenya estimated that between 35,000 and 40,000 test kits were lost. In Uganda, the forecast is for 10,000,000 HIV screening tests to be conducted, whereas health management information systems data report 5 million tests as being run.

In all three countries, there is a major challenge with data for quantification of RDTs. People responsible for quantification are estimating quantities based on need. However, facilities report that number of RDTs received do not match the need. In addition, extra stock is usually calculated and requested, but because initial demand is never met, extra stock is not available. A further challenge to quantification was cited as donor procurements, which are welcome but unpredictable, sometimes resulting in laboratories canceling their regular orders so as not to have overstock and expired tests.

Interviewees attributed the discrepancy of number of procured tests versus actual number of tests run to:

• Poor records of tests conducted. Failure to account for tests used. Countries appear to be in the process of establishing data reporting systems for testing.

• Pilfering. Test kits are being siphoned to private markets. One interviewee familiar with distribution felt this may be unlikely and that the private market for HIV rapid testing is “dead.” The general population knows that HIV testing should be offered for free, so there is little market for offering HIV testing in the private sector. Others stated that many patients may seek care in private clinics because of the increased privacy. Further, private clinics can charge patients a small administrative fee for services that may include HIV testing.

• Expired kits. In Kenya, one rural facility reported that at one point they were instructed to use the confirmation (second) test in the algorithm for the screening (first) test because there was an overstock of that test about to expire. One interviewee suggested packaging tests in smaller quantities would be helpful to prevent wasted kits due to expiration.

• Hoarding. Interviewees suggested that staff do not like to admit that they are out of stock so they “hoard” supplies. It is unclear how this contributes to kit loss except perhaps by contributing to more expired test kits.

The extent to which each of the above reasons contributes to loss of test kits is unknown.

Supply All countries reported having stock-outs. The frequency and duration of stock-outs varied by country.

Malawi interviewees reported long and frequent stock-outs. Kenya and Uganda reported less frequent stock-outs that lasted for shorter timeframes (e.g., one to two months). We also did hear that the procurement agency for private clinics in Uganda was out of stock of the screening test for seven months in a recent year. The improvement in supply was attributed to Supply Chain

13

Management System (Kenya), which conducted training and capacity building; the Global Fund, United Nations Children’s Fund (UNICEF), Japan International Cooperation Agency, and other entities who have assisted with purchase of tests; as well as technical assistance with forecasting (e.g., Management Sciences for Health in Uganda). However, particularly in more rural areas and lower level health facilities, interviewees reported that supply is inconsistent and often less than requested. In addition, the recent recall of SD Bioline was causing a lot of problems with supply in Kenya and Malawi at the time of our visits.

Storage and distribution Storage of test kits varies by country and by location in each country. In Kenya and Uganda, test kits are stored at the central medical store until distribution to field sites. Tests are distributed to the district level, which then distributes directly to lower-level facilities. Slow distribution (one to two months) has been somewhat addressed by contracting with private couriers. This, of course, takes additional financial resources. The central warehouses (for public and private facilities) we visited in Uganda were temperature-controlled and appeared well-organized and managed. On the other hand, in Malawi, tests are distributed immediately after arrival into the country. The distribution is conducted by UNICEF. A few years back, tests were stored at the central medical store in Malawi. However, there were many irregularities at that location that prompted the Global Fund to assign UNICEF as a procurement and distribution agency in that country.

Some interviewees pointed out that a shelf life of 12 months is too short given that distribution can take up to two months. In addition, some stock is not shipped immediately, further decreasing shelf life.

4.3.6 Quality assurance and quality control Methods The main QA/QC procedures that were reported being utilized are the following:

• Testing a sampling of devices from a new batch of test kits with known positives and negatives (i.e., from patient previously tested).

• Proficiency Testing (PT). Using dried blood spot panel. The panels are sent from National Quality Assurance or the reference lab to test sites. Test sites reconstitute the dried blood spot panel samples and they are tested by the staff at the test site. The site then sends their results to the National Lab, which provides feedback on their performance, usually in form of a pass or fail grade. PT is conducted in all three countries though it varies by facility and even user.

• Supportive supervision. Observation, retraining, coaching. Extent of implementation and results of QA/QC In Kenya, Uganda, and Malawi, QA is mandated by the ministry of health, handled by the national reference laboratory, and currently funded by the CDC. In all three countries, policy for testing and quality management is set by the ministry of health and advised by the National Reference lab.

In Uganda, a few larger, better resourced, more central labs reported doing retesting with known positives/negatives on a regular basis (e.g., with each new batch of test kits). One interviewee

14

estimated PT is conducted on less than 10% of HIV testing facilities. One recent report of QA implementation in Uganda (including ten testing sites) reported a 3% to 18% discordance rate with average of 12% discordance (oral communication, 2012).

More rural, lower-level facilities in Kenya reported that QC monitoring was done only through observation and supportive supervision. This is not conducted regularly, consistently, nor rigorously. Kenya reported starting QA with PT in 2008, estimating the participation rate has increased up to 50% currently. Results indicated 40% discordance in early efforts, reduced to 5% in the most recent rounds. Since October 2011, Kenya selects individual testers instead of facilities for EQA. Kenya’s EQA is now completely administered by the National Lab. The Kenya CDC originally helped set up the system and provided technical assistance, but has successfully transferred all responsibilities to the National Lab.

In Malawi, QC is reported to be conducted on a daily or weekly basis depending on the volumes handled at the facility. In addition, facilities are instructed to conduct QC using predetermined control samples once a new batch will be used. Some facilities do not have refrigerators to store QC samples provided by the ministry of health. Only central hospitals and testing centers with high demand use dry blood spots for QC. Results of QA Uganda and Kenya reported that administration of PT in conjunction with feedback and retraining has resulted in improved performance. Over several quarters, Uganda reported that performance increased from 60% passing to 95% (with 10 sites participating). Kenya reported passing grades from 79.7% to 87.6% to 91.5% (50% of sites participating in the most recent rounds of PT). Some failing grades may be attributed to users learning to conduct PT. For example, we heard reports from interviewees that they did not know they needed to reconstitute the panel specimens within 24 hours, store within certain temperatures, and conduct the tests on the panel samples according to the algorithm. On the other hand, the passing grades still do not approximate 99% clinical sensitivity.

Malawi reported only 60% PT following immediate training. Testers are paired with experienced counselors until they acquire proficiency. After testers go on their own, there is report of PT of 85% for lay workers and approximately 95% for trained health workers. Challenges to QA/QC Testing sites visited reported that PT was supposed to be done for 10% of clients. This has proven to be a burden, both for testing sites as well as the national referral lab. PT is administered by the National Lab and has needed support by the CDC and other partners for implementation. Some cited challenges to implementing PT have been specimen/panel transport including maintaining the cold chain, getting completed panels back to reference labs, and feedback and required retraining to testing sites.

Other reported challenges to QA include lack of tests (undersupply of stock) to do PT, lack of budget for QA, and staff turnover. We also noted that in Uganda and Kenya, PT was administered to lab staff separately from non-lab staff. Hence, testing is conducted in parallel with limited sharing of information, training, and resources. In some cases, lab staff members were conducting PT, but non-lab staff members were not reached. Kenya has started selecting individual testers to conduct PT in an effort to address this issue.

15

Some successful interventions to improve proficiency testing have been setting up couriers to deliver the panels to sites, emailing results of the PT to sites, having results available on-line, and even using short message service (SMS) to transmit PT results to printers (for sites that do not have computer access).

Other opinions of PT note that testers are aware that their results will be scored, so they pay better attention to how they conduct the PT even though they are instructed to conduct testing for PT as if they are conducting routine testing.

Post-market surveillance Post-market surveillance (checking the quality of tests that have entered the routine procurement/distribution flow) is not conducted routinely by any facility in any of the three countries.

Post-market surveillance is mainly conducted via research institutes for research purposes. In Uganda, surveillance records indicating HIV prevalence rates are also used as a proxy to assess the need to change the algorithm. This is done by examining HIV positivity rates to see if these are as expected (indicating no change in the prevalence of HIV types).

In Kenya after the SD Bioline recall, the National AIDS Control Program is assembling a technical advisory group to develop a protocol for regularly checking test quality from lot to lot. In-country research cited by interviewees indicates that HIV RDTs have high performance rates.

Uganda reported a Medical Research Council study of thousands of samples showing a 99% concordance rate for HIV RDTs (oral communication 2012). A Kenya interviewee citied research that demonstrated high concordance for HIV with RDTs and PCR (n=~3000) (oral communication, 2012). A small vaccine study (n=23) in Kenya (Walter Reed) conducted parallel testing with Determine® and Unigold™ and reported 28.6% discordance. Shott et al.6 reports lowered specificity when weak positives are included in analysis.

4.3.7 User errors Interviewees reported a number of user errors that they have directly observed or that they have experienced. These include:

• Inappropriate storage: Storing test kits in non-recommended conditions (e.g., leaving test kits on windowsill; putting kits in a car or pocket to transport to another site; leaving them in a hot car for a period of time).

• Incorrect use of algorithm: Running the test even in cases when only one or two tests in the algorithm are available. This means the lab would be unable to conduct the confirmatory and/or tie-breaker tests if needed. It may also mean the client is given a positive result without a confirmatory and/or tie-breaker test.

• Inappropriate use of sample collection devices: Collecting the sample without using the recommended sample collection devices provided in the kit or specified in the product insert, which can result in inadequate sample volume.

• Lack of adherence to standard operating procedures: Using too much buffer (buffer always runs out before test kits despite surplus supply) or using buffer from different kits and even using saline and water when the lab is out of buffer.

16

• Incorrect incubation time: Not waiting long enough (e.g., 1 to 2 minutes instead of 15), or leaving it in too long. One supervisor reported a lack of timers. People were using timers on their phones but would then get a phone call and then lose track of time. Timers were purchased for all testing sites in this case.

• Erroneous interpretation of results: Interpretation can be challenging (e.g., weak positives). In some cases, poor eyesight was discovered.

Interviewees were not able to quantify these errors but incubation time was the most commonly reported error as well as deviations in sample collection. In general, interviewees suggested non-medical/lab staff appear to have less proficiency in performing the test than medical/lab staff. However, they reported that training and experience improves performance.

4.3.8 Staffing and capacity Many facilities stated they are understaffed and under-capacity, particularly in more rural and lower-level facilities. For example, in Malawi, the shortage of health workers is so extreme that there is only around one doctor for every 100,000 people.13

Interviewees reported the impact of understaffing is that personnel are often stretched to cover evening and weekend shifts when needed. Staff fatigue was also cited. Of note is the report that the staffing structure for health systems in many African countries does not give incentives for staff who seek further education and training. That is, staff who take training courses and pursue higher education do not necessarily receive a higher position or salary. As a result, many staff leave the public system when they have obtained more education and training. This often results in higher-level and supervisory positions being filled by under-qualified staff.

4.3.9 Perception of home-based and self testing Perception of self testing was mixed. Some interviewers perceived it as presenting a good possibility to increase access to vulnerable populations, while others thought it was not a favorable practice. The need and importance of counseling at the point of diagnosis was the consistent factor that influenced interviewers’ opinions. Some respondents pointed to the benefits for groups that are difficult to reach, discordant couples, and for retesting.

To begin to further explore the concern around self testing, we asked about stigma surrounding HIV testing and being diagnosed as HIV positive. In general, more urban, younger interviewees indicated that the stigma for getting tested for HIV is much lower, with some stating stigma is basically nonexistent now. These respondents pointed out that on campaign days, there are long lines of people waiting to take HIV tests in public. Respondents also reported that the stigma surrounding an HIV-positive diagnosis and taking HIV medication has also decreased considerably. Some cited stories of neighbors encouraging HIV-positive neighbors to take their medications.

Similarly, clients are not inhibited to wait outside clinics that are clearly offering testing or serving HIV-positive clients. Other interviewees told stories of friends or relatives who still would not admit that a family member died of HIV implying that stigma is still quite high. Some rural interviewees were cautious that clients in their areas would access needed services if they were offered self testing. Others indicated more community input is needed. It is clear that this area merits much more investigation.

17

There are many additional facets of self sampling that call for more research. Some respondents were very interested in tests that could test saliva because it is less invasive. However, some respondents wondered about potential increased misinformation and stigma. At this time, the general public no longer fears transmission of HIV via casual contact. However, if saliva testing becomes more widespread, some respondents feel the public will begin to associate saliva with HIV and hence assume that transmission is possible via saliva. This factor also points to further work needed prior to more widespread implementation of self testing.

5. Opportunities for Improvements in HIV testing Interviewees identified areas for improvement in the technical performance of HIV RDTs. These opportunities include technological improvement of current tests, development of new tests, as well as specific health system issues that will influence the overall testing process as shown in Table 2 below:

Table 2. Opportunities for Improvement of HIV Testing Based on Interview Responses

Technological Improvements

Development of New Technology

Health Systems

• Make a test that is easier to use correctly and less susceptible to errors, which could decrease need for training.

• Decrease incubation time of the current test.

• Eliminate the need for a buffer.

• Make a test with clearer positive or negative test lines; no weak positives.

• Make a test with increased shelf life.

• Make a test with increased ability to withstand transport and storage variability.

• Package in smaller quantities.

• Provide better options for EQA than dried blood spots: One interviewee proposed a simple dipstick test that is coded (and can be administered more easily).

• Make a test that allows health workers to accurately diagnose HIV infection during the window period.

• Improve lab capacity, which in turn can lead to increased QA capacity for facilities.

• Increase health worker capacity:

o Increase the number of staff.

o Strengthen training.

o Improve staffing structure to encourage more training and education.

o Guarantee supervision of skill.

• Improve supply chain: o Strengthen accuracy of

data for quantification.

o Ensure regular supply of test kits.

o Improve efficiency of financing mechanisms for procurement of test kits.

18

6. Conclusions and Recommendations Based on the findings of this rapid assessment, we are able to define three different domains that directly or indirectly influence the quality and field performance of HIV RDT. These domains are closely linked, and an imbalance in one may affect the other two. Improvements in HIV testing will necessarily have to take a closer look at a multitude of issues to arrive at a comprehensive solution that can address key challenges.

Figure 1. Domains affecting quality and field performance of HIV RDTs

There was consistency in the responses between global stakeholders and in-country stakeholders regarding issues affecting the quality and performance of HIV RDTs (described in Figure 1). Health systems issues were identified as a key contributor to field failure of HIV RDTs. In particular, deficient training and supervision were mentioned consistently as important factors that influence poor performance. Deficient training was also seen as the main reason for user error. In addition, lack of human resources and high demand for services were seen as significant contributors to deficient performance of tests.

Although many efforts have been put in place to improve supply chain management of HIV RDTs, stock-outs were reported as a prevalent issue in one of the countries where we interviewed. Furthermore, accurate quantification of needs for diagnostic products is required to ensure adequate supply.

QC practices are not implemented consistently in countries. There is a need for standardization and ongoing implementation of QC practices that can reach the most remote areas of countries.

Respondents also identified several opportunities for improving HIV testing services. The international community should take a look at these suggestions to drive change in implementation of quality procedures that can ultimately ensure good practices and accurate HIV test results. With the impetus of addressing the three main domains that affect quality and field performance of HIV RDTs, we are presenting the following recommendations.

Field Performance

of HIV RDT

Human Factors

Health Systems Technology

19

Recommendations for Improving HIV RDT field performance: • Conduct evaluations of existing technologies that address test weaknesses identified through

this assessment.

• Identify field research gaps and implement research that can reveal solutions to field performance failure of RDTs, for example, defining influence of temperature variability in field performance of HIV RDTs.

• Improve QA systems by identifying QC methods that are easier to use (internal or external to the tests).

• Support government or HIV implementers with post-market surveillance efforts. • Support policy efforts to institute training certification, ongoing retraining, and supervision

of personnel who conduct tests. • Engage with manufacturers to consider technology improvement of RDTs.

• Increase awareness of issues regarding HIV field test performance to increase resource allocation for QC efforts.

• Explore sustainable product evaluation mechanisms of manufacturers with the highest demand for products.

• Support efforts to improve quantification of the need for HIV RDTs. • Support ongoing initiatives to meet staffing needs in developing countries. • Increase capacity building for laboratories including QA.

Recommendations for HIV self testing • Identify target population(s) for a test based upon who could most benefit from self testing

and/or will be most likely to use the test. • Define market size of this potential product, which will be crucial for the value proposition to

a commercial manufacturer. • Identify service-seeking behavior of various populations who would access self testing (e.g.,

for follow-up). • Evaluate performance and ease of use of self tests in the hands of lay users.

• Understand current self testing and instructions by target populations to identify the need for adaptations or modifications.

• Link existing systems for providing crucial ancillary services like counseling and enrollment in care to those who perform HIV testing.

• Further understand barriers and facilitators to testing.

20

7. References 1 World Health Organization (WHO), US Department of Health and Human Services, Centers for Disease Control and Prevention (CDC). Guidelines for assuring the accuracy and reliability of HIV rapid testing: Applying a quality system approach. Geneva: WHO; 2005. 2 Wolpaw et al. The Failure Of Routine Rapid Hiv Testing: A Case Study Of Improving Low Sensitivity In The Field. BMC Health Services Research. 2010;10:73. 3 Parekh BS, Kalou MB, Alemnji G, Ou CY, Gershy-Damet GM, Nkengasong JN. Scaling up HIV rapid testing in developing countries: comprehensive approach for implementing quality assurance. American Journal of Clinical Pathology. 2010;4(4):573–584. 4 Plate D, Evaluation and implementation of HIV rapid tests: The experience in 11 African countries. Aids Research and Human Retroviruses. 2007; 23(12):1491–1498. 5 Kagulire S C, Opendi P, Stamper P D, et al. Field evaluation of five rapid diagnostic tests for screening of HIV-1 infections in rural Rakai, Uganda. International Journal of STD AIDS. 2011: 22(6):308–309. 6 Shott JP, Galiwango RM, Reynolds SJ. A Quality Management Approach to Implementing Point-of-Care Technologies for HIV Diagnosis and Monitoring in Sub-Saharan Africa. Journal of Tropical Medicine, 2012:Article ID 651927, doi:10.1155/2012/651927 7 Moodley D, Pravi M, Themba N, Esterhuizen T. Reliability of HIV Rapid Tests in a Clinical Setting is Largely Dependent on User Performance. South African Medical Journal. 2008;98(9):707–709. 8 Jogersen P., Malaria apid diagnostic tests in tropical climates: the need for a cool chain. American Journal of Tropical Medicine and Hygiene. 2006;74(5):750–754. 9 Kenya National HIV Testing Guidelines. Available at: http://nascop.or.ke/library/HTC/National%20Guidelines%20for%20HTC%20in%20Kenya%202010.pdf. Accessed April 9, 2012. 10 Kenya HIV Quality Management guidelines. Available at: http://nascop.or.ke/library/HTC/National%20QMG%20Framework%20Final.pdf. Accessed April 9, 2012. 11 Uganda HIV National Guidelines. Available at: http://www.who.int/hiv/topics/vct/UG_HCT%20Policy%20DRAFTFeb05.pdf. Accessed April 9, 2012. 12 Malawi HIV National Guidelines. Available at: http://gametlibrary.worldbank.org/FILES/577_VCT%20guidelines%20-%20Malawi.PDF. Accessed April 9, 2012. 13 WHO. The World Health Report 2006—Working together for health. Geneva: WHO; 2006.

Appendix I. List of References for Literature Search

1. Abimiku AG, Croxton T, Akintunde E, et al. Experiences in establishing a PEPFAR-supported laboratory quality system in Nigeria. American Journal of Clinical Pathology. 2010;134(4):541–549.

2. Baghdasarian SB, Gordon SM, Yen-Lieberman B. Evaluation and interventions to reduce errors in HIV-2 testing. Journal of Healthcare Quality. 2002;24(6):23–5, 61.

3. Black V, von Mollendorf CE, Moyes JA, Scott LE, Puren A, Stevens WS. Poor sensitivity of field rapid HIV testing: implications for mother-to-child transmission programme. BJOG. 2009;116(13):1805–1808.

4. Boeras DI, Luisi N, Karita E, et al. Indeterminate and discrepant rapid HIV test results in couples' HIV testing and counselling centres in Africa. Journal of the International AIDS Society. 2011;14:18.

5. Desai D, Wu G, Zaman MH. Tackling HIV through robust diagnostics in the developing world: current status and future opportunities. Lab on a Chip. 2011;11(2):194–211.

6. Dessie A, Abera B, Walle F, Wolday D, Tamene W. Evaluation of Determine™ HIV-1/2 rapid diagnostic test by 4th generation ELISA using blood donors’ serum at Felege Hiwot Referral Hospital, Northwest Ethiopia. Ethopian Medical Journal. 2008;45(1):1–5.

7. Ishengoma DR, Rwegoshora RT, Mdira KY, et al. Health laboratories in the Tanga region of Tanzania: the quality of diagnostic services for malaria and other communicable diseases. Annals of Tropical Medicine & Parasitology. 2009;103(5):441–453.

8. Jorgensen A, Shao J, Maselle S, et al. Evaluation of simple tests for detection of HIV antibodies: analysis of interobserver variation in Tanzania. Scandinavian Journal of Infectious Disease. 1990;22(3):283–285.

9. Jorgensen P, Lon L, Rebueno A, Tsuyuoka R. Bell D. Malaria rapid diagnostic tests in tropical climates: the need for a cool chain. American Journal of Tropical Medicine and Hygiene. 2006;74(5):750–754.

10. Kabbash IA, Mekheimer S, Hassan NM, Al-Nawawy AN, Attalla AA. Evaluation of HIV voluntary counselling and testing services in Egypt. Part 2: service providers' satisfaction. Eastern Mediterranean Health Journal. 2010;16(5):491–497.

11. Kagulire SC, Opendi P, Stamper PD, Nakavuma JL, Mills LA. Field evaluation of five rapid diagnostic tests for screening of HIV-1 infections in rural Rakai, Uganda. International journal of STD AIDS. 2011;22 (6): 308–309. PubMed: 21680664.

12. Kavinya T. Opinions on government introduction of mandatory HlV testing. Should there be mandatory testing for HIV in Malawi? Malawi Medical Journal. 2010;22(2):62.

13. Louis FJ, Perret JL, Morillon M, et al. [Use in Mozambique of WHO diagnostic strategies in HIV infection]. Sante. 1999;9(2):111–114.

14. Mandatory HIV Testing. A Public Debate. Report of an open Panel Discussion convened in Blantyre, December 2010. Malawi Medical Journal. 2010;22(4):132–133.

2

15. Medley AM, Kennedy CE. Provider challenges in implementing antenatal provider-initiated HIV testing and counseling programs in Uganda. AIDS Education and Prevention. 2010;22(2):87–99.

16. Menard D, Mairo A, Mandeng MJ, et al. Evaluation of rapid HIV testing strategies in under equipped laboratories in the Central African Republic. Journal of Virological Methods. 2005;126(1-2):75–80.

17. Moodley D, Moodley P, Ndabandaba T, Esterhuizen T. Reliability of HIV rapid tests is user dependent. South African Medical Journal. 2008;98(9):707–709.

18. Mwau M, Wangai M, Wachira J, Ibrahim M, Nyamongo J, Thuo M. 2008 Report on Supply and Consumption of Rapid HIV Test Kits In Kenya, January 2007 to May 2008.

19. Odaibo GN, Donbraye E, Adewumi MO, Bakarey AS, Ibeh MA, Olaleye DO. Reliability of testing and potential impact on HIV prevention in Nigeria. African Journal of Medicine and Medical Sciences. 2006;35 Suppl:131–135.

20. Pai NP, Balram B, Shivkumar S, Martinez-Cajas JL, Claessens C, Lambert G, Peeling RW, Joseph L. Head-to-head comparison of accuracy of a rapid point-of-care HIV test with oral versus whole-blood specimens: a systematic review and meta-analysis. Lancet. Published online January 24, 2012 DOI:10.1016/S1473-3099(11)70368-1.

21. Parekh, BS, Kalou MB, Alemnji G, Ou CY, Gershy-Damet GM, Nkengasong NJ. Scaling Up HIV Rapid Testing in Developing Countries, A Comprehensive Approach for Implementing Quality Assurance. American Journal of Clinical Pathology. 2010;134:573–584.

22. Parekha BS, Anyanwua J, Patela H, et al. Dried tube specimens: A simple and cost-effective method for preparation of HIV proficiency testing panels and quality control materials for use in resource-limited settings. Journal of Virological Methods. 2010;163: 295–300.

23. Peter TF, Rotz PD, Blair DH, Khine AA, Freeman RR, Murtagh MM. Impact of laboratory accreditation on patient care and the health system. American Journal of Clinical Pathology. 2010;134(4):550–555.

24. Piwowar-Manning EM, Tustin NB, Sikateyo P, et al. Validation of rapid HIV antibody tests in 5 African countries. Journal of the International Association of Physicians in AIDS Care. 2010;9(3):170–172.

25. Plate DK. Evaluation and implementation of rapid HIV tests: the experience in 11 African countries. AIDS Research and Human Retroviruses. 2007;23(12):1491–1498.

26. Richter M, Venter WD, Gray A. Home self-testing for HIV: AIDS exceptionalism gone wrong. South African Medical Journal. 2010;100(10):636–642.

27. Shott JP, Galiwango RM, Reynolds SJ. A quality management approach to implementing point-of-care technologies for hiv diagnosis and monitoring in sub-Saharan Africa. Journal of Tropical Medicine. 2012: 1–8, Article ID 651927.

28. Strategic Evaluation, Advisory, & Development Consulting. Analysis of POCT/VCT performed at South African primary health care clinics. Available at: http://www.sead.co.za/downloads/POCT-clinics-2011.pdf. Accessed April 10, 2012.

3

29. Wachira J, Kimaiyo S, Ndege S, Mamlin J, Braitstein P. What Is the Impact of Home-Based HIV Counseling and Testing on the Clinical Status of Newly Enrolled Adults in a Large HIV Care Program in Western Kenya? Clinical Infectious Diseases. Advance Access published December 8, 2011. 1–7.

30. World Health Organization. HIV Assays: Operational Characteristics, Report 16, Rapid Assays. Available at: http://www.who.int/diagnostics_laboratory/publications/Report16_final.pdf. Accessed April 10, 2012.

31. World Health Organization. HIV self-testing among health workers, A review of the literature and discussion of current practices, issues, and options for increasing access to HIV testing in sub-Saharan Africa. 2011.

32. Wolpaw BJ, Mathews C, Chopra M, et al. The failure of routine rapid HIV testing: a case study of improving low sensitivity in the field. BMC Health Services Research. 2010;10:73.

33. Yao K, Wafula W, Bile EC, et al. Ensuring the quality of HIV rapid testing in resource-poor countries using a systematic approach to training. American Journal of Clinical Pathology. 2010;134(4):568–572.

Appendix II: Interviewees List A. Global Stakeholders

Name/Position Organization Contact Information Rachel Baggaley, Field Evaluation

World Health Organization (WHO) baggaleyr@who.int

Shaffiq Essajee, Pediatric and Family Care

WHO essajees@who.int

Anita Sands, Technical Officer

WHO sandsa@who.int

Bharat Parekh, Team Leader, Serology/ Incidence

US Centers for Disease Control (CDC) bsp1@cdc.gov

Cara Kosack, Leader Diagnostic Network

Médicines Sans Frontières, Netherlands cara.kosack@amsterdam.msf.org

Ben Plumley, CEO Pangea bplumley@pgaf.org Maurine Murtagh, Consultant

The Murtagh Group maurinemurtagh@sbcglobal.net

Robert Martin, Professor of Public Health Practice

International Training and Education for Health (I-TECH)

rmartin1@u.washington.edu

Liz Corbett, Professor London School of Hygiene and Tropical Medicine

liz.corbett@lshtm.ac.uk

Leslie Scott, Head of Research and Development

National Health Lab Service (NHLS) Lesley.Scott@nhls.ac.za

Trevor Peter, Lead Laboratory Advisor

Clinton Health Access Initiative (CHAI)

tpeter@clintonhealthaccess.org

Zach Katz, Director of Diagnostics

CHAI zkatz@clintonhealthaccess.org

2

B. Kenya Interviewees Name/Position Organization Contact Information Jackson Hungu, Deputy Country Director/ Diagnostics Lead

CHAI Timau Plaza, Argwings Kodhek Rd. 3879714

Sam Kalibala Population Council skalibala@popcouncil.org Tel: 2713480

Dr. Okal, Provincial Level HIV Program Manager

Provincial Level HIV Program Manager (PASCO Nyanza)

Tel: 722-941507

Lab Manager, Lab Techs, VCT Health Workers

Nyanza Provincial General Hospital

Lab Manager, Lab Techs, VCT Health Workers

Kisumu East District General Hospital

John Waitumbi, Principal Investigator and Mary Elise Lynch, Researcher

Kenya Medical Research Institute/ Walter Reed, Kisumu

Tel: 0733-616548

Dr. Frederick, Deputy Medical Superintendant, Provincial Level HIV Program Manager

Provinical AIDS Control Western (PASCO Western—Kakemega)

Tel: 0720-494325

Lab Manager, Lab Techs, VCT Health Workers

Vihiga District General Hospital

Lab Manager, Lab Techs, VCT Health Workers

Western Provincial General Hospital

Dr. Matilu Mwau, Director Centre for Infectious and Parasitic Diseases Control Research/KEMRI

mmwau@kemri.org matilu.mwau@gmail.com Tel: 2722541/ 2713349 or 0722-205901 PO Box 3-50400, Busia, Kenya

Lab Manager National HIV Reference Lab Dr. Jane Carter, Technical Director, Clinical and Diagnostics

African Medical and Research Foundation (AMREF)

jane.carter@amref.org Tel: 699-4000

Dr. Ernest Makokha, CDC Lab Director, Kenya

CDC Global AIDS Program emakokha@ke.cdc.gov Tel: 254 20 2867-143 Cell: 0722-509167 PO Box 606-00621 Nairobi, Kenya

Mr. Ngige Determine Distributor Sales Manager Macmed Healthcare (K) Ltd.

Tel: +254-20-6005135, 6003582/3 Mobile :0722-202552, Fax: +254-20-6003049 Wilson Airport, New Lengai House, Second Floor PO Box 72299-00200

3

C. Malawi Name/Position Organization Contact information Dr Frank Chimbwandira, Director for HIV/AIDS unit

Ministry of Health

fchimbwandira@yahoo.com

Mtemwa Nyangulu,National Program Officer

HIV Testing and Counseling (HTC) mnyangulu@yahoo.co.uk Tel: +258-(0)884358298

Norman Chelewani, Lab Supervisor

Mission Hospital Deayang Luke

Innocent Mayenda, VCT Program

Mission Hospital Deayang Luke

Joyce Bakka, Senior Procurement Officer

UNICEF jbakka@unicef.org

Ms. Nakholi, Ms. Emid Mfune (PMTCT, HCT, ART)

Area 18 health center

Gertrude Mwale, Julian Nwwira, Jacqueline Murotho (PMTCT)

Area 18 health center–PMTCT

Samuel Chirwa Logistics Department, MOH schirwa@gmail.com Tel: +258-(0) 999552977

Mrs Limbe, HTC Coordinator

Lilongwe District

Dorica Chirwa, Logistics Officer for HIV/AIDS unit

MOH doricas@gmail.com

Joyce Bakka, Senior Procurement Officer

UNICEF jbakka@unicef.org

HCT staff Macro Hospital

4

D. Uganda Name/Position Organization Contact Information George SA. Kessia, Lab Supervisor

AIDS Information Centre (AIC) kegsia@yahoo.com; kessia@aicug.org Tel: 0414-271920; 0772-397602

Dr Cissy Kityo, Lab Supervisor

Joint Clinical Research Centre, Head–Laboratory Services

ckityo@yahoo.com Tel: 0414-342521

James Nkalubo and Peter Awio, Lab Techs

Joint Clinical Research Centre Head–Laboratory Services

jnkalubo1@yahoo.com; awiong@yahoo.com Tel: 0414-270622

Dr. Christina W. Mwangi, Lab Systems Policy Advisor

CDC Uganda mwn0@ug.cdc.gov Tel: 0752-751075

Zainab Akol, Program Manager STD/AIDS Control Program

Ministry of Health, AIDS Control Program Manager

stdacp@health.go.ug; akolzainabdr@yahoo.co.uk Tel: 0414-231563/9 Ext.248; 0414-257409; 0772-451008;

Dr. Robert G. Downing, Lab Systems Policy Advisor

CDC Uganda rgd6@cdc.gov Tel: 0752-788222

Emmanuel Higenyi, Procurement Private Sector

Joint Medical Stores (Private) emmanuelh@jms.co.ug Tel: 0772-873877

Eric Jenera, HIV RDT Quantification Advisor

MSH—Supply Chain Manager ejenera@msh.org Tel: 0414-505057; 0759-800079

Denis Oola, Laboratory Supervisor

Mulago-Makerere Joint AIDS Program (MJAP)

doola@mjap.or.ug Tel: 0755-553108

Ron (Infectious Disease Institute Project, Capacity Building), Lab Techs

Kiswa Health Center 4

Fatima, Lab Supervisor Health Center 3 Kitebe Bakunda Kamaranzi, HIV Lab QA Officer

Central Public Health Laboratory cphlug@gmail.com; akolzainabdr@yahoo.co.uk; ttenywa2000@yahoo.co.uk; kamaranzibakunda@yahoo. com Tel: 0772-406280; 0772-647676

Dr. Christine Watera, Project Coordinator; Prossi Nampijja, Lab Scientist; and Joseph Bukusuba, Lab Head

Uganda Virus Research Institute

Alfred and Eric, Procurement Team

National Medical Stores

Appendix III: Semi-Structured Interview Guides

INTERVIEW GUIDE FOR PROCUREMENT AND SUPPLY CHAIN STAFF 1 Updated March 12, 2012

Evaluation of Performance of HIV Rapid Tests in Field Settings (HIV EQA) Interview for Procurement and Supply Chain staff

For staff involved in procurement and supply chain of HIV Rapid Tests. Good morning/afternoon/evening. My name is ___________, and I am working for PATH, an international organization that focuses on improving health in developing countries. We are interviewing various stakeholders to understand the procurement and supply chain mechanisms for HIV Rapid Tests and to understand the challenges in doing this. This interview will probably take about 30-45 minutes. What questions do you have for me about the interview?

1. Procurement 1.1 Who performs procurement of HIV rapid diagnostic test kits for

government programs (government, donor, NGO, procurement agency, other)? [If possible, list the estimated percentage procured by each organization]

1.2 Who funds HIV rapid diagnostic test kits?

1.3 Please supply the following information for HIV rapid diagnostic test kits imported or purchased during the years 2010-2011:

a. Manufacturer (name and country) b. Product description (brand name etc) c. Product pack size (number of kits per test) d. Number of packs (number of kits)

[If this information cannot be obtained, please provide the brand names of kits being imported by each organization]

1.4 Does the amount procured by these organizations provide enough to cover the public sector needs? If no,

a. What is the gap? b. How is it being filled?

1.5 What percentage of HIV rapid diagnostic test kits are being supplied by the public sector versus the private sector?

1.6 When procuring HIV rapid diagnostic test kits, is competitive bidding performed? If no, please explain the mechanism for selecting which HIV rapid diagnostic test kits are purchased.

1.7 How are specifications developed, including quality standards and

INTERVIEW GUIDE FOR PROCUREMENT AND SUPPLY CHAIN STAFF 2 Updated March 12, 2012

product details for HIV rapid diagnostic test kits [review tender specification if possible]?

1.8 What criteria are used when procuring HIV rapid diagnostic test kits (quality, cost, lead-time, etc)?

1.9 How are the rapid diagnostic test kits shipped into the country (air or ocean)?

1.10 How do you ensure that the temperature range of the HIV rapid diagnostic test kits has been maintained from the time of manufacture to the point of end user?

2. Suppliers

2.1 What standards are used for qualifying quality of products e.g., requiring certification from international or national agencies?

2.2 How many sources are available for the supplies of HIV rapid diagnostic test kits?

3. Linkages to regulatory and importation processes

3.1 What are the National Regulatory Authority processes for registering and importing HIV rapid diagnostic test kits? Other involved agencies?

3.2 Is there an existing national list of approved manufacturers of HIV rapid diagnostic test kits? If yes, please supply a copy

3.3 If potential products are not currently on the registered list, is there a process for expediting, such as emergencies or humanitarian aid?

3.4 May we obtain information regarding all imports of HIV rapid diagnostic test kits into the country so that we can understand what is being supplied for the private sector also (distributors/importers, private health facilities, etc) [If yes, what is the process and timeframe for doing this?]

4. General Conditions

4.1 Is procurement centralized, or decentralized? If decentralized, how many levels are there? Vertical systems? Push or pull system?

4.2 Please describe the supply chain for HIV diagnostic test kits (from the time they arrive in-country through to the end user, including standard delivery times)

INTERVIEW GUIDE FOR PROCUREMENT AND SUPPLY CHAIN STAFF 3 Updated March 12, 2012

5. Forecasting Linkage

5.1 Which department/organization is responsible for forecasting?

5.2 How are forecasts developed i.e., based on what information sources?

5.3 How is the forecast data connected to procurement in terms of total needs, buffer stock, timing of orders and deliveries?

6. Linkages to logistics: receiving, storage, and on-forwarding

6.1 What receiving information is collected at the various levels of warehousing? [Review actual test kits AND paperwork if possible, check for quantity, date, manufacturer, brand, expiration date, FIFO (first in, first out), etc]

6.2 Are there stock outs at any of the warehousing points? If yes, please describe (quantity, frequency, action).

6.3 How are third party (donor) commodities managed to ensure delivery for the third party program, and against mingling and distribution with other products?

7. Public health facilities 7.1 Do you purchase HIV rapid diagnostic test kits? If yes,

a. Where from (local supplier, government stores, etc.)? b. What is the process? c. How do you assure quality? d. Do you have any quality issues (if yes what are they)? e. Is there a list of approved suppliers you can purchase from? [If

yes, obtain a copy of the list] f. What are the brand names of the products you purchase? g. How many do you purchase per month or per annum?

[Look at stock records and test kits if possible]

7.2 Do you experience stock-outs of HIV rapid diagnostic test kits? If yes, a. How often? b. What quantity? (If possible review stock records) c. What is the response to the stock-out?

7.3 Do hospitals and health centers have revolving funds or cost-sharing plans for HIV rapid diagnostic kits, or receive a budget from the government?

INTERVIEW GUIDE FOR PROCUREMENT AND SUPPLY CHAIN STAFF 4 Updated March 12, 2012

7.4 What are the capacity limits on your ability to store goods?

7.5 What temperature ranges do the HIV rapid diagnostic kits need to be stored in? Are the kits kept in these temperature ranges? How is this assured/checked?

7.6 Do you ever have expired HIV rapid diagnostic test kits in your store-rooms? If yes, how many per annum?

8. Private health facilities 8.1 Do you purchase HIV rapid diagnostic test kits? If yes,

a. Where from (local supplier, importer/distributor, central stores, etc)?

b. What is the process? c. How do you assure quality? d. Do you have any quality issues (if yes what are they)? e. Is there a list of approved suppliers you can purchase from? [If

yes, obtain a copy of the list] f. What are the brand names of the products you purchase and

how many do you purchase per annum?

9. Private Sector (importer/distributor, pharmacy) 9.1 Do you purchase HIV rapid diagnostic test kits? If yes, please supply

the following information: a. Manufacturer (name and country) b. Product description (brand name etc) c. Product pack size (number of kits per test) d. Number of packs (number of kits)

[If this information cannot be obtained, please provide the brand names of kits being imported by each organization]

9.2 How do you ensure that you are procuring quality HIV rapid diagnostic test kits?

9.3 Have you had any reports of quality issues of HIV rapid diagnostic test kits from your customers? If yes, what are they?

9.4 Who are your main customers in the private sector? Public sector?

9.5 What are the temperature ranges of the HIV rapid diagnostic test kits that you procure? Are you able to maintain the temperatures? How do you assure that the temperatures are maintained throughout the entire supply chain?

9.6 How are the HIV rapid diagnostic test kits shipped (air or ocean)?

INTERVIEW GUIDE FOR STAKEHOLDERS 1 Updated March 12, 2012

Evaluation of Performance of HIV Rapid Tests in Field Settings (HIV EQA) Interview for Key Stakeholders

For key stakeholders including international, national and regional directors of HIV control, MOH, coordinators, directors of other HIV-RDT programs. Good morning/afternoon/evening. My name is ___________, and I am working for PATH, an international organization that focuses on improving health in developing countries.

We are interviewing various stakeholders to understand the current quality standard practices involved for HIV testing and to understand the challenges for delivering HIV testing. This interview will probably take about 45 minutes.

What questions do you have for me about the interview?

TOPICS FOR DISCUSSION

General information What is your professional designation?

What is your role in HIV control and/or management?

How long have you been working in HIV control and/or management?

Please describe the scope and type of your HIV program. For example: Prevention of Mother to Child Transmission (PMTCT), Blood bank screening, STD clinic, antenatal clinic (ANC), surveillance program, etc

Please describe the process for HIV testing in your program/jurisdiction/region.

Can we please get a copy of the testing standards, guidelines, protocols and/or algorithm? • Probe on testing algorithm used to diagnose patients. How often does this algorithm

changes? What is process for changing the algorithm? • Probe on protocols for confirmatory diagnosis of patients. How is confirmation of

diagnosis achieved? Do patients have to go to a reference laboratory to get confirmation?

• Probe of type of tests used to confirm diagnosis • Probe on follow up when a patient test positive with a HIV RDT

What type of HIV rapid tests are used (in your program/ jurisdiction/country/region)?

• Please obtain information on brand names • How many tests are used per month/year in your program?

What do you think of the current HIV rapid tests you use to diagnose HIV?

• What do you see as opportunities for HIV rapid test programs? • What do you see as challenges for HIV rapid test programs? What do you think are most challenging technical aspects of conducting the test? (not implementing testing programs)

INTERVIEW GUIDE FOR STAKEHOLDERS 2 Updated March 12, 2012

Standards for quality of HIV rapid testing Are you familiar with a program to ensure the quality of HIV rapid testing? Can we please get a copy of any quality standard protocols? Probe on the following:

• What are the results of the program? • What are the most common causes of failure of HIV rapid tests? • How can the program for ensuring quality be improved?

Is your HIV rapid test program required to comply with standards of quality?

• What types of quality standards are required for your program? How does your program/country/region ensure compliance with these standards?

• Probe on whether external or internal assurance is conducted? • Probe on frequency of external or internal assurance evaluation • Probe on type of programs, institutions required to do these evaluations • Who is doing EQA in-country? To what extent?

Who requires/mandates these quality standard procedures/controls are in place?

• What policies/guidelines are used? • Who pays for your program/region/country to conduct the quality standard evaluations?

What are the procedures if the testing does not meet the quality standards?

What do you think are the biggest current challenges to monitor quality of HIV rapid tests?

• How do you think these challenges/barriers can be overcome? • If you had the opportunity, what would you change about the current quality standard

mechanisms?

How important do you feel quality assurance is for HIV rapid testing at different levels of the health system? District level, state level, central level. Training and supervision Who in your program/country is authorized to perform HIV testing using HIV rapid tests?

• Probe on minimal qualifications of personnel

What type of training is given to authorized employees? • Probe whether competency certification is part of the training, who certifies? • How often do staff turnover? • What are the challenges to providing training?

How does your program ensure health workers are trained adequately to perform HIV rapid tests?

• Probe on type of supervision available

Are health workers in your program/country/jurisdiction offered refresher trainings?

• How often are these trainings conducted?

INTERVIEW GUIDE FOR STAKEHOLDERS 3 Updated March 12, 2012

• Who pays for these trainings? Selection and procurement of HIV-RDT What attributes of a HIV RDT are important for purchase decisions?

• Probe on price • Probe on other attributes: acceptability, ease of use, safety, need for training,

transport/storage, detection of specific antibodies (HIV-1 vs. HIV-2), sensitivity, specificity, shelf life

Please describe the process of decision-making regarding selection of HIV rapid tests in your program/country/region

• Who is responsible for this decision? Please describe the process for procuring HIV rapid tests in your program/region/country?

• What are the major challenges for procurement of these tests? Please describe the process for distribution of HIV rapid tests from your level to other levels, institutions, etc

• Are there any challenges experienced in distribution of HIV rapid tests? Please describe • How frequent do you change the type of tests procured? Why? Do you consider this a

challenge for training of users? What other supplies are needed for testing?

• Please describe the process for procurement/ of other supplies needed for testing? • Challenges?

Explore potential for field-testing kits with panels.

What is extent of private market?

Storage Where are rapid tests (kits) stored before distribution to peripheral levels?

Please describe the conditions of storage of the RDT.

• Probe on requirements for cold chain. If cold chain require, how is cold chain maintained?

• What is average storage time of test kits in your facility? • Probe on other type of requirements needed for storage (controlled temperature, air

conditioning, space • Do you have any concerns about storage of HIV rapid tests? Please describe.

How do you keep inventory?

• Who is responsible for maintaining inventory? • Have you encountered issues rotating inventory? • Probe on issues regarding expiration date • Probe on issues relating space to keep inventory and training of people keeping

inventory How are other supplies needed for testing stored?

INTERVIEW GUIDE FOR STAKEHOLDERS 4 Updated March 12, 2012

Referrals Can you please recommend two additional contacts that you feel we should interview for this topic? Name organization, contact information?

INTERVIEW GUIDE FOR USERS 1 Updated March 12, 2012

Evaluation of Performance of HIV Rapid Tests in Field Settings (HIV EQA) Interview for Users and Clinic-level supervisors

For personnel involved in direct HIV testing (Laboratory technicians, blood bank screening, VCT) Good morning/afternoon/evening. My name is ___________, and I am working for PATH, an international organization that focuses on improving health in developing countries.

We are interviewing various stakeholders to understand the current quality standard practices involved for HIV testing and to understand the challenges for delivering HIV testing. This interview will probably take about 45 minutes.

What questions do you have for me about the interview?

TOPICS FOR DISCUSSION

General information

What is your professional designation?

What is your role in HIV testing, control and/or management?

• Where do you work? i.e. community, district, state, central level • How long have you been working in HIV control?

What HIV test methods do you currently use where you work? What amounts are procured annually?

Please describe the process for HIV testing in your institution/program

• Probe on testing algorithm used to diagnose patients. • How often does the algorithm change? What is process for changing? • Probe for how positive tests are confirmed if negative results are followed/questioned?

How indeterminates are handled? • Probe on protocols for confirmatory diagnosis of patients. How is confirmation of

diagnosis achieved? Do patients have to go to a reference laboratory to get confirmation?

• Probe of type of tests used to confirm diagnosis • Probe on follow up when a patient test positive with a HIV RDT

On average, how many patients are tested in one week at your institution?

• Of those tested, more less how many are confirmed positive? Technical

Note: If the facility conducts rapid tests, focus on these and not other methods that may be used.

What do you think of the current methods used to test for HIV?

• In your opinion, how easy or difficult is to run the HIV tests?

INTERVIEW GUIDE FOR USERS 2 Updated March 12, 2012

What is the most challenging technical aspect for conducting HIV testing? Steps? Or issues? (ALSO OBSERVE)

Probe on the following aspects:

• Following instructions of use • Collecting the sample? • Running/conducting the test? • Interpreting test results? • Reporting/transfer of information e.g., to medical records • Supplies? • Storage? • Training? • Staffing? How often do staff turnover/change? • Patient/testing flow? Workload?

What would you change to improve the test to make it easier to run? Less likely to have errors?

Probe on the following aspects:

• Collecting the sample? • Format? Reagent stability? Ease of use? • Running the test? • Interpreting results? Reading window timeframe? Stability of results? • Output? Preference of output? How would you like the results to be reported? • Processing? How would you like the sample to be processed for accurate results?

Serum, whole blood, oral fluid, other? • Supplies? • Storage? What do you think the ideal storage requirements would be? • Who runs? Who do you think should operate this system/perform the test? • Training? What training requirements would be needed?

Standards for quality of HIV rapid testing

How were you trained to conduct HIV testing?

• Are you supervised on a regular basis? What kind of supervision? Do you feel that the tests are reliable? Accurate? Error prone?

How consistently do you follow instructions on how to use the tests? If/when you don’t follow instructions consistently, please explain why?

How do you identify if there is a problem with the test?

If there is a problem with the test, what do you do?

• Is the test repeated? • How is it reported? To whom?

How frequently do you (or your facility) experience problems with the tests?

Describe some of the problems you have encountered with and how they were handled.

• How would you fix these problems?

INTERVIEW GUIDE FOR USERS 3 Updated March 12, 2012

What mechanisms are used to ensure quality standards of testing?

• How often are these conducted? Who requires/mandates these quality standard procedures/controls?

• What guidelines/policies are used? • Who pays for these procedures to be conducted? • What are the procedures if the testing does not meet the quality standards? • Who are the results reported to and how recorded?

What do you think are the biggest current challenges to monitor quality standards of HIV testing?

• How do you think these challenges/barriers can be overcome? • If you had the opportunity, what would you change about the current quality standard

mechanisms? Storage

Where are test (kits) stored? Note to interviewer: please visit and take notes of storage conditions

How do you keep inventory?

• Who is responsible? • How do you track expiry date? • What is average storage time of HIV RDT kits? • Other supplies needed for testing?

What are challenges for storage?

• Have you encountered issues rotating inventory? • Probe on issues regarding expiration date • Probe on issues relating space to keep inventory and training of people keeping

inventory Selection and Procurement

How are rapid tests selected at your institution?

• What influences decision of which tests to purchase? What is the process for procuring rapid tests at your institution?

Are these test (kits) usually available when you need them?

• If no, can you please explain why they are not available every time you need them? • What do you do when there is a stock out? • Do you get test kits through the private market at all? • How frequent do you change the type of tests procured? What are the reasons for this change?

What are the major challenges for procurement of these tests?

What about other supplies are needed to perform testing with rapid tests?

INTERVIEW GUIDE FOR USERS 4 Updated March 12, 2012

• What is the process for procuring/supplying other supplies needed for testing? • What are the Challenges?

Does your lab/institution distribute tests to other labs or institutions?

• If so, please describe the process for distributing (to other levels, institutions, etc)? • What are challenges experienced in distribution? Potential for testing kits with panels? Extent of private market?

Other Referrals

Who else do you recommend we interview for this project? Name? Contact information?