WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 June 2017
WHO/EMP/RHT/PQT/2017.xx Page 1 of 21
WHO PROTOCOL FOR PERFORMANCE
LABORATORY EVALUATION OF
LYMPHOCYTE SUBSETS ENUMERATION
TECHNOLOGIES
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 2 of 21
© World Health Organization 2017
Some rights reserved. This work is available under the Creative Commons Attribution Non-Commercial-
Share Alike 3.0 IGO license (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).
Under the terms of this license, you may copy, redistribute and adapt the work for non-commercial
purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there
should be no suggestion that WHO endorses any specific organization, products or services. The use of the
WHO logo is not permitted. If you adapt the work, then you must license your work under the same or
equivalent Creative Commons license. If you create a translation of this work, you should add the following
disclaimer along with the suggested citation: “This translation was not created by the World Health
Organization (WHO). WHO is not responsible for the content or accuracy of this translation. The original
English edition shall be the binding and authentic edition”.
Any mediation relating to disputes arising under the license shall be conducted in accordance with the
mediation rules of the World Intellectual Property Organization
(http://www.wipo.int/amc/en/mediation/rules).
Cataloguing-in-Publication (CIP) data: CIP data are available at http://apps.who.int/iris. Sales, rights and
licensing. To purchase WHO publications, see http://apps.who.int/bookorders. To submit requests for
commercial use and queries on rights and licensing, see http://www.who.int/about/licensing.
Third-party materials: If you wish to reuse material from this work that is attributed to a third party, such
as tables, figures or images, it is your responsibility to determine whether permission is needed for that
reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of
any third-party-owned component in the work rests solely with the user.
General disclaimers: The designations employed and the presentation of the material in this publication do
not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any
country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or
boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not
yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are
endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned.
Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by WHO to verify the information contained in this publication.
However, the published material is being distributed without warranty of any kind, either expressed or
implied. The responsibility for the interpretation and use of the material lies with the reader. In no event
shall WHO be liable for damages arising from its use.
WHO and the WHO Prequalification Evaluating Laboratory do not warrant or represent that the evaluations
conducted with the HIV test kits referred to in this document are accurate, complete and/or error-free.
WHO and the WHO Prequalification Evaluating Laboratory disclaim all responsibility for any use made of
the data contained herein, and shall not be liable for any damages incurred as a result of its use. This
document must not be used in conjunction with commercial or promotional purposes.
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 3 of 21
1. Introduction.
1.1 Prequalification of in vitro diagnostics
The World Health Organization (WHO) Prequalification of In Vitro Diagnostics is coordinated through the
Prequalification-Diagnostics Assessment team (PQDx). The aim of the WHO Prequalification of In Vitro
Diagnostics Assessment is to promote and facilitate access to safe, appropriate and affordable diagnostics
of good quality in an equitable manner. Focus is placed on products for high burden diseases and their
suitability for use in resource-limited settings.
The WHO prequalification of in vitro diagnostics process includes three main components:
Review of an pre-submission form and product dossier;
Performance evaluation of the product;
Inspection of the manufacturing site(s).
Performance evaluation will be conducted by two different mechanisms as described at:
http://www.who.int/diagnostics_laboratory/evaluations/alternative/en/. Performance evaluation in List 1
laboratories will be coordinated and paid for by WHO. Performance evaluation in List 2 laboratories will be
coordinated and paid for by the manufacturer.
This protocol provides the details of the procedure for evaluation of Lymphocyte subsets enumeration
technologies submitted for laboratory evaluation as part of the WHO Prequalification of In-vitro Diagnostic
assays on behalf of World Health Organization at the WHO Prequalification Evaluating Laboratory.
This protocol shall not replace more extensive IVD manufacturer’s protocols required for validation and
verification studies of their products.
1.2 WHO Performance evaluation of Lymphocyte subsets enumeration technologies
The performance evaluation determines the accuracy of Lymphocyte subsets enumeration technologies
in comparison with established performance criteria. These characteristics include: repeatability,
reproducibility, accuracy of measurement. In addition, a number of operational characteristics are
assessed including the suitability for use in laboratories and/or testing settings with limited infrastructure.
Dedicated and point-of-care CD4+ T-lymphocytes enumeration technologies used for monitoring of HIV
disease progression in resource limited setting are covered in this protocol
5. Study objectives
Overall objective 5.1.
To conduct multicentre and independent evaluation of dedicated and point-of-care CD4+ T-
lymphocytes enumeration technologies used for monitoring of HIV disease progression in resource
limited settings and disseminate their performance and operation characteristics.
Specific objectives: 5.2.
The specific objectives of the performance evaluation are
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 4 of 21
1. To assess repeatability and reproducibility of dedicated and point-of-care CD4+ T-cell enumeration
technologies.
2. To assess the accuracy of measurement of dedicated and point-of-care CD4+ T-cell enumeration
technologies compared to the classical flow cytometer.
3. To determine operational characteristics of dedicated and point-of-care CD4+ T-cell enumeration
technologies.
6. Study design
WHO Prequalification Evaluating Laboratory 6.1.
The WHO Prequalification Evaluating Laboratory shall be one that has undergone assessment using the
WHO Alternative Laboratory Evaluation Mechanism which include submission of an EoI, Stage 1 audit
(Assessment of EoI and specific quality management system (QMS) documentation), Stage 2 audit
which include On-site audit of the laboratory to assess compliance with WHO requirements and lastly
listed as WHO Prequalification Evaluating Laboratories.
http://www.who.int/diagnostics_laboratory/evaluations/alternative/en/.
The laboratory shall hold the following certification for quality management within the laboratory:
ISO17025 (General requirements for the competence of testing and calibration laboratories), ISO15189
(Medical laboratories: Particular requirements for quality and competence) or equivalent.
The person(s) listed in the EoI letter to WHO will act as the Principal Investigator (PI) for the work
performed by the WHO Prequalification Evaluating Laboratory
The laboratory shall hold the following certification for quality management within the laboratory:
ISO17025 (General requirements for the competence of testing and calibration laboratories),
ISO15189 (Medical laboratories: Particular requirements for quality and competence) or equivalent.
The person(s) listed in the EoI letter to WHO will act as the Principal Investigator (PI) for the work performed by the WHO Prequalification Evaluating Laboratory.
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 5 of 21
Training performance evaluation and supervision 6.2.
The following issues are key to minimizing error and maximizing the value of this performance
evaluation:
• The PI will be responsible for training the laboratory technicians on the details of the evaluation
protocol and on the performance of each assay undergoing evaluation;
• Only those personnel who have received specific training for this evaluation will be employed in
the evaluation;
• Accurate record keeping is crucial to the success of the evaluation and the PI will be responsible
for ensuring that all data required for the evaluation are recorded on the agreed data collection
sheets, and are accurate and up to date;
• It is important to plan work in advance and follow standard operating procedures as prepared and
controlled by the WHO Prequalification Evaluating Laboratory;
• To reduce the risk of adding an incorrect specimen to a test device/well, before starting the test
run, the operator will prepare worksheets and label all tubes, dilution vessels, test devices or
plates with the specimen’s unique number;
• Because objective, machine-generated, permanent results for simple/rapid diagnostic tests are
not feasible, it is essential that the PI emphasizes to the operator performing the tests the need
for accurate recording of results and recordkeeping;
• To allow immediate correction of erroneous recording of results (rather than differences in visual
interpretation), the PI or designee should assess the results as soon as possible to allow her/him
to return to the original test device to investigate apparently discordant readings;
Safety 6.3.
HIV, hepatitis B and hepatitis C and other viruses are transmissible by blood and body fluids.
Therefore, all types of specimens (including venous and capillary whole blood, serum/plasma, oral
fluid, etc.) must be handled as potentially infectious. Appropriate precautions to minimize infectious
hazards must be taken at all stages from the collection of specimens to the disposal of used materials
from the laboratory. The WHO Guidelines on HIV Safety Precautions, and Guidelines for the Safe
Transport of Specimens (WHO/EMC/97.3) and the WHO Prequalification Evaluating Laboratory
guidelines on laboratory safety should be followed carefully by the laboratory staff.
Storage of assays 6.4.
All reagents must be stored as indicated in the instructions for use. Some assays may not need
refrigeration. If refrigerated storage space is inadequate to store the entire test kit, they may be
divided so that labile reagents can be refrigerated separately from the non-labile supplies. Calibrated
thermometers are placed at each location where reagents and specimens are stored, i.e. ambient,
refrigerator and freezer. Temperatures are recorded daily on the WHO Prequalification Evaluating
Laboratory temperature logs. The lot numbers of the test kits received/used and their expiry dates are
recorded on the individual run worksheets
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 6 of 21
7. Study samples.
Residual fresh unlinked whole blood samples collected in Ethylene Diamine Tetra acetic Acid (EDTA)
vacutainer tubes submitted for routine CD4 enumeration will be used for the study. However, in those
laboratories where point-of-care CD4 technologies will be evaluated both venous and capillary blood
samples will be used. Efforts will done to include samples from both adults and children with CD4 + T cell
counts ranging from 50 to more than 1000 CD4 T cells/µl of blood in order to cover the clinical decision
range. Samples with higher CD4 values from children less than 6 years will be necessary to determine the
upper analytical limit of the various CD4 technologies being evaluated. Also efforts will be done to include
equal number of males and females, samples from patients with pulmonary tuberculosis, malaria and other
conditions which have been reported to influence CD4 counts. The clinical information including disease
conditions, age, sex and provisional diagnosis will be reported on the data collection form (Appendix 13.1).
Laboratories will be expected to strictly adhere to the 1997 revised Centres for Disease Control and
Prevention (CDC) guidelines for the performing CD4 T cell determinations. The samples will be transported
and maintained at room temperature (18-22oC) before testing unless otherwise specified by the
manufacturer.
8. Sample testing
The WHO Prequalification Evaluating Laboratory will participate in the laboratory testing. The selected
laboratories should have in-house, one of the single platform flow cytometry reference method which for
this study can either be FACSCalibur (BD) or Coulter EPIXS (Beckman Coulter); adherence to Good Clinical
and Laboratory Practice and quality management. The laboratories should have good track record in doing
flow cytometry for CD4+ T lymphocyte determinations and participation in an established external quality
assessment program. During the evaluation all laboratories will be provided with a single batch on CD4
controls which will be run daily and results reported.
The blood samples will be tested in all the technologies following agreed protocol in all the sites and
following the manufacturer's recommendations. In general, all samples will be tested within six to eight
hours of collection by all methods. The reference methods must be well calibrated and all quality control
results should be recorded. The antibody panels for the reference method must contain monoclonal
antibody combinations to accurately enumerate absolute and percent CD3, CD3+CD4+ and CD3+CD8+ T
lymphocytes. One such example is to include a panel with CD3/CD4/CD8/ CD45 monoclonal antibodies. The
stained samples should be analysed immediately after staining, otherwise, they must be stored at 4oC in
the dark and analysed within 24 hours of staining. Two copies of results will be made available from each
sample tested. The extra or scanned copy of the laboratory results will be attached to the clinical form and
sent to WHO Geneva at the end of the study. In those instruments which utilize printer paper which fade
on storage, photocopies of initialized and dated result slips will be made. A maximum of two randomly
selected dedicated CD4 technologies and one reference methods will be evaluated in one laboratory. All
dedicated assays will undergo intra-laboratory variation and agreement between methods in the reference
laboratory. At the end of the study, laboratories will store both the electronic and hard copies of the results
for a minimum of five years or according of local regulations.
Precision measurements 8.1.
Whenever applicable, precision measurements will include repeatability (intra assay variation, and run-to
run variability. Reproducibility will include inter assay variation (day-to-day variability) and inter-instrument
variation. In addition carry-over-studies will be done.
Intra assay variation
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 7 of 21
To assess intra assay variation will be done on twelve fresh venous whole blood samples. The samples will
be in three groups with 4 samples each in the range 150-250, 300-400 and 450-550 CD4+ T counts/µL. The
samples will be prepared ten times by only one operator, using the same test method and then analysed on
same cytometer. The mean, standard deviation (SD) and percentage coefficient of variation (% CV) will be
calculated and compared.
Run-to-run variation
Run-to-run variation will be assessed in both capillary and venous blood samples by preparing 15 fresh
blood specimen. The samples will be in three groups with 5 samples each in the range 150-250, 300-400
and 450-550 CD4+ T counts/µL. The prepared cartridge will be analysed ten times in a particular instrument
by one operator. The mean, SD and % CV of the obtained results will be calculated and compared
Inter assay variation (day to-day variation)
Inter assay variation will be done on twelve fresh venous whole blood samples. The samples will be in three
groups with 4 samples each in the range 150-250, 300-400 and 450-550 CD4+ T counts/µL. An aliquot from
the samples will be prepared by only one operator at three time points since collection to include 6, 24 and
48 hours and then analysed run on the same cytometer. The mean, SD and % CV of the obtained results will
be calculated and compared
Inter-instrument variation
Inter-instrument variability will be assessed by preparing and analysing 6 fresh venous blood and capillary
samples 10 times on each of the 3-5 analysers. The samples will be in three groups with two samples in the
range 150-250, two samples in the range 300-400 and two samples in the range 450-550 CD4+ T counts/µL.
For each sample, 10 cartridges will be prepared, and each cartridge will be read on each of the study
instruments. The five samples can be analysed by different operators in order to evaluated inter-operator
variability. The mean, SD and % CV of the obtained results will be calculated and compared.
The acceptability criteria for the four precision studies is shown in table 1 below
Carry-Over-studies
This will be measured by analysing five batches of two identical blood specimens with a high CD4 counts
greater than 600/µL (recorded as al and a2) followed by two identical blood specimens with CD4 counts
between 100 and 300/µL (which are recorded as b1 and b2).
(A1 A2 B1 B2). The carry-over (k) is expressed as described by Broughton.
%100b - a
b - b
22
21 k
The mean result should be the same for high-low and low-high sequences. With most automatic analysers,
carry-over is less than 2%.
Table 1 The optimal/desirable precision measurements criteria for dedicated and POC CD4
technologies
CD4 levels Coefficient
of variation
Standard
deviation
95% CI
Intra-assay variability ~200/µl <15% <30cells/µl 21, 55
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 8 of 21
~350/µl <10% <35cells/µl 24, 64
~500/µl <10% <50cells/µl 34, 91
inter-instrument variability ~200/µl <15% <30cells/µl 24, 40
~350/µl <10% <35cells/µl 28, 47
~500c/µl <10% <50cells/µl 40, 67
inter- assay variability ~200/µl <15% <30cells/µl 24, 40
~350/µl <10% <35cells/µl 28, 47
~500c/µl <10% <50cells/µl 40, 67
Trueness of measurement (Agreement between methods) 8.2.
Consecutive routine blood samples in EDTA vacutainer tubes with at least 3.0 ml of blood brought into the
laboratory (and capillary blood collected directly from the patient where applicable) will be used to
compare the methods. In each site a total of 200 samples of which 50 will have CD4+T cell range of <200/µL,
100 samples will have CD4 counts between 201 and 500/µL and the remaining 50 blood samples will
have >500 CD4+ T lymphocytes/µL will be stained by the test method and the reference method and run in
the respective instruments. Agreement between the dedicated and the reference method will be assessed
using the Bland Altman plots and/or percentage similarity methods. The overall bias should be less than 10%
with SD shown in Table 2 below
Table 2. The minimum and optimal performance criteria (bias) for instrument trueness of measurement.
Minimum Optimal
CD4 levels Number of
samples
Standard
Deviation
Width of
95% CI
Number of
samples
Standard
Deviation
Width of
95% CI
<200/µl 50 40cells/µl 23cells/µl 50 30 cells/µl 17cells/µl
200 -500/µl 100 50 cells/µl 20cells/µl 100 35 cells/µl 14cells/µl
>500c/µl 50 75 cells/µl 43cells/µl, 50 50 cells/µl 28cells/µl
Safety 8.3.
The testing of all specimens should be performed following WHO guidelines for good laboratory practice to
minimize occupational exposure risks. [For further details see Laboratory Biosafety Manual 3rd ed., Geneva,
World Health Organization, 2004 (ISBN 9241546506), Biosafety Guidelines for Diagnostic and Research
Laboratories Working with HIV, Geneva, World Health Organization, 1991 (WHO AIDS Series 9) and
Guidelines for the Safe Transport of Infectious Substances and Diagnostic Specimens, Geneva, World Health
Organization, 1997 (WHO/EMC/97.3) ].
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 9 of 21
9. Ethical considerations
The protocol will be submitted to WHO Ethical Review Committee, National and Institutional ethical review
boards in each participating country for ethical clearance. Residual routine patient blood samples which
have been submitted to the laboratories for enumeration of CD4+ T lymphocytes as part of routine
management will be used in the study. However in those sites where point-of-care CD4 technologies which
require capillary blood will be evaluated an informed consent will be obtained from the patients (Annex
13.3). Patient's results will be issued as per standard operating procedures (SOP) effective in each of the
study sites. The study will not interfere with the patients results as the study results will be unlinked. Age
and sex of the patient will be recorded in the study form and all other patient's identifiers will then be
removed from the sample tube, and given a new identification number. Results obtained from the CD4
technologies being evaluated will not be used for patient's management. The following clinical information
will also be collected from the patient's forms; patient's provisional clinical diagnosis, such tuberculosis,
malaria or HIV. It is presumed that when the above measures are taken it may not be necessary to request
informed verbal or written consent from the patients during blood collection and ethical review board will
be requested to waive the need for informed consent. The manufacturers of the CD4 technologies will be
provided with the protocol before the commencement of the study
10. Data management and Statistical analysis
Laboratory technologists performing the assay will also complete a standard form detailing its operational
characteristics relevant to resource limited settings (Annex 13.4), daily control log and adverse event log
(Annex 13.5). The individual laboratories will enter the data in the provided excel sheets under the
supervision of the Principal Investigator (Annex 13.6). A scanned copy of all data collection forms will be
also be sent to WHO/DLT Geneva for verification and further analysis. Data from individual laboratories will
be verified and combined into one data set for analysis at WHO HQ, Geneva.
Precision of the technology will be determined by calculation of the mean, standard deviation and
coefficient of variation as defined below.
The mean (x) is a measure of central tendency of a set of values.
The standard deviation (SD) is a measure of spread of a set of measurements about the mean.
The coefficient of variation (CV) expresses the standard deviation as a percentage of the mean.
Agreement between methods which is the estimates of analytical errors can be obtained by calculating
the bias using the modified Bland-Altman analysis and/or the “percentage similarity scatter plot” by
Scott et al.
The test method must demonstrate good precision as indicated in the respective section in the study
design above.
11. Time frame
The evaluations will be expected to be completed within 3 months.
12. Communications
Principal Investigators of the participating laboratories will be in constant communication with the WHO
HQ PQ Diagnostics Assessment team in Geneva to follow-up the progress during the whole process of
evaluation. The preliminary report on the CD4 technologies will be shared with individual manufactures,
who will be required to file any disagreements within one month. The results will be presented in
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 10 of 21
scientific forum, published in international scientific journal and will also be posted on the WHO/DLT
website. The prequalified products will also be included in the WHO WebBuy.
13. Responsibilities of the partners.
Manufacturers of the CD4 technologies 13.1.
Will provide the instruments and appropriate number of kits (about 500 tests per site) free-of-charge
for the study;
Ensure that they ship the instruments to the testing laboratory under appropriate conditions and in
time for the commencement of the evaluation;
Ensure that all test kits have at least 6 months remaining shelf life at the start of the evaluation.
Storage conditions during transport must be recorded;
Carry out training on the use of the instruments for the laboratory staff carrying out the evaluation;
Ensure prompt instrument service when there is breakdown.
Agree to sign a contract provided by WHO.
Respond to WHO their concerns regarding the evaluation results of their technologies within one
month.
WHO Prequalification Testing Laboratory 13.2.
Sign WHO contract for performance of work
Declare any conflict of interest
Will submit the agreed protocol to the national bodies for ethical clearance ;
Liaison with manufacturer regarding appropriate date(s) for delivery and installation of the instruments
and associated test kits;
Ensure all specimens are properly processed as specified in the protocol including strict quality
assurance for all the methodologies under evaluation;
Ensure that results are stored securely;
Ensure that instruments are evaluated according to the protocol;
Ensure that testing records are complete and accurate;
Fill the patient information form and attach copies of results from the instrument printer have been
initialized and dated;
Perform data entry and ease of use of the technology;
Ensure confidentiality of the result.
Diagnostics and Laboratory Technologies 13.3.
Submit the protocol to WHO ethical clearance committee;
Finalize formal contracts with CD4 companies and the participating laboratories;
Will be the body which is allowed to communicate with the companies once the study start
Provide technical and administrative support of the project;
Ensure availability of funds for procurement and delivery of all essential supplies required for the
reference method;
Supervise preparation and dissemination of the final report;
Perform final analysis and publish the results.
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 11 of 21
14. References
1. CDC. 1997 Revised guidelines for performing CD+ T- cell determinations in persons infected with Human
Immunodeficiency Virus (HIV) MMWR 1997;6 (RR-2):-29.
2. NCCLS, Evaluation of Precision Performance of Quantitative Measurement Methods; Approved
Guideline— Second Edition EP5-A2
3. Chesher D. Evaluating assay precision. Clinical Biochemistry Review. 2008 Aug;29 Suppl 1:S23-6.
4. Broughton PM, Gowenlock AH, McCormack JJ, Neill DWA. Revised scheme for the evaluation of
automatic instruments for use in clinical chemistry. Ann Clin Biochem. 1974 Nov;11(6):207-18.
5. Martin Bland J., Douglas G. Altman. Statistical methods for assessing agreement between two methods
of clinical measurement. Lancet, 1986; i: 307-310
6. Scott LE, Galpin JS, Glencross DK. Multiple method comparison: statistical model using percentage
similarity. Cytometry B Clinical Cytometry. 2003 Jul;54(1):46-53.
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 12 of 21
13. Annexes
13.1. Patient clinical information form
Performance evaluation of dedicated CD+ T lymphocytes enumeration technologies
Name of the evaluation site ………………………………………………………….
Date …………………………
Sample ID …………………………
Time of collection o blood sample …………………………
1. Age of the patient (years) ………………………………………..
2. Sex ………
3. Provisional Diagnosis ………………………………………..
4. Current patient medication ………………………………………..
a. Patient on ARV YES NO UNKNOWN
5. Patient has TB YES NO NO UNKNOWN
6. Patient has Malaria YES NO UNKNOWN
Initials of the technologist Initials of the validating PI
____________________ ____________________
Date Date
________________ __________________
Note: Please attach a copy of the print out from the CD4 instrument
MALE
FEMALE
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 13 of 21
13.2. Summary of the experimental protocol
*INTRA
LABORAT
ORY
*Sample
type
Sample
size
CD4 T-
cell
strata
Details Remark
Assay
precision
Intra
Assay
Tube-
to-
Tube
Venous 4 150-250 10
cartridges/sa
mple, each
sample on a
single
instrument
1 out of the 4 samples
acceptable on a 75-cell
distance from threshold in
place of 50 (125-275 for 200;
275-425 for 350, and 425-575
for 500)
4 300-400
4 450-550
Total=12
Inter
Assay
Assay-
to-
Assay
Venous 4 150-250 6 hours, 24
hours, 48
hours
1 out of the 4 samples
acceptable on a 75-cell
distance from threshold in
place of 50 (125-275 for 200;
275-425 for 350, and 425-575
for 500)
4 300-400
4 450-550
Total=12
Instrumen
t
precision
Run-
to-
Run
Venous
and
Capillary
5 150-250 10 runs of one
cartridge (on
the same
instrument)
1 out of the 5 samples
acceptable on a 75-cell
distance from threshold in
place of 50 (125-275 for 200;
275-425 for 350, and 425-575
for 500)
5 300-400
5 450-550
Total=15
Inter-
instru
ment
Venous
and
Capillary
2 150-250 10
cartridges/sa
mple, each
sample on
each of the 4
instruments (3
in study + 4th
in stock)
2 300-400
2 450-550
Total=6
Carry-
over
Carry-
over
Venous 5 100-300 Sequence
High-high-
low-low
* May not be applicable in
most of POC assays
5 >600
Total=10
INTER
LABORAT
ORY
Inter
lab
Venous
and
Capillary
50 <200 Run in the
POC and
reference
method
100 201-500
50 >500
Total 200
*Sample types and type of studies will depend on the type of technology to be evaluated
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 14 of 21
13.3. Informed Consent Form
[YOUR INSTITUTIONAL LETTERHEAD]
Informed Consent for Collection of Blood in a multi-centre evaluation of dedicated and point of care CD4
technologies
[Name of Principle Investigator] ………………………………………………..
[Name of Principal Investigator] ………………………………………………..
[Name of Organization] ………………………………………………..
[Name of Sponsor] World Health Organization
This Informed Consent Form has two parts:
1. Information Sheet (to share information about the research with you)
2. Certificate of Consent (for signatures if you agree to take part)
You will be given a copy of the full Informed Consent Form
PART I: Information Sheet
Introduction
I am …….., working for ………… [Name of the hospital]. We are doing an evaluation of instruments used in
monitoring patients with human immunodeficiency virus (HIV) infection, which is very common in this
country. I am going to give you information and invite you to participate in this research. You do not have
to decide today whether or not you are willing to participate in the research. Before you decide, you can
talk to anyone you feel comfortable with about the research. There may be some words that you do not
understand. Please ask me to stop as we go through the information and I will take time to explain. If you
have questions later, you can ask me, the study staff.
Purpose of the research
HIV is one of the most common diseases in this region. It is important for health care providers to
determine the number of specific blood cells in order to accurately determine the time to initiate various
treatment options and also monitor response to treatment. Specific instruments are required to count
these blood cells. We are doing this research to evaluate the ability of newly developed instruments to
accurately count the cells.
Type of Research Intervention
This research will involve measuring blood from various patients in two or more instruments and then
compare the results.
Participant selection
We are inviting all patients attending this clinic to participate in the research to evaluate the performance
of the instrument. Your participation in this research is entirely voluntary. It is your choice whether to give
a blood sample or not. Whether you choose to participate or not, all the services you receive at this clinic
will continue and nothing will change. You may change your mind later and stop participating even if you
agreed earlier.
Procedures and Protocol
About three able spoons of blood will be collected from the arm using a syringe. In addition we will take a
drop of blood from your finger tip. We will take your blood only once.
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 15 of 21
Risks
We do not expect that any harm will happen to you because of joining this study. Sometimes, you may feel
some little pain and a small bruise may occur due to the finger prick.
Benefits
If you participate in this research, you will not have immediate individual benefits but your participation is
likely to help us ensure availability of better and cheaper service to patients in the future.
Reimbursements
You will not be given any money or gifts to take part in this research.
Confidentiality
The information that will be collected from this research project will be kept confidential. Information
about you that will be collected during the research will be put away and no-one but the researchers will be
able to see it. Any information about you will have a number on it instead of your name. Only the
researchers will know what your number is and we will lock that information up with a lock and key. It will
not be shared with or given to anyone except [name] who will have access to the information, such as
research sponsors, etc.)
Right to Refuse
You do not have to take part in this research if you do not wish to do so and refusing to participate will not
affect your treatment at this clinic in any way. You will still have all the benefits that you would otherwise
have at this clinic.
Who to Contact
If you have any questions you may ask them now or later, even after the study has started. If you wish to
ask questions later, you may contact any of the following: [name, address/telephone number/e-mail]
This proposal has been reviewed and approved by [name of the local IRB], which is a committee whose task
it is to make sure that research participants are protected from harm. If you wish to find about more about
the IRB, contact [name, address, telephone number.]. It has also been reviewed by the Ethics Review
Committee of the World Health Organization (WHO), which is funding/sponsoring/supporting the study.
You can ask me any more questions about any part of the research study, if you wish to. Do you have any
questions?
PART II: Certificate of Consent
I have read the foregoing information, or it has been read to me. I have had the opportunity to ask
questions about it and any questions that I have asked have been answered to my satisfaction. I consent
voluntarily to participate as a participant in this research.
Name of Participant__________________
Signature of Participant ___________________
Date ___________________________ (Day/month/year)
If illiterate
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 16 of 21
I have witnessed the accurate reading of the consent form to the potential participant, and the individual
has had the opportunity to ask questions. I confirm that the individual has given consent freely.
Name of witness_____________________ AND Thumb print of participant
Signature of witness ______________________
Date ________________________ (Day/month/year)
Statement by the researcher/person taking consent
I have accurately read out the information sheet to the potential participant, and to the best of my ability
made sure that the participant understands that the following will be done:
1. We will collect few mls of blood using venepuncture
2. We will also collect capillary blood using the finger tip
I confirm that the participant was given an opportunity to ask questions about the study, and all the
questions asked by the participant have been answered correctly and to the best of my ability. I confirm
that the individual has not been coerced into giving consent, and the consent has been given freely and
voluntarily. A copy of this ICF has been provided to the participant.
Name of Researcher/person taking the consent________________________
Signature of Researcher /person taking the consent__________________________
Date ___________________________ (Day/month/year)
13.4. Assessment of operational characteristics of CD4 Technology
Operational characteristic Options Yes/No
Characteristics of the technology
Technology format
Portable bench top instrument
Hand-held instrument
Instrument free test
Instrument Calibration Auto-calibration or calibration not required
Portability Transportable by back pack (<10Kg)
Transportable by hand (<2 Kg)
In-built software Present (for instrumentation)
Data capture capability (instrument free)
Instrument Connectivity Universal communication protocol (USB)
Wireless Data transfer
Power Source (when needed or
applicable)
Mains and rechargeable battery
Rechargeable battery
Alternate Power Sources Solar energy, car battery
Lockout instrument features Administrative control of user, expired test lots, error
(critical system check failure shutdown, internal QC
failure,)
Waste
Minimal solid waste (biohazard material contained)
Minimal liquid and solid waste
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 17 of 21
Result Display
Permanent record (displayed on the device and
printed)
Electronic captured result
Result Storage Results stored in the instrument and there is an
interface
External Equipment(s) Not required
Frequency of breakdown/
blockage over the study period (1-
3 months)
None
1-5
>5
Technology output characteristics
Assay output CD4 counts
CD4 counts and percentage
Multiplex capability
Analytical range for quantitative
variables
0-1000 CD4 cell/µl
0-2000 CD4 cells/µl
Sample throughput 5-15 per day per instrument per operator
Turn-around-time ~30 minutes
<10 minutes
Quality control
Internal Quality Control available Yes
Assay run controls available Yes
External Quality Control(EQA Compatible with a recognised EQA samples
System check (instrument) Available
Reagents
Instruction for use Concise and clear
Heat Stability of Reagents and
Controls
Less than 30oC for minimum of 12 months
Max 40oC degrees for minimum of 6 months
Shelf life of reagents upon
manufacture
12 Months at room temperature (18-25oC)
>12 Months
Reagent & Control Preparation Not required
Sample preparation and analysis
Type of Sample Venous blood
Capillary blood
Venous and capillary blood
Sample collection Not require specialised tube
No need of tubes
Sample Labelling options Manual label
Bar code reader
Precise Sample Measurement No manual pipetting of precise quantity required
Specimen volume required Maximum 100µl
<50ul
Stability of the stored blood Can be tested up to 8 hours
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 18 of 21
specimen Can be tested up to 48 hours
Stability after staining/adding to
cartridge/devise
0 – 8 hours
Sample failure rates Failure rates <5%
Failure rate <3%
Regulatory and operational issues
Service and Maintenance Easily available in-country/region
Rent or lease agreement Available
Capital Cost of Equipment Less than $1,000
Reagent, Consumables, controls
cost
Less than $5 cost per test
Connectivity cost Included in cost per test/equipment
Operator skills required Trained health care worker
Installation Do not require Vendor or Technician
Operating environment Wide operating range for temperature humidity
altitude, dust
Preventative Maintenance All routine maintenance by Operator
No routine maintenance
Training Maximum 1 day if formal training required
<0.5days maximum if formal training required
Supply Chain and Distribution Easily available in-country/region
Test availability and Regulatory
Status
Commercially available and approved in-
country/region
Best suited for Health centre/Dispensary
13.5. Adverse events reporting log
WHO multicentre evaluation of CD4 technologies. Adverse event reporting log
Name of the laboratory____________________________________________
Name of the instrument___________________________________________
Date Details of the event Corrective action
done
Lab technician
initial and date
Supervisor
initials and
date
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 19 of 21
13.6. Data collection template (Using Excel sheet)
INTRA-ASSAY VARIATION
Sample Aliquots
CD3
counts
CD3
%
CD4
counts
CD4
%
CD8
counts
CD8
%
A1
A2
A3
A4
A5
A6
A7
A8
A9
INTER ASSAY VARIATION COUNTS
SIX HOURS 24 HOURS 48 HOURS
Sample
CD3
counts
CD4
counts
CD8
counts
CD3
counts
CD4
counts
CD8
counts
CD3
counts
CD4
counts
CD8
counts
B1
B2
B3
B4
B5
B8
B9
B10
INTER ASSAY VARIATION PERCENTAGES
SIX HOURS 24 HOURS 48 HOURS
sample CD3% CD4% CD8% CD3% CD4% CD8% CD3% CD4% CD8%
B1
B2
B3
B4
B5
B8
B9
B10
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 20 of 21
Precision
CD3
counts
CD3
%
CD4
counts
CD4
%
CD8
counts
CD8
%
1st run
2nd run
3rd run
4th run
5th run
6th run
7th run
8th run
Carry over
Sequence
CD3
counts
CD3
%
CD4
counts
CD4
%
CD8
counts
CD8
%
3
3
1
1
3
3
3
2
2
2
1
1
1
AGREEMENT WITH THE REFERENCE METHOD (For each method)
CD3
counts
CD3
%
CD4
counts
CD4
%
CD8
counts
CD8
%
1
2
3
4
5
6
7
11
74
WHO Protocol for Performance Evaluation of Lymphocyte subsets Enumeration Technologies WHO PQDX_030 v5 February 2017
WHO/EMP/RHT/PQT/2017.xx Page 21 of 21
QUALITY CONTROL RESULTS (For each method)
Dates
CD3
counts
CD3
%
CD4
counts
CD4
%
CD8
counts
CD8
%