WHO WHO EQAP for the detection of iEQAP for the detection of influenza A virus subtype by PCRnfluenza A virus subtype by PCR

Centre for Health ProtectionCentre for Health Protection

Hong Kong SAR, ChinaHong Kong SAR, China

Aims/objectivesAims/objectives

To monitor quality and standards of To monitor quality and standards of performanceperformance

To facilitate information exchangeTo facilitate information exchange To identify problems with assaysTo identify problems with assays To provide mechanisms to remedy To provide mechanisms to remedy

any deficiencies revealedany deficiencies revealed

Benefits to participantsBenefits to participants

Able to Able to • compare performancecompare performance• provide evidence of qualityprovide evidence of quality• minimize errorsminimize errors• identify training if neededidentify training if needed

MethodologyMethodology

TargetsTargets• mostly NICs and some non-NICsmostly NICs and some non-NICs

MaterialsMaterials• dried RNA of H5, H1, H3, H1v virusesdried RNA of H5, H1, H3, H1v viruses

ScheduleSchedule• twice a yeartwice a year

Data analysisData analysis• qquestionnairesuestionnaires on on

testing strategytesting strategy test test methodologymethodology good laboratory practice (GLP)good laboratory practice (GLP)

Distribution of panels and response Distribution of panels and response of participantsof participants

WHOWHORegionRegion

No. of laboratoriesNo. of laboratories

InvitedInvited Received samplesReceived samples Reported resultsReported results

P 1P 1 P 2P 2 P 3P 3 P 4P 4 P 5P 5 P 1P 1 P 2P 2 P 3P 3 P 4P 4 P 5P 5 P 1P 1 P 2P 2 P 3P 3 P 4P 4 P 5P 5

AFRAFR 1010 1111 1212 1919 1919 66 88 99 1616 1717 66 66 88 1313 1717

AMRAMR 2626 2828 2828 2727 2727 66 1616 1919 2222 2424 55 1414 1616 2121 2323

EMREMR 88 99 99 1010 1010 22 55 66 77 88 22 44 66 66 77

EUREUR 5050 5151 5353 5252 5757 3535 4040 4343 4545 4848 3434 3939 4343 4545 4545

SEARSEAR 99 99 1010 88 88 33 44 55 66 66 33 44 55 55 66

WPRWPR 1919 2121 2020 2121 2121 1515 1717 1717 1919 1818 1414 1616 1717 1919 1616

TotalTotal 122122 129129 132132 137137 142142 6767 9090 9999 115115 121121 6464 8383 9595 109109 114114

AFRO, AMRO, EMRO, EURO

SEARO, WPRO

RResponse of participantsesponse of participants

28 30 30 29 29 30

94 99 102 108 113 119

0102030405060708090

100110120130140150160

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

No. invited

No. received

18 21 22 25 24 28

49

69 7790 97

104

0102030405060708090

100110120130140

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

No. reported

17 20 22 24 22 22

47

6373

85 92 83

0102030405060708090

100110120

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

Reasons for laboratories not Reasons for laboratories not receiving panelsreceiving panels

Problem

No. (%) of laboratories

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5

(N=122) (N=129) (N=132) (N=137) (N=142)

No response 38 (31) 26 (20) 20 (15) 11 (8) 9 (6)

Unwilling to participate 5 (4) 7 (5) 7 (5) 10 (7) 9 (6)

Import permit or logistical problems 12 (10) 6 (5) 6 (5) 1 (1) 3 (2)

Panel Contents

No. of samples in the panel

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

2007 2007 2008 2008 2009 2009

Feb-Mar Aug-Oct Jan-Feb Jun-Jul Jan-Feb Jun-Aug

H5 sample:

- clade 1 2 1 1 2

- clade 2.1 2 2 1 1

- clade 2.2 2 2 1 1

- clade 2.3.2 2 2 2

- clade 2.3.4 2 2 1 1 2

H1 sample 1 1 1 1 2

H3 sample 1 1 1 1 1

H1v sample

Negative sample 2 4 2 2 1

Total 10 14 10 10 10 10

Composition of panelsComposition of panels

Conventional

Real-time

2020

4040

6060

8080

Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5

H5 PCRH5 PCR

2020

4040

6060

8080

Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5

H1 PCRH1 PCR

2020

4040

6060

8080

Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5

H3 PCRH3 PCR

Method of detectionMethod of detection(% of participants)(% of participants)

Nucleic acid amplification testsNucleic acid amplification tests

Most were developed in-houseMost were developed in-house• the primes/probes:the primes/probes:

most commonly adapted from other most commonly adapted from other researchersresearchers

minority were own designedminority were own designed

Minority were commercial kitsMinority were commercial kits

Assessment criteriaAssessment criteria

The performance of individual laboratories was The performance of individual laboratories was assessed by adding up the number of correct assessed by adding up the number of correct results.results.

Incorrect responses:Incorrect responses:• failing to detect H5 samples and/or reporting the results failing to detect H5 samples and/or reporting the results

as non-H5 subtypeas non-H5 subtype• failing to detect H1 samples and/or reporting the results failing to detect H1 samples and/or reporting the results

as non-H1 subtypeas non-H1 subtype• failing to detect H3 samples and/or reporting the results failing to detect H3 samples and/or reporting the results

as non-H3 subtypeas non-H3 subtype• failing to report correct influenza A test resultsfailing to report correct influenza A test results for H1/H3 samfor H1/H3 sam

ples if H1/H3 subtyping was notples if H1/H3 subtyping was not performedperformed• reporting positive results for a sample that did not reporting positive results for a sample that did not

contain any viral RNAcontain any viral RNA

Performance of laboratoriesPerformance of laboratories

Performance

No. (%) of laboratories

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5

(N=64) (N=83) (N=95)(N=109

)(N=114

)

All samples correct 43 (67) 54 (65) 70 (74) 84 (77) 87 (76)

All but 1 sample correct 6 (9) 14 (17) 10 (11) 11 (10) 12 (11)

50–89% of samples correct 9 (14) 12 (14) 11 (12) 10 (9) 14 (12)

<50% of samples correct 6 (9) 3 (4) 4 (4) 4 (4) 1 (1)

Testing errors made by laboratoriesTesting errors made by laboratories

Comparison factors

No. (%) of laboratories

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5

(N=64) (N=83) (N=95) (N=109) (N=114)

Incorrect H5 results 15(23

) 17

(20)

17(18

)13

(12)

21 (18)

False-positive results

9(14

) 5 (6) 2 (2) 6 (6) 1 (1)

Overall performanceOverall performance

1010

2020

3030

4040

5050

6060

7070

8080

9090

100100

AFRAFR AMRAMR EMREMR EUREUR SEARSEAR WPRWPR TotalTotal

Panel 1Panel 1 Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5

H5 performanceH5 performance

1010

2020

3030

4040

5050

6060

7070

8080

9090

100100

AFRAFR AMRAMR EMREMR EUREUR SEARSEAR WPRWPR TotalTotal

Panel 1Panel 1 Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5

AFRO, AMRO, EMRO, EURO

SEARO, WPRO

PerformancePerformance

% of all correct % of H5 all correct

59 60

77

8386

82

55

65

75

85

95

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

7175

86 88 8682

55

65

75

85

95

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6

Problems identified in EQAPProblems identified in EQAP

Positive control not used appropriatelyPositive control not used appropriately Lab contaminationLab contamination Misinterpretation of resultsMisinterpretation of results Primers and probes mismatchPrimers and probes mismatch

Primer/Probe name Primer/Probe Sequence 5' to 3' Length Position Dir JournalH5 + 1456 Primer ACG TAT GAC TAT CCA CAA TAC TCA G 25 1512-1536 F J Clin Microbiol 2002 40 3256-3260H5 - 1685 Primer AGA CCA GCT ACC ATG ATT GC 20 1663-1644 R J Clin Microbiol 2002 40 3256-3260H5 + 1637 Probe TCA ACA GTG GCG AGT TCC CTA GCA 24 1617-1640 F J Clin Microbiol 2002 40 3256-3260

H5Forward Primer GCC GAA TGA TGC MAT MAA YT 20 758-777 F J Clin Microbiol 2007 45 1535-1543H5Reverse Primer CGC ACC CAT TGG AGT TTG AC 20 908-889 R J Clin Microbiol 2007 45 1535-1543H5probe Probe CAT TGC TCC AGA AWA T 16 797-812 F J Clin Microbiol 2007 45 1535-1543

H5-943 Primer GCC ACT CCA CAA TAT ACA CCC 21 923-943 F Lancet 1998 351 467-471H5-1300 Primer CAA ATT CTC TAT CCT CCT TTC CAA 24 1285-1263 R Lancet 1998 351 467-471

HA-1144 Primer GGA ATG ATA GAT GGN TGG TAY GG 23 1092-1114 F WHO (2002)H5-1735R Primer GTG TTT TTA AYT AMA ATC TGR ACT MA 26 1746-1721 R WHO (2002)

H5-1 Primer GCC ATT CCA CAA CAT ACA CCC 21 923-943 F WHO (2007), CHP, Hong Kong SARH5-3 Primer CTC CCC TGC TCA TTG CTA TG 20 1141-1122 R WHO (2007), CHP, Hong Kong SAR

H5-266F Primer TGC CGG AAT GGT CTT ACA TAG TG 23 274-296 F WHO (2007), CHP, Hong Kong SARH5-1615F Primer GTG GCG AGC TCC CTA GCA 18 1623-1640 F WHO (2007), CHP, Hong Kong SARH5-347R Primer TCT TCA TAG TCA TTG AAA TCC CCT G 25 355-331 R WHO (2007), CHP, Hong Kong SARH5-1695R Primer TCT GCA TTG TAA CGA CCC ATT G 22 1703-1682 R WHO (2007), CHP, Hong Kong SARH5-290P Probe AGA AGG CCA ATC CAG TCA ATG ACC TCT GTT A 31 298-328 F WHO (2007), CHP, Hong Kong SARH5-1634P Probe TGG CAA TCA TGG TAG CTG GTC TAT CCT TAT GG 32 1642-1673 F WHO (2007), CHP, Hong Kong SAR

H5-248-270F Primer GTG ACG AAT TCA TCA ATG TRC CG 23 256-278 F WHO (2007), NIID, JapanH5-671-647R Primer CTC TGG TTT AGT GTT GAT GTY CCA A 25 679-655 R WHO (2007), NIID, Japan

H5HA-205-227v2-For Primer CGA TCT AGA YGG GGT GAA RCC TC 23 182-204 F WHO (2007), NIID, JapanH5HA-326-302v2-Rev Primer CCT TCT CCA CTA TGT ANG ACC ATT C 25 303-279 R WHO (2007), NIID, JapanH5-Probe-239-RVa Probe AGC CAY CCA GCT ACR CTA CA 20 238-219 R WHO (2007), NIID, JapanH5-Probe-239-RVb Probe AGC CAT CCC GCA ACA CTA CA 20 238-219 R WHO (2007), NIID, Japan

RF 1151 Primer GGA ACT TAC CAA ATA CTG TCA ATT TAT TCA 30 1590-1619 F WHO (2007), EMC, the NetherlandsRF 1152 Primer CCA TAA AGA TAG ACC AGC TAC CAT GA 26 1673-1648 R WHO (2007), EMC, the NetherlandsRF 1153 Probe TTG CCA GTG CTA GGG AAC TCG CCA C 25 1647-1623 R WHO (2007), EMC, the Netherlands

PCR7ModMMForward Primer GCC GAA TGA TGC MAT MAA YT 20 758-777 F HPA VSOP41PCR7Reverse Primer CGC ACC CAT TGG AGT TTG AC 20 908-889 R HPA VSOP41PCR7degen Probe CAT TGC TCC AGA AWA T 16 797-812 F HPA VSOP41

H5VietFor Primer GGA TGG CAG GGA ATG GTA GA 20 1083-1102 F HPA VSOP46H5VietRev Primer TCT ATT GCC TTT TGA GTG GAT TCT T 25 1183-1159 R HPA VSOP46H5VietProbe Probe TGG GTA CCA CCA TAG CAA YGA GCA GG 26 1112-1137 F HPA VSOP46

The sequences of the H5 primers/probes were obtained from PubMed and WHO website.

The positions of the oligonucleotides are based on the HA gene of A/HongKong/156/1997(H5N1), GenBank accession number AF046088.

Dir, direction; F, forward; R, reverse

The primers/probes sequences of the H5 gene used by participants

The most widely adopted H5 PCR The most widely adopted H5 PCR primers/probesprimers/probes

PanelPanel

No. of laboratoriesNo. of laboratories

reportedreportedresultsresults

performedperformedH5 subtypingH5 subtyping

adopted CDCadopted CDCprimers/probesprimers/probes

with incorrectwith incorrectH5 resultsH5 resultsaa

NN NN %%bb NN %%cc NN %%dd

Panel 2Panel 2 8383 8181 9898 1717 2121 33 1818

Panel 3Panel 3 9595 9494 9999 2929 3131 44 1414

Panel 4Panel 4 109109 108108 9999 4444 4141 22 55

Panel 5Panel 5 114114 114114 100100 4747 4141 55 1111

aa Incorrect results were due to either false-positive, false negative or any non-H5 results reported in H5 samples. Incorrect results were due to either false-positive, false negative or any non-H5 results reported in H5 samples.

bb Percentages are based on the number of laboratories reported results. Percentages are based on the number of laboratories reported results.

cc Percentages are based on the number of laboratories performed H5 subtyping. Percentages are based on the number of laboratories performed H5 subtyping.

dd Percentages are based on the number of laboratories adopted CDC primers/probes. Percentages are based on the number of laboratories adopted CDC primers/probes.

Factors affecting H5 performanceFactors affecting H5 performance

PanelPanelTechnical factorTechnical factoraa

Real-time assayReal-time assay Commercial kitCommercial kit TAT > 28 daysTAT > 28 days

Panel 2Panel 2 0.0020.002 0.6780.678 Not applicableNot applicable

Panel 3Panel 3 0.0060.006 0.6370.637 0.0020.002

Panel 4Panel 4 0.5650.565 0.1880.188 0.9450.945

Panel 5Panel 5 0.0060.006 0.3480.348 0.0220.022

a a PP values were calculated by Yates-corrected chi-square test. values were calculated by Yates-corrected chi-square test.

H1v performance in Panel 6H1v performance in Panel 6

AFRO

AMRO

EMRO

EURO

SEARO

WPRO

% of H1v all correct

One participant each in AFRO, AMRO and EMRO reported incorrect H1v subtyping results.

80

85

90

95

100

Panel 6

Source No.* %

Centres for Disease Control and Prevention 71 78

Institut Pasteur, Paris, France 5 5

Robert Koch-Institute 4 4

Health Protection Agency, United Kingdom 3 3

Other 19 sources 19 1

H1v primers/probes adopted H1v primers/probes adopted byby 91 91 participants in Panel 6participants in Panel 6

* More than one set of primers/probes were used by 10 participants, the total number is larger than 91.

GLPGLP SurveySurvey

2007 survey c2007 survey composed of 73 questionsomposed of 73 questions 2008 survey composed of 25 questions2008 survey composed of 25 questions Both surveys composed questions Both surveys composed questions on the following on the following

seven categories:seven categories:• personnelpersonnel• quality managementsquality managements• design, equipment and consumablesdesign, equipment and consumables• pre-analytical procedurespre-analytical procedures• analytical proceduresanalytical procedures• post-analytical procedurespost-analytical procedures• reporting and record keepingreporting and record keeping• safetysafety

Parameter

GLP in

2007 2008

N n % N n %

Quality management

- Laboratory has a continuous improvement programme 84 68 81 92 85 92

- Laboratory is accredited by national/international laboratory accreditation organizations

85 53 62 92 29 32

Facility design

- Laboratory has separate rooms for:

- sample preparation 84 67 80 94 93 99

- reagent preparation 84 80 95 94 92 98

- amplification and product detection 84 83 99 94 92 98

- Laboratory has documented policy requiring a unidirectional workflow

84 70 83 94 64 68

Quality management and facility design

Parameter

GLP in

2007 2008

N n % N n %

Examination procedures

- Laboratory has SOP for the following individual testing procedure:

- viral RNA extraction 84 80 95 92 90 98

- preparation of in-house controls 77 49 64 92 48 63

- Controls included when performing molecular diagnostic tests:

- positive control 82 78 95 93 91 98

- negative control 79 74 94 94 93 99

Post-examination procedures

- Laboratory has worksheet to record:

- the date of testing 85 84 99 91 91 100

- the reagent expiry dates 84 59 70 91 59 65

- Laboratory has another staff to countercheck the test results 85 59 69 94 67 71

Examination procedures and post-Examination procedures and post-examination proceduresexamination procedures

Factors affecting performance Factors affecting performance based on GLP analysisbased on GLP analysis

Lab did not return all correct results tends to Lab did not return all correct results tends to meet less quality parameters; significantly more meet less quality parameters; significantly more likely (p < 0.05) not havinglikely (p < 0.05) not having- audits of personnel- audits of personnel- separate rooms for all steps involving PCR- separate rooms for all steps involving PCR

- programme to monitor equipment- programme to monitor equipment- 100 samples in recent representative month- 100 samples in recent representative month- SOP on preparation of in-house controls- SOP on preparation of in-house controls- established test turn-around-time- established test turn-around-time

Way forwardWay forward

Regularly review results of EQAPRegularly review results of EQAP Identify problems related to test Identify problems related to test

protocolsprotocols Enhance performance through trainingEnhance performance through training

Thank YouThank You