WHO EQAP for the detection of WHO EQAP for the detection of influenza A virus subtype by PCRinfluenza A virus subtype by PCR

Wilina LimWilina Lim

Centre for Health ProtectionCentre for Health Protection

Hong Kong SAR, ChinaHong Kong SAR, China

Aims/objectivesAims/objectives

To monitor quality and standards of To monitor quality and standards of performanceperformance

To facilitate information exchangeTo facilitate information exchange To identify problems with assaysTo identify problems with assays To help develop testing strategiesTo help develop testing strategies To provide mechanisms to remedy any To provide mechanisms to remedy any

deficiencies revealeddeficiencies revealed Requirement of lab accreditation in Requirement of lab accreditation in

accordance with international standard accordance with international standard such as ISO 15189such as ISO 15189

Benefits identified by NICs

To monitor feasibility to ship samples to

countries laboratory capability timeliness of reporting

1A. Invitation

2.2. Preparation of PanelsPreparation of Panels

4.4. Data CollectionData Collection

3.3. Panel DistributionPanel Distribution

5.5. Preliminary reportPreliminary report

6.6. Data Analysis Data Analysis

7.7. Final reportFinal report

QAP Process

QAP Process

QAP Process

QAP Process

1B. New Participants

The EQAP has beenaccredited in accordanceWith ISO 17043

Preparation of panelsPreparation of panels

Include different subtypes/cladesInclude different subtypes/cladesDried RNA of influenza AH5, H1, H3, H1v and influenza B virusesDried RNA of influenza AH5, H1, H3, H1v and influenza B virusesGamma-ray inactivated seasonal influenza samplesGamma-ray inactivated seasonal influenza samples

Verify sample contentVerify sample content

Verify sufficient homogeneityVerify sufficient homogeneitySamples in final packaged form selected and tested Samples in final packaged form selected and tested

Assure sufficient stabilityAssure sufficient stability

Test over a range of storage conditions prior to distributionTest over a range of storage conditions prior to distributionSamples were tested after 7 days of storage at 37Samples were tested after 7 days of storage at 37ooC using both C using both conventional and real-time PCR assays conventional and real-time PCR assays

Temperature surveyTemperature survey Monitor the temperature change during shipmentMonitor the temperature change during shipment Target participants Target participants

• With temperature record higher than 37℃ during sample With temperature record higher than 37℃ during sample dispatch perioddispatch period

• Panel 9: Jan-Mar 2011Panel 9: Jan-Mar 2011

RegionRegionNo. of logger No. of logger

sentsentNo. of logger No. of logger

returnedreturned

AFROAFRO 55 11

AMROAMRO 33 22

EMROEMRO 22 22

WPROWPRO 33 22

TotalTotal 1313 77

Temperature survey on EQAP panel 9 shipmentTemperature survey on EQAP panel 9 shipment

0

24

48

72

96

120

144

168

Lab A Lab B Lab C Lab D Lab E Lab F

Ship

men

t tim

e (h

r)

above 37℃

25℃-37℃

below 25℃

Longest duration with temperature higher than 37 was 6 hours℃

Lab D

0

10

20

30

40

50

60

Destination time

Tem

pera

ture

(℃

)

Duringshipment

Delivered

Temperature above 37℃ were recorded during the transit in daytime

Lab B

0

10

20

30

40

50

60

Destination time

Tem

pera

ture

(℃) During

shipmentdelivered

Panel contentsPanel contents

No. of samples in the panelNo. of samples in the panel

Panel 1Panel 1 Panel 2Panel 2 Panel 3Panel 3 Panel 4Panel 4 Panel 5Panel 5 Panel 6Panel 6 Panel 7Panel 7 Panel 8Panel 8 Panel 9Panel 9

20072007 20072007 20082008 20082008 20092009 20092009 20102010 20102010 20112011

Feb-MarFeb-Mar Aug-OctAug-Oct Jan-FebJan-Feb Jun-JulJun-Jul Jan-FebJan-Feb Jun-AugJun-Aug Jan-MarJan-Mar Jun-AugJun-Aug Jan-MarJan-Mar

RNA sampleRNA sample                           

H5 sample:H5 sample:                           

- clade 1- clade 1    22 11 11 22       22   

- clade 2.1- clade 2.1 22 22 11 11    22         

- clade 2.2- clade 2.2 22 22 11 11    22 22    22

- clade 2.3.2- clade 2.3.2       22 22 22       22 22

- clade 2.3.4- clade 2.3.4 22 22 11 11 22 22 22      

H1 sampleH1 sample 11 11 11 11 22 11 11 11   

H3 sampleH3 sample 11 11 11 11 11 11 11 11 11

H1pdm sampleH1pdm sample                22 22 22 22

Influenza B sampleInfluenza B sample                   11 11 11

Negative sampleNegative sample 22 44 22 22 11    11 11 22

                             

Inactivated virus sampleInactivated virus sample                           

H1 sampleH1 sample                         11

H3 sampleH3 sample                         11

TotalTotal 1010 1414 1010 1010 1010 1010 1010 1010 1212

Composition of panelsComposition of panels

WHOregion

No. of laboratories participated and reported results

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

2007 2007 2008 2008 2009 2009 2010 2010 2011

Feb-Mar Aug-Oct Jan-Feb Jun-Jul Jan-Feb Jun-Aug Jan-Mar Jun-Aug Jan-Mar

AFR 66 66 88 1313 1717 1919 21 22 24

AMR 55 1414 1616 2121 2323 2222 26 33 32

EMR 22 44 66 66 77 99 11 13 14

EUR 3434 3939 4343 4545 4545 5151 52 59 59

SEAR 33 44 55 55 66 66 6 8 8

WPR 1414 1616 1717 1919 1616 2121 23 23 23

All 6464 8383 9595 109109 114114 128128 139 158 160

0%

10%

20%

30%

40%

50%

60%

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

AFR AMR EMR EUR SEAR WPR

Proportion of laboratories participated and reported results

Response of Response of participantsparticipants

Problems encounteredProblems encountered

No PCR capacityNo PCR capacity No reagentsNo reagents Delay in obtaining import permitDelay in obtaining import permit Varying requirements at the customsVarying requirements at the customs Shipment detained at customs for Shipment detained at customs for

prolonged periodprolonged period

0%

5%

10%

15%

20%

25%

30%

35%

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

0

20

40

60

80

100

120

140

160

180

% of laboratories w ithout response

% of laboratories not w illing to participate

% of laboratories w ith import permit or logistical problems

No. of laboratories invited"

No. of laboratories not receiving samples

No. of laboratories% of laboratories

Reasons for laboratories not receiving Reasons for laboratories not receiving panels among total invitedpanels among total invited

Performance of participantsPerformance of participants

67%65%

74%77% 76%

79%

85% 86%

76%

160

95

139

128

114

109

83

64

158

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

160% of laboratories with all correct resultsNo. of laboratories invited

No. of participants% of laboratories

PanelNo. of participants

invited responded participated received reported all correct

Panel 1 99 44 44 33 33 2

Panel 2 99 55 55 44 44 2

Panel 3 1010 77 66 55 55 4

Panel 4 88 77 66 66 55 4

Panel 5 88 88 66 66 66 5

Panel 6 8 8 6 6 6 4

Panel 7 8 7 6 6 6 5

Panel 8 9 8 8 8 8 7

Panel 9 9 8 8 8 8 6

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

Performance of SEAR participantsPerformance of SEAR participants

% of all correct

PanelNo. of participants

invited responded participated received reported all correct

Panel 1 1919 1616 1616 1515 1414 8

Panel 2 2121 1717 1717 1717 1616 10

Panel 3 2020 1919 1919 1717 1717 13

Panel 4 2121 2121 1919 1919 1919 16

Panel 5 2121 2020 2020 1818 1616 14

Panel 6 22 22 22 22 21 16

Panel 7 23 23 23 23 23 19

Panel 8 24 24 24 24 23 22

Panel 9 24 24 24 24 23 21

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Panel 1 Panel 2 Panel 3 Panel 4 Panel 5 Panel 6 Panel 7 Panel 8 Panel 9

Performance of WPR participantsPerformance of WPR participants

% of all correct

Performance in Panel 8

Performance in Panel 9

Problems identified in EQAPProblems identified in EQAP

Inconsistent technical performanceInconsistent technical performance Positive control not used appropriatelyPositive control not used appropriately Lab contaminationLab contamination Misinterpretation of resultsMisinterpretation of results Primers and probes mismatchPrimers and probes mismatch Transcriptional errorTranscriptional error

False negative results due to probe False negative results due to probe mis-matchesmis-matches

An example of a H5 real-time PCR An example of a H5 real-time PCR primer/probe set primer/probe set

Forward primer: 1 bp mis-matchForward primer: 1 bp mis-match Probe: 2 bp mis-matchesProbe: 2 bp mis-matches Reverse primer: noneReverse primer: none

GLP sGLP surveyurvey

2007 survey composed of 73 questions2007 survey composed of 73 questions 2008 survey composed of 25 questions2008 survey composed of 25 questions 2010 survey composed of 32 questions2010 survey composed of 32 questions Questions on the following seven categories:Questions on the following seven categories:

• personnelpersonnel• quality managementsquality managements• design, equipment and consumablesdesign, equipment and consumables• pre-analytical procedurespre-analytical procedures• analytical proceduresanalytical procedures• post-analytical procedurespost-analytical procedures• reporting and record keepingreporting and record keeping• safetysafety

Molecular diagnosis (PCR) & Good laboratory practice (GLP)Molecular diagnosis (PCR) & Good laboratory practice (GLP)

Laboratories returning completed GLP survey formsLaboratories returning completed GLP survey forms

2007200764 ( 96%)64 ( 96%)

2008200894(82%)94(82%)

20102010142 (89%)142 (89%)

Good laboratory practices

More than 80% of laboratories

Separate work room for molecular diagnosis (99%)

Separate set of equipment and consumables in each working area (94%)

Equipment maintenance programme (87%)

Control materials for molecular diagnosis (100%)

Standard operating procedures (94%)

Separate work room for molecular diagnosis (99%)

Separate set of equipment and consumables in each working area (94%)

Equipment maintenance programme (87%)

Control materials for molecular diagnosis (100%)

Standard operating procedures (94%)

Less than 80% of laboratories

Evaluation of the reagents used for molecular tests (67%)

Evaluation of the sensitivity/specificity of the molecular tests (59%)

Internal audit programme (54%)

Accredited by international/national scheme (34%)

Countercheck results (78%)

Evaluation of the reagents used for molecular tests (67%)

Evaluation of the sensitivity/specificity of the molecular tests (59%)

Internal audit programme (54%)

Accredited by international/national scheme (34%)

Countercheck results (78%)

Data from GLP survey 2010

GLP and EQAP performance

Compare GLP with EQAP results

• Group A (laboratories returned correct answers for all 10 samples)

• Group B (laboratories returned less than 10 corrects answers)

Laboratories with less good performance (Group B)

Lab did not return all correct results tends to meet less quality parameters; significantly more likely (p < 0.05) not having- audits of personnel

- separate rooms for all steps involving PCR

- programme to monitor equipment

- more samples in recent representative month

- SOP on preparation of in-house controls

- established test turn-around-time

Reagents/tests evaluation, internal audit and accreditation Reagents/tests evaluation, internal audit and accreditation in laboratories of different WHO regionsin laboratories of different WHO regions

67 6569 67 65

7568

59

35

59

33

6975

70

54 55

38

50

65

100

44

34

0

24

8

56

25

39

0

20

40

60

80

100

120

All region AFR AMR EMR EUR SEAR WPR

% o

f la

bora

tori

es

Reagents evaluation Tests evaluation Internal audit Accreditation

Data from GLP survey 2010

The way forwardThe way forward

ReviewReview- materials for simulated specimens/scope- materials for simulated specimens/scope- type/subtype/clade to include in the panel- type/subtype/clade to include in the panel- frequency of shipment- frequency of shipment

Enhance performance through trainingEnhance performance through training Accreditation by recognized authorityAccreditation by recognized authority

http://www.wpro.who.int/sites/htl/documents/Laboratory+Quality+Standards+and+Implementation.htm

Thank You