JOURNAL OF CLINICAL MICROBIOLOGY, July 2010, p. 2536–2540 Vol. 48, No. 70095-1137/10/$12.00 doi:10.1128/JCM.01145-09Copyright © 2010, American Society for Microbiology. All Rights Reserved.

Multicenter External Quality Assessment of Molecular Methods forDetection of Human Herpesvirus 6�

P. J. de Pagter,1* R. Schuurman,1 N. M. de Vos,1 W. Mackay,2 and A. M. van Loon1

Department of Virology, University Medical Center Utrecht, Heidelberglaan 100, Utrecht 3584CX, Netherlands,1 andQuality Control for Molecular Diagnostics, Block 4, Kelvin Campus, West of Scotland Science Park,

Glasgow G20 0SP, Scotland, United Kingdom2

Received 11 June 2009/Returned for modification 7 August 2009/Accepted 3 February 2010

The purpose of this study was to evaluate the performance of laboratories for the detection and quantifi-cation of human herpesvirus 6 (HHV-6) by an external quality assessment (EQA) evaluation. The HHV-6 EQApanel consisted of eight samples containing various concentrations of HHV-6 type A (strain GS) or type B(strain Z29), two samples containing other herpesviruses (i.e., human cytomegalovirus [HCMV] and Epstein-Barr virus [EBV]), and two HHV-6-negative samples. Panel samples were prepared in human plasma, heatinactivated, and lyophilized. Panel distribution, data management, and analysis were coordinated by QualityControl for Molecular Diagnostics (QCMD), Glasgow, United Kingdom. Fifty-one laboratories participatedand submitted 57 data sets. Eleven (19.3%) data sets were generated using conventional in-house assays, 11(19.3%) data sets using commercial real-time PCR assays, and 35 (61.4%) data sets using in-house real-timePCR assays. The presence of HHV-6 DNA at viral loads exceeding 6,000 copies/ml was detected by allparticipants, and over 80% of the participants still reported correct qualitative results for the sample con-taining just over 200 copies/ml. The false-positivity rate was 1.8% for both the negative samples and thesamples containing HCMV or EBV DNA. The majority (23/33; 69.7%) of quantitative data sets were generatedusing in-house real-time PCR assays. The standard deviations of the geometric means of the samples rangedfrom 0.5 to 0.7 log10. The results of this first international EQA demonstrate encouraging analytical sensitivityfor the detection of HHV-6-DNA in human plasma, although we observed extensive interlaboratory variationof quantitative HHV-6 DNA results. Standardization needs to be improved to allow further elucidation of theclinical significance of HHV-6 loads.

Human herpes virus type 6 (HHV-6) is a member of theBetaherpesvirus subfamily (genus Roseolovirus). Two distinctvariants have been described: HHV-6 type A (HHV-6A) andHHV-6B. HHV-6 type B infection is recognized as the cause ofa febrile disease and exanthem subitum in early childhood.Over 90% of the population worldwide is infected within thefirst 18 months of life (5, 12). After primary infection, HHV-6establishes life-long persistence in the host and is detectable inmultiple tissues, similar to other herpesviruses, such as humancytomegalovirus (HCMV) (6). HHV-6 infection rarely causessevere disease in healthy children, but viral reactivation inimmunocompromised patients is associated with severe mor-bidity (encephalitis and acute graft-versus-host disease) andincreased mortality (11). The majority of HHV-6 infections arecaused by HHV-6 type B (2, 9). Although HHV-6 type A israrely detected, this variant is suggested to act as a moreneurotropic pathogen in the context of severe central nervoussystem infections (4).

Because of their speed, sensitivity, and specificity, moleculardiagnostic assays are increasingly used for the detection ofHHV-6 DNA to diagnose viral disease. Various commercialand in-house assays are currently available. Due to the lack ofany available well-characterized reference reagents, most of

these assays lack standardization, both in performance as wellas in viral load calculation.

This lack of standardization causes difficulties in comparingresults between laboratories and, as a consequence, compli-cates the clinical interpretation of laboratory results forHHV-6. In an attempt to compare the results for HHV-6 DNAdetection and quantitation between laboratories, a first inter-national external quality assessment (EQA) study was orga-nized.

MATERIALS AND METHODS

To assess the proficiency of laboratories in the detection and quantification ofHHV-6 by nucleic acid amplification, a blinded panel of 12 samples containingvarious concentrations of HHV-6 types A and B as well as negative controls andspecificity controls was sent to laboratories worldwide. The laboratories wereinstructed to analyze the panel according to their local routine procedure forHHV-6 PCR detection and quantitation.

Preparation of samples. Samples were prepared at the Department of Virol-ogy, University Medical Center Utrecht, Utrecht, Netherlands, by serial dilutionof HHV-6 type A strain GS and HHV-6B strain Z29 (Advanced BiotechnologiesInc., MD). The concentration of virus particles in the viral stocks of the sourceviruses had been previously determined by electron microscopy (Advanced Bio-technologies Inc.). The virus stocks were heat inactivated at 56°C for 30 min andkept at �80°C before they were used for the preparation of the EQA panel.Preparation of serial bulk dilutions of the viral stocks was performed in HHV-6-,Epstein-Barr virus (EBV)-, and HCMV-negative human plasma in EDTA. Thetarget concentrations of the panel samples ranged from 200 to 25,000 (log10 2.3to 4.4) copies/ml. The bulk dilutions were sent on dry ice to Quality Control forMolecular Diagnostics (QCMD) in Glasgow, United Kingdom, where 1-ml ali-quots were prepared by QCMD and subsequently lyophilized prior to distribu-tion at ambient temperature.

The final panel consisted of eight HHV-6-containing human plasma samples,

* Corresponding author. Mailing address: University Medical CenterUtrecht, Department of Pediatrics, Stem Cell Transplantation Unit, Lun-dlaan 6: KE 04.133.1, Utrecht 3584CX, Netherlands. Phone: 31-88-7553888. Fax: 31-88-7555349. E-mail: p.j.depagter@umcutrecht.nl.

� Published ahead of print on 10 February 2010.

2536

on April 9, 2021 by guest

http://jcm.asm

.org/D

ownloaded from

two plasma samples from healthy individuals, and two plasma samples containingHCMV or EBV at a concentration of log10 4.4 (25,000) copies/ml to assess thespecificities of the assays.

Program description. The panel samples were coded, as indicated in Table 1.The participants were instructed to analyze the samples according to their stan-dard laboratory protocols. The results were reported to the QCMD neutral officeonline via a data submission form on the QCMD website within 6 weeks of thereceipt of the panel. To ensure confidentiality, all participating laboratoriesreceived a code number for data reporting and further communication.

Data analysis. The QCMD neutral office confidentially handled the coding ofthe panel, collected the results and questionnaires, and analyzed the results, asdescribed before (8).

The data were analyzed according to qualitative and quantitative criteria. Inaddition, the results were analyzed and compared by type of technology, i.e.,in-house-developed versus commercial assays and conventional versus real-timePCRs.

A quantitative geometric mean value was calculated for each panel sample intwo ways, namely, as the consensus concentration (the geometric mean of theparticipants’ results, once outliers had been removed) and as the technologyconsensus concentration (the geometric mean of the participants’ results pertechnology group once, outliers had been removed). For each sample in a dataset of quantitative results, scores were awarded on the basis of the distance fromthe calculated geometric mean value for each panel sample. Zero points wereawarded if the quantitative value returned was within 1 standard deviation (SD)from the geometric mean. One point was awarded if the quantitative value wasbetween 1 and 2 SDs, 2 points if the value was within 2 and 3 SDs, and 3 pointsfor quantitative values more than 3 SDs from the geometric mean. For each dataset, a cumulative performance score was calculated on the basis of the sum of theperformance scores of the individual samples in that data set. Thus, zero was thebest score that could be achieved for the panel as a whole; i.e., all results within1 SD from the geometric means were calculated for the samples (7).

RESULTS

Performance of laboratories. Fifty-one laboratories from 20countries (the number of participating laboratories per coun-try: Australia, 2; Austria, 2; Belgium,1; Canada, 1; Czech Re-public, 2; Finland, 1; France, 4; Germany, 3; Israel, 2; Italy, 2;Lebanon, 1; Netherlands, 6; New Zealand, 1; Norway, 1; Por-tugal, 3; South Africa, 2; Sweden, 4; Switzerland, 4; UnitedKingdom, 5; United States, 4) participated in this first inter-national HHV-6 EQA program in 2008. In total, 57 completeddata sets were received from 50 participants, 2 laboratories

reported 2 data sets, and 1 laboratory reported 3 data sets. Oneparticipant withdrew, reporting that the assay was still underdevelopment. Eleven (19.3%) data sets were generated usingconventional in-house assays, 11 (19.3%) data sets using com-mercial real-time PCR assays, and 35 (61.4%) data sets usingin-house real-time PCR assays (Table 1). A range of nucleicacid extraction methods have been used. No significant differ-ences in correct results between these methods were found(Fig. 1). In this study, the heterogeneity of the nucleic acidextraction methods used did not allow further in-depth analysisof the results.

Overall, the data sets obtained with real-time commercialtechnologies (n � 11) reported more correct results for thepositive samples than the data sets obtained with in-housereal-time and conventional in-house technologies (100% ver-sus 96.1%), indicating that they have a higher sensitivity thansome in-house-developed assays (Table 1).

The consensus geometric mean viral load for the samplesranged from approximately 25,800 to 240 (log10 4.411 to 2.383)copies/ml, closely matching the target concentrations that were

TABLE 1. Qualitative results of HHV-6 DNA EQA, 2008

Sample Sample contentConcn in sample

(no. ofcopies/ml)

Consensuslog10

No. of correct data sets/total no. tested (%)

Total(n � 57)

PCR

Conventional in-house(n � 11)

Real-time commercial(n � 11)a

Real-time in-house(n � 35)

HHV608-04 HHV-6 type A 7,430 3.871 57/57 (100) 11/11 (100) 11/11 (100) 35/35 (100)HHV608-11 HHV-6 type A 2,500 3.398 56/57 (98) 11/11 (100) 11/11 (100) 34/35 (97)HHV608-02 HHV-6 type A 568 2.754 54/57 (95) 10/11 (91) 11/11 (100) 33/35 (94)HHV608-09 HHV-6 type B 25,763 4.411 57/57 (100) 11/11 (100) 11/11 (100) 35/35 (100)HHV608-07 HHV-6 type B 6,516 3.814 57/57 (100) 11/11 (100) 11/11 (100) 35/35 (100)HHV608-01 HHV-6 type B 1,148 3.060 55/57 (97) 10/11 (91) 11/11 (100) 34/35 (97)HHV608-03 HHV-6 type B 977 2.991 55/57 (97) 10/11 (91) 11/11 (100) 34/35 (97)HHV608-10 HHV-6 type B 242 2.384 49/57 (86) 8/11 (73) 11/11 (100) 30/35 (86)HHV608-06 CMV (25000 cp/ml) 56/57b (98) 11/11 (100) 11/11 (100) 34/35 (97)HHV608-12 EBV (25000 cp/ml) 56/57b (98) 10/11 (91) 11/11 (100) 35/35 (100)HHV608-05 HHV-6 negative 56/57 (98) 11/11 (100) 10/11 (91) 35/35 (100)HHV608-08 HHV-6 negative 56/57 (98) 10/11 (91) 11/11 (100) 35/35 (100)

a Commercial assays: HCMV and HHV-6, -7, abd -8 R-gene, Argene, Verniolle, France (n � 3); HHV-6 QPCR Alert, Nanogen Advanced Diagnostics, San Diego,CA (n � 6); CMV/EBV/HHV-6 Screen, Sacace Biotechnologies s.r.l, Como, Italy (n � 1); and LightMix for detection of HHV-6, Tib MolBiol GmbH, Berlin, Germany(n � 1).

b Results reported to be HHV-6 negative were considered correct. All false-negative results for HHV-6A and HHV-6B were reported by two laboratories. The twofalse-positive results were provided by two different laboratories.

FIG. 1. Nucleic acid extraction procedures.

VOL. 48, 2010 MULTICENTER QUALITY ASSESSMENT OF HUMAN HERPESVIRUS 6 2537

on April 9, 2021 by guest

http://jcm.asm

.org/D

ownloaded from

defined prior to preparation of the panel. All participants wereable to detect HHV-6 DNA in the samples containing 6,500(log10 3.814) copies/ml or more. A viral load of 240 (log10

2.384) copies/ml was still reported positive by 86% of theparticipants. For the commercial real-time PCR-based assays,11/11 (100%) data sets correctly detected the presence ofHHV-6 DNA in the positive sample with the smallest amountof HHV-6 DNA (242 copies/ml). No differences in results werefound for HHV-6A and HHV-6B DNA. The performancerates declined with lower HHV-6 concentrations in the sam-ples. The overall proportion of false-negative results was 15/456 (3.5%). All these false-negative results were reported bytwo laboratories.

Two false-positive results were reported for the negativesamples (2/114, 1.8%), one from an in-house conventionalPCR and the other from an in-house real-time PCR. Thespecificities of the assays for the detection of HHV-6, as shownby the results for samples with other herpesviruses (HCMVand EBV), was high. A false-positive result was reported onlytwice, and therefore, the percentage of false-positive results forthe HCMV- and EBV-positive samples was the same as thatobserved for the true-negative samples (1.8%).

In 44/57 (77%) data sets, correct qualitative results werereported for each of the panel samples.

Quantitative results. Thirty-three (58%) data sets based onquantitative results were submitted. The geometric means andthe ranges of the reported viral load results are shown in Table2. The majority of the data sets with quantitative results wereobtained with in-house real-time PCR assays (24/33; 73%).Nine data sets were obtained with a commercial assay/kit. Themean viral load results obtained with commercial methodswere similar to those obtained with the in-house-developedtechnologies for HHV-6 type B but were approximately 3-foldlower for HHV-6 type A (Table 2). Furthermore, the ranges ofthe viral load results were wider for the HHV-6 type A resultsobtained with commercial assays than for the results obtainedwith in-house assays (Table 2). The ranges for the HHV-6 typeA results generated with commercial assays started at concen-trations lower than those for in-house assays. This differencewas not observed for HHV-6 type B results.

The variation in viral loads per sample was large, with thestandard deviations of the log10 number of copies/ml rangingfrom 0.5 to 0.7. However, in general, the laboratories werecorrectly able to rank consecutive samples with differences inHHV-6 loads (Fig. 2). In 6/33 (18%) data sets, the reported

viral load was within 1 SD from the geometric mean for eachpanel sample. In an additional five data sets, the results werewithin 1 SD for all except one sample, the latter being �2 SDsfrom the geometric mean.

For 8/33 (24%) data sets, quantitative results were not re-ported for one or more samples, and therefore, overall perfor-mance scores could not be calculated (Fig. 3).

DISCUSSION

The program described here is the first international EQAprogram to evaluate the detection and quantification ofHHV-6 DNA. Fifty-one laboratories participated, underliningthe need for an EQA program in this area. The majority of theHHV-6 DNA detection assays used by the participants aresensitive and specific. All participants obtained excellent re-sults for qualitative performance, but the quantitative resultsclearly need improvement.

The use of well-validated and standardized assays for thedetection of HHV-6 DNA in clinical samples is important toevaluate the clinical significance of HHV-6. However, an in-ternational HHV-6 standard is not yet available.

More than 80% of the participants recorded correct quali-tative results for the sample containing the lowest viral con-centration (approximately 200 copies/ml), indicating that theoverall sensitivity of the nucleic acid amplification methods forthe detection of HHV-6 is at least comparable to the sensitivitylevel generally accepted for other herpesviruses, such asHCMV and EBV, or even better. In comparison, the sensitiv-ities for samples with similar viral loads in QCMD’s 2008 EQAprogram for HCMV and EBV were only 61.3% and 77.8%,respectively. Overall, the commercial assays appeared to beslightly more sensitive than some in-house assays (conven-tional or real-time PCR assays).

In this study, 81% data sets were obtained with in-houseassays, the majority of which (65%) were based on real-timePCR. In-house assays are usually based on different assay set-ups and often vary in virtually all aspects of the procedure,including nucleic acid extraction, amplification, primers,probes, etc. Remarkably, the concentrations reported for theHHV-6 type A samples were, on average, higher among par-ticipants using real-time in-house-developed assays than com-mercial real-time PCR assays, although the difference was notsignificant. This difference was not observed for the HHV-6type B samples (Table 2). Flamand et al. previously analyzed

TABLE 2. Quantitative results of HHV-6 DNA EQA

Sample Samplecontent

Geometric mean (range) no. of copies/ml

Consensus viral load(n � 33)a

Commercial assays(n � 9)

Real-time in-houseassays (n � 23)

HHV608-04 HHV-6A 7,430 (1,774–31,117) 3,373 (413–27,542) 9,772 (3,126–30,549)HHV608-11 HHV-6A 2,500 (705–8,872) 1,172 (186–7,396) 3,258 (1,180–8,995)HHV608-02 HHV-6A 568 (109–2,944) 194 (21–3,105) 800 (257–2,489)HHV608-09 HHV-6B 25,763 (8,974–73,961) 28,774 (11,641–71,121) 24,660 (8,166–74,473)HHV608-07 HHV-6B 6,516 (2,286–18,578) 7,656 (2,317–25,293) 6,152 (2,275–16,634)HHV608-01 HHV-6B 1,148 (303–4,355) 1,361 (209–8,851) 1,091 (348–3,428)HHV608-03 HHV-6B 977 (238–4,009) 971 (170–5,546) 979 (270–3,556)HHV608-10 HHV-6B 242 (66–881) 222 (47–1,062) 248 (74–830)

a One participant reported quantitative results from a conventional assay.

2538 DE PAGTER ET AL. J. CLIN. MICROBIOL.

on April 9, 2021 by guest

http://jcm.asm

.org/D

ownloaded from

nine molecular HHV-6 detection assays systematically by usingHHV-6-spiked serum samples with between 0 and 100,000HHV-6 genomes copies per sample (3). In their study, threeTaqMan-based PCR assays showed results closest to the ex-

pected values (56,406 to 172,000 for the three TaqMan-basedPCR assays versus 16,707 to 245,250 expected genome copies)(3). In our HHV-6 EQA program, we evaluated more labora-tories and different types of assays, but our results were not in

FIG. 2. Individual results for HHV-6 EQA, 2008. The different lines represent the viral loads (numbers of copies/ml) in consecutive sampleswith increasing HHV-6 type B and A loads, respectively.

VOL. 48, 2010 MULTICENTER QUALITY ASSESSMENT OF HUMAN HERPESVIRUS 6 2539

on April 9, 2021 by guest

http://jcm.asm

.org/D

ownloaded from

agreement with the TaqMan assay-based results of Flamand etal. (3). Our EQA program was primarily designed to assess alaboratory’s ability to detect HHV-6 DNA correctly ratherthan test the performance of in-house versus commercial as-says. Such a comparison is possible only if each method is usedin sufficient numbers and would probably require a differentstudy design. In this study, the number of quantitative com-mercial assays (9/33, 27%) did not allow such a comparison; afuture distribution with higher numbers of participants mayallow such a comparison.

Additionally, different PCRs which amplify different HHV-6gene regions were used in our EQA. The differences in thequantitative and qualitative results might be explained by thesedifferences in assay setup. To date, calibration of assays is noteasily achieved, due to a lack of standardized reference re-agents. Further standardization is needed to allow comparisonof quantitative results between different laboratories and toallow elucidation of the clinical role of the HHV-6 load.

The increasing demand for rapid and reliable methods forthe diagnosis of HHV-6 disease has led to the widespreadintroduction of molecular diagnostic procedures into the clin-ical virological laboratory. Different studies have shown anassociation between increased HHV-6 load and clinical disease

(e.g., encephalitis) (10) or complications after hematopoieticstem cell transplantation (1, 11). With standardized referencematerials, the different molecular HHV-6 assays might be eval-uated and the sensitivity and specificity may further improve.Finally, this may lead to a comparison of quantitative resultsbetween different laboratories and may allow elucidation ofthe clinical role of the HHV-6 load.

ACKNOWLEDGMENTS

We thank all participants of this first HHV-6 EQA program coor-dinated by QCMD.

We all declare that we have no conflict of financial interest.

REFERENCES

1. de Pagter, A. P. J., R. Schuurman, H. Visscher, M. de Vos, M. Bierings, A. M.van Loon, C. S. Uiterwaal, D. van Baarle, E. A. Sanders, and J. Boelens.2008. Human herpes virus 6 plasma DNA positivity after hematopoietic stemcell transplantation in children: an important risk factor for clinical outcome.Biol. Blood Marrow Transplant. 14:831–839.

2. Drobyski, W. R., M. Eberle, D. Majewski, and L. A. Baxter-Lowe. 1993.Prevalence of human herpesvirus 6 variant A and B infections in bonemarrow transplant recipients as determined by polymerase chain reactionand sequence-specific oligonucleotide probe hybridization. J. Clin. Micro-biol. 31:1515–1520.

3. Flamand, L., A. Gravel, D. Boutolleau, R. Varez-Lafuente, S. Jacobson, M. S.Malnati, D. Kohn, Y. W. Tang, T. Yoshikawa, and D. Ablashi. 2008. Multi-center comparison of PCR assays for detection of human herpesvirus 6 DNAin serum. J. Clin. Microbiol. 46:2700–2706.

4. Hall, C. B., M. T. Caserta, K. C. Schnabel, C. Long, L. G. Epstein, R. A.Insel, and S. Dewhurst. 1998. Persistence of human herpesvirus 6 accordingto site and variant: possible greater neurotropism of variant A. Clin. Infect.Dis. 26:132–137.

5. Okuno, T., K. Takahashi, K. Balachandra, K. Shiraki, K. Yamanishi, M.Takahashi, and K. Baba. 1989. Seroepidemiology of human herpesvirus 6infection in normal children and adults. J. Clin. Microbiol. 27:651–653.

6. Ruell, J., C. Barnes, K. Mutton, B. Foulkes, J. Chang, J. Cavet, M.Guiver, L. Menasce, M. Dougal, and R. Chopra. 2007. Active CMVdisease does not always correlate with viral load detection. Bone MarrowTransplant. 40:55–61.

7. Staines, H., L. Garcia-Fernandez, R. Pogothata, P. Wallace, W. MacKay,and A. Van Loon. 2009. Monitoring performance of nucleic acid-based di-agnostic measurement system users by EQA. Accreditation Quality Assur-ance: J. Quality Comparability Reliability Chem. Measurement 14:243–252.

8. Valentine-Thon, E., A. M. van Loon, J. Schirm, J. Reid, P. E. Klapper, andG. M. Cleator. 2001. European proficiency testing program for moleculardetection and quantitation of hepatitis B virus DNA. J. Clin. Microbiol.39:4407–4412.

9. Wang, F. Z., A. Linde, H. Dahl, and P. Ljungman. 1999. Human herpesvirus6 infection inhibits specific lymphocyte proliferation responses and is relatedto lymphocytopenia after allogeneic stem cell transplantation. Bone MarrowTransplant. 24:1201–1206.

10. Zerr, D. M. 2006. Human herpesvirus 6 and central nervous system diseasein hematopoietic cell transplantation. J. Clin. Virol. 37(Suppl. 1):S52–S56.

11. Zerr, D. M., L. Corey, H. W. Kim, M. L. Huang, L. Nguy, and M. Boeckh.2005. Clinical outcomes of human herpesvirus 6 reactivation after hemato-poietic stem cell transplantation. Clin. Infect. Dis. 40:932–940.

12. Zerr, D. M., A. S. Meier, S. S. Selke, L. M. Frenkel, M. L. Huang, A. Wald,M. P. Rhoads, L. Nguy, R. Bornemann, R. A. Morrow, and L. Corey. 2005.A population-based study of primary human herpesvirus 6 infection. N. Engl.J. Med. 352:768–776.

FIG. 3. Quantitative results: participants’ scores. Bars representthe number of data sets with the corresponding overall quantitativeresult score (n � 25). Quantitative results for one or more of theHHV-6-positive samples were lacking in 8/33 data sets. For thoseparticipants, an overall performance score could therefore not be cal-culated. LOD/NR, level of detection/no result reported.

2540 DE PAGTER ET AL. J. CLIN. MICROBIOL.

on April 9, 2021 by guest

http://jcm.asm

.org/D

ownloaded from