Clin Infect Dis. 2011 Tenover S338 45

S U P P L E M E N T A R T I C L E

Developing Molecular Amplification Methodsfor Rapid Diagnosis of Respiratory TractInfections Caused by Bacterial Pathogens

Fred C. Tenover

Cepheid, Sunnyvale, California

Current diagnostic methods for bacterial respiratory tract infections are slow and often of marginal value for

patient management if the adequacy of the specimen is not confirmed before culture. Molecular amplification

tests, which are highly sensitive, can provide results in hours rather than days but may not distinguish

colonization from infection unless a quantification step is included. Defining the reference method to be used

for evaluating a novel molecular assay, with input from the US Food and Drug Administration (FDA), is critical

before initiating development of a potential product. Although expectorated sputum may be the clinician’s

specimen of choice for testing because of ease of collection, the poor quality of such specimens may pose

problems for clinical trials of novel amplification tests. There are still many gaps in our understanding of

the interplay between colonization and infection and of the role that amplification tests may play in guiding

anti-infective therapy. Thus, the performance parameters of a new diagnostic method should be closely

matched to a precisely defined intended use statement.

Developing accurate methods for diagnosing bacterial

respiratory tract infections has long been a challenge for

the clinical microbiology laboratory [1, 2]. The semi-

quantitative culture methods used in most clinical

microbiology laboratories today, although adequate for

recovering and identifying a wide variety of bacterial

species from respiratory specimens, cannot differentiate

between colonization and infection, especially when the

majority of specimens submitted for testing either are

contaminated with upper respiratory tract flora or are

simply saliva, as shown by Gram stain [2, 3]. In fact,

a Gram stain of expectorated sputum can be used both

to ascertain the quality of a specimen and to guide

empirical therapy [4], but the closing of satellite labo-

ratories where residents, fellows, and attending physi-

cians could perform and interpret their own Gram

stains to aid in rapid decision making for patient

management has significantly lengthened the time it

takes for physicians to receive results [1, 5]. Recognizing

infection in a ventilated patient is even more of a chal-

lenge, because colonization of the respiratory tree with

a succession of gram-negative bacteria is common

during prolonged hospital stays, and most of the species

recovered from endotracheal aspirates would typically

be considered true pathogens if recovered from the

bloodstream [6]. Thus, many physicians feel compelled

to initiate empirical antimicrobial therapy on the basis

of culture results from endotracheal aspirates [7]. The

key reason for performing bacterial culture is to opti-

mize anti-infective therapy, which hinges not only on

identifying the causative agent of infection in a timely

manner but also on determining its antimicrobial sus-

ceptibility profile as well. In this age of antimicrobial

resistance, the current turn-around times (ranging from

48 to 96 hours to complete testing) are often inadequate

for optimal patient care [6, 8]. With the ever-expanding

array of multidrug-resistant pathogens, including Kleb-

siella pneumoniae, Pseudomonas aeruginosa, and Acine-

tobacter species, getting anti-infective therapy for

bacterial pneumonia correct the first time is critical for

Correspondence: Fred C. Tenover, PhD, Cepheid, 904 Caribbean Dr, Sunnyvale,CA 94089 ([email protected]).

Clinical Infectious Diseases 2011;52(S4):S338–S345� The Author 2011. Published by Oxford University Press on behalf of theInfectious Diseases Society of America. All rights reserved. For Permissions,please e-mail: [email protected]/2011/52S4-0009$14.00DOI: 10.1093/cid/cir049

S338 d CID 2011:52 (Suppl 4) d Tenover

by guest on October 7, 2015

http://cid.oxfordjournals.org/D

ownloaded from

http://cid.oxfordjournals.org/

positive patient outcomes [4, 6, 7]. Thus, to many physicians,

the rapid and accurate identification of bacterial respiratory

pathogens, particularly for hospital-acquired pneumonia (HAP)

and ventilator-associated pneumonia (VAP), has become an

unmet clinical need.

In 2005, the disease management guidelines published by

the European Respiratory Society noted the potential role of

molecular amplification methods in improving the diagnosis of

lower respiratory tract infections [8]. Although many physicians

find the potential for enhanced sensitivity and more rapid

turn-around times attractive, there are both challenges and

barriers that industry will face in developing and marketing

molecular amplification tests for bacterial respiratory pathogens.

This article will review those challenges and barriers.

CURRENT DIAGNOSTIC METHODS FOR

RESPIRATORY TRACT INFECTIONS

The current semiquantitative agar plate-based culture method

used for analyzing expectorated sputum from patients with

suspected community-acquired pneumonia (CAP) or HAP is

slow and results may bemisleading, particularly if a Gram stain is

not performed in parallel to ascertain specimen adequacy [6, 8].

Expectorated sputum samples that contain .10 squamous epi-

thelial cells and ,25 polymorphonuclear leukocytes per low

power field are likely to contain saliva and not lower respiratory

tract material from the site of infection [2]. Unfortunately,

however, only 32% to 76% of sputum samples obtained from

hospitalized patients recently admitted with severe CAP met the

above acceptance criteria, according to a recent meta-analysis of

published CAP studies [8]. Even if the sputum sample is deemed

to be adequate using Gram stain criteria, the culture may still not

yield an obvious pathogen [2, 8]. The highest predictive value for

a culture of expectorated sputum occurs when Gram stain shows

a predominant morphotype (eg, gram-positive lancet-shaped

cocci in pairs) and the culture yields predominant growth of

a single recognized respiratory pathogen of that morphotype

(eg, Streptococcus pneumoniae) [9]. Unfortunately, such con-

cordance decreases rapidly when specimens are collected after

initiation of antimicrobial therapy [9] or when their arrival at the

microbiology laboratory is significantly delayed. On the other

hand, for aspiration pneumonia, which is often caused by a mix

of anaerobic bacterial species, a Gram stain of expectorated

sputum is often the only diagnostic test of value, because sputa

are not cultured anaerobically and aerobic cultures will yield only

mixed normal respiratory flora.

The other issue that impacts patient care is the slow turn-

around time of the test procedure. When antimicrobial sus-

ceptibility testing is included, the culture method requires

a minimum of 48 hours to complete. This is often too late to be

effective in guiding anti-infective therapy, especially when the

bacterial pathogen is multidrug resistant. Urine antigen tests for

pneumococci and Legionella pneumophila can provide rapid

results, but the available commercial tests have variable sensi-

tivity (range 66%–92% [4, 10]) and antigen tests for

L. pneumophila are limited to serogroup 1[10].

CHALLENGES FOR DIAGNOSTIC TEST

MANUFACTURERS

One approach that may improve the diagnosis of respiratory

tract infections and shorten the time necessary to place patients

on appropriate therapy is the use of nucleic acid amplification

methods. Real-time polymerase chain reaction (PCR) assays

[11–13], pyrosequencing [14, 15], and nucleic acid arrays [16]

are all options for improving the speed and accuracy of labo-

ratory results. The key challenge for industry is to develop assays

that are not only rapid but also readily accessible and perceived

by either laboratories or health care systems as cost-effective.

Development of an assay that is rapid but unavailable on evening

or night shifts or on weekends because of its technical complexity

limits the medical value of the test. From the industry perspec-

tive, cost-effectiveness should be determined not only by com-

parison to costs of performing slower conventional methods in

the laboratory but also by consideration of the cost savings

achieved from optimized antimicrobial therapy, decreased use of

additional diagnostic tests, and shorter hospital stays.

To have a positive impact on patient management, molecular

amplification tests will need to provide clear, definitive results

that will give physicians the data necessary to start, or in some

cases withhold, antimicrobial agents [6]. To be successful,

industry must determine (1) which combination of molecular

targets and clinical specimens will produce results that will

effectively guide anti-infective therapy regimens for patients

with pneumonia or other respiratory tract diseases; (2) which

reference method will provide the most meaningful results

for test development and clinical trials; and (3) which assays

companies will be able to afford to develop, given both the costs

of development and clinical trials and the ever-changing

reimbursement schedules from both private insurers and the

federal government. The potential return on investment is a key

factor when industry chooses which assays to develop. Input

from the US Food and Drug Administration (FDA) regarding

each of these determinations can be helpful and should be

sought. The development of novel diagnostic assays, especially

when the reference methods are not obvious, is certainly one of

the major challenges facing the industry.

CHOOSING MOLECULAR TARGETS

Molecular assays may target either a single pathogen, such as

Mycobacterium tuberculosis [17], or multiple respiratory

Molecular Tests for Respiratory Infection d CID 2011:52 (Suppl 4) d S339



ownloaded from


pathogens in a single assay. Several multiplex assays for viral

respiratory pathogens have been described [12, 13, 16], a few of

which have been commercialized and have received FDA

clearance. There are merits to both single-pathogen and multi-

plex approaches.

DETECTION OF A SINGLE BACTERIAL SPECIES

Certain bacterial respiratory pathogens cause such distinct

clinical syndromes that assays that target them individually

still have clinical utility. These include organisms such

as M. tuberculosis, L. pneumophila, and Bordetella pertussis.

Molecular assays have been developed for all of these organisms,

although for the sake of brevity, discussion will be limited to the

issues surrounding the development of assays forM. tuberculosis,

where there is little doubt that detection of the organisms is

highly associated with disease.

RAPID DETECTION OF MYCOBACTERIUM

TUBERCULOSIS

The World Health Organization has reported that in 2008 there

were an estimated 9.4 million new cases of tuberculosis world-

wide [18]. Although the overall number of cases of tuberculosis

is decreasing in the United States, the proportion of cases among

foreign persons is steadily increasing [19, 20]. Of particular

importance from a diagnostic point of view is the fact that in

more than 26,000 cases of tuberculosis diagnosed in US-bound

refugees and immigrants, the acid-fast stains of expectorated

sputum samples from the patients yielded negative results. Such

infections are likely to be missed by tuberculosis screening

programs that rely primarily on acid-fast staining of respiratory

secretions instead of chest x-rays and slower culture methods

[19]. In addition, infections among foreign-born persons are

more likely to be caused by multidrug-resistant M. tuberculosis

strains than are cases among US domestic persons [20, 21].

Thus, particularly from the standpoint of immigrant and refugee

screening programs, there is a need for rapid tests that have

adequate sensitivity for detecting M. tuberculosis among smear-

negative samples. Although molecular diagnostics have been

used for direct detection ofM. tuberculosis in clinical samples for

more than a decade, these assays typically show unacceptably

low sensitivity levels for smear-negative specimens [17]. How-

ever, a recently released PCR-based commercial assay (which is

currently available only outside of the United States) showed

sensitivity of.90% on smear-negative culture-positive samples

in a multicountry, multicenter study [22]. Assays that in-

corporate direct detection of resistance markers in addition to

direct detection ofM. tuberculosisDNA targets are important for

guiding anti-tuberculosis therapy, particularly in an era of

multidrug-resistant and extensively drug-resistant strains [11,

23, 24]. The current drawback to developing and marketing tests

for M. tuberculosis in the United States is that tests for this

organism are classified by the FDA as Class III (ie, having high

impact on patient management). Thus, molecular tests to detect

the organism require what is known as a Pre-Market Approval

(PMA) application rather than the less rigorous, but still tech-

nically demanding and costly, Pre-Market Notification (510(k))

submission. In a PMA submission, the company must

independently show that its device is safe and effective in the

context of its intended use. This includes demonstrating

through carefully conducted clinical trials that the results of the

tests are highly accurate compared with a robust reference

method and that the company has designed and implemented

sufficient controls of its manufacturing process (including

product design, production, and quality control) to assure

the FDA that the method or device can be manufactured

reproducibly. An FDA Expert Panel is typically convened to

review the data to further assure the safety and effectiveness of

the device. By contrast, a 510(k) submission requires only that

the company demonstrate through clinical trials that its device

or method is ‘‘substantially equivalent’’ to a ‘‘predicate device or

method’’—that is, one that previously has been cleared by the

FDA for use in the United States. Although early molecular

probe assays for M. tuberculosis did not require a PMA, the

reclassification ofM. tuberculosis tests as Class III under current

regulations poses a significant barrier to test development and

marketing in the United States primarily because of the com-

plexity, breadth, and cost of the clinical trials required. Nucleic

acid amplification tests for M. tuberculosis, including at least 2

that simultaneously identify antimicrobial resistance markers

[11, 23], are widely available outside of the United States.

However, although there are published data describing the

performance of the GenProbe Amplified M. tuberculosis Direct

(MTD) test, the Roche Amplicor MTB test, the Cobas Amplicor

test, the Abbott LCx test, and the BD-ProbeTec Strand Dis-

placement Amplification (SDA) test for detection of M. tuber-

culosis in clinical samples, as noted by Ling et al [25] in their

recent meta-review of nucleic acid amplification testing meth-

ods, the GenProbe MTD is now the only FDA-cleared test

available in the United States. The paucity of molecular di-

agnostic options for M. tuberculosis detection in the United

States may reflect both real and perceived regulatory and mar-

ket-related barriers that are substantial enough to keep many in

the diagnostics industry on the sidelines.

DIRECT DETECTION OF MULTIPLE

ORGANISMS IN RESPIRATORY TRACT

SAMPLES BY SYNDROME

Potential multiplex assays for respiratory tract disease caused by

bacterial pathogens include those for CAP, HAP, VAP, acute




ownloaded from


bronchitis, sinusitis, exacerbation of chronic obstructive pul-

monary disease (COPD), and aspiration pneumonia. In

designing assays, it is critical to understand whether an assay for

a limited number of bacterial pathogens will meet physicians’

needs and provide adequate data for initiating or altering anti-

infective therapy. The targets chosen for molecular assays are

usually the key pathogens likely to be present in respiratory

samples. However, in this age of multidrug resistance, expand-

ing the target selection to include key antimicrobial resistance

genes that would alter existing therapy or guide empirical

therapy should also be considered. Potential groups of targets

for respiratory tract samples obtained from patients with

CAP, HAP, or VAP are presented in Table 1. The addition of

resistance gene targets, particularly those that mediate carba-

penem resistance (ie, resistance to doripenem, ertapenem, imi-

penem, and meropenem), may be very useful, especially because

carbapenems now play a prominent role in empirical therapy of

both HAP and VAP [6, 8]. For example, the emerging Klebsiella

pneumoniae carbapenemase (KPC) b-lactamases (encoded by

blaKPC genes, which mediate resistance to penicillins; cepha-

losporins, including extended-spectrum cephalosporins; and

carbapenems) [26–28] can compromise therapy for infections

caused by K. pneumoniae, Enterobacter aerogenes, and other

species of Enterobacteriaceae [29, 30], whereas the metallo-

b-lactamases, IMP and VIM (encoded by the blaIMP and blaVIMgenes, respectively) that mediate resistance to all b-lactam drugs

including carbapenems (with the exception of the mono-

bactams), can compromise therapy for Pseudomonas aeruginosa

infections [30]. Specific OXA b-lactamases (eg, OXA-40,

which also mediates resistance to extended-spectrum cepha-

losporins and carbapenems) can also compromise therapy for

Acinetobacter infections [31]. For gram-positive pathogens, the

presence of mecA in samples positive for S. aureus suggests that

vancomycin (or perhaps linezolid), either alone or in combi-

nation with other anti-infective agents, is necessary. If physicians

have confidence in the results of molecular assays, then negative

results for a specific pathogen should prompt physicians to

withhold treatment for that infectious agent (eg, to withhold

therapy for suspected methicillin-resistant Staphylococcus aureus

[MRSA] if the molecular assay for MRSA is negative). Of course,

laboratories may still culture respiratory specimens that are

positive by molecular methods to recover the pathogen to per-

form standard antimicrobial susceptibility tests on organisms

obtained in pure culture. Standard susceptibility testing provides

a broader susceptibility profile that extends beyond the few

antimicrobial resistance genes likely to be included in a molec-

ular assay.

RESPIRATORY SAMPLES FOR ANALYSIS

A variety of respiratory samples are amenable to molecular

testing, including expectorated sputum, bronchoalveolar lavages

(BALs), protected bronchial brushes, and endotracheal aspirates.

Of these, expectorated sputum samples are by far the most

common respiratory samples submitted to the clinical micro-

biology laboratory but are also the poorest in overall quality. As

the Infectious Diseases Society of America/American Thoracic

Society CAP guideline notes, one of the problems with di-

agnostic tests for respiratory tract infections ‘‘is driven by the

poor quality of most sputum microbiological samples and the

low yield of positive culture results’’ [4]. Getting specimens from

the site of infection that are not contaminated with upper

respiratory tract flora is a constant problem [3, 6, 32]. Endo-

tracheal aspirates from ventilated patients are often of better

Table 1. Potential Targets for Multiplex or Individual Molecular Amplification Assays by Syndrome [2, 4, 6, 8, 41]

CAP/exacerbations of COPD HAP/VAP Individual organisms

Streptococcus pneumoniae Staphylococcus aureus Mycobacterium tuberculosis

Haemophilus influenzae mecA genea Bordetella pertussis

blaTEM geneb Pseudomonas aeruginosa .

Moraxella catarrhalis blaVIM, blaIMP genesc .

Staphylococcus aureus Acinetobacter spp .

mecA genea blaOXA genesd .

Mycoplasma pneumoniae Enterobacteriaceae .

Chlamydophila pneumoniae blaKPC genec .

Chlamydophila psittaci Stenotrophomonas maltophilia .

Legionella pneumophila . .

NOTE. CAP, community-acquired pneumonia; COPD, chronic obstructive pulmonary disease; HAP, hospital-acquired pneumonia; VAP, ventilator-associated

pneumonia.a Mediates resistance to all b-lactam agents with the exception of the novel anti–methicillin-resistant S. aureus cephalosporins.b Mediates resistance to penicillins and first-generation cephalosporins.c Mediates resistance to cephalosporins and carbapenems; metallo-b-lactamases, such as VIM and IMP, typically do not mediate resistance to monobactams.d Some OXA b-lactamases can mediate resistance to carbapenems.




ownloaded from


quality than that of expectorated sputum from patients with

HAP but may still be contaminated with upper respiratory tract

flora [6, 32]. BALs and protected brush samples are more likely

to yield samples from the site of infection but require signifi-

cantly more effort to obtain and thus offer a much smaller

market for a new molecular test. In fact, there is a significant gap

in our knowledge as to how well molecular tests for bacterial

pathogens would perform on expectorated sputum samples,

compared with performance on BALs or protected brush

samples from the same patient collected within a similar period.

This knowledge gap is also a barrier to test development, because

a molecular test that cannot be performed on expectorated

sputum (given all the problems with specimen quality) may

not have broad enough appeal among physicians to make it

a financially viable product (from the industry perspective).

CHOOSING A REFERENCE METHOD FOR TEST

DEVELOPMENT AND CLINICAL TRIALS

Choosing a reference method for both test development and

clinical trials again highlights the issue of the ability of multiple

bacterial species to colonize and to infect the respiratory tract

[4, 32]. Antimicrobial stewardship programs frequently stress

the need to treat infections but to withhold antimicrobial agents

from patients who are asymptomatically colonized [6, 33]. The

enhanced sensitivity of molecular methods is essentially

negated if physicians perceive the test as ‘‘overly sensitive,’’

detecting colonization in the absence of infection. Balancing

sensitivity and specificity (and by analogy positive and negative

predictive values) for molecular tests is another example of

a critical gap in our understanding of the best manner in which

to apply molecular diagnostics. Is quantification of the targets

the answer for molecular tests? A recent Cochrane report

concludes that the use of quantitative cultures did not improve

the outcomes of patients with VAP, compared with the out-

comes of patients who were managed with qualitative results

[34]. However, the decreased turn-around time ofmolecular test

results may well prove advantageous over the two culture

methods (ie, qualitative and quantitative) that produced results

in similar, and significantly slower, time frames.

To illustrate this point, a brief review of the semiquantitative

culture method for expectorated sputum is appropriate.

Assessments of bacterial growth are performed on a series of

agar plates without an enrichment step. The constellation of

growth media usually includes blood agar, chocolate agar (for

fastidious organisms), MacConkey agar (or a similar selective

medium for gram-negative organisms), and colistin-nalidixic

acid agar (or a similar selective medium for gram-positive

pathogens). Results such as rare (often recorded as ‘‘one plus’’)

growth of pneumococci mixed with other respiratory flora from

an expectorated sputum sample on blood or chocolate agar

would likely be reported as ‘‘mixed respiratory flora’’ (often with

no specific mention of pneumococci on the report), whereas

heavy growth of pneumococci on the same agar plate may be

reported as ‘‘four plus growth, S. pneumoniae.’’ Although both

samples contain S. pneumoniae, in the former example, the or-

ganism is regarded as normal flora and is not reported, whereas

in the latter it gains ‘‘pathogen’’ status and is reported. It is likely

that both samples would yield a positive molecular amplification

test result. Thus, quantification of the amount of target present

with a defined threshold value that correlates with infection

rather than colonization may be necessary to provide confidence

in the molecular results [4, 32]. This adds complexity to the test

development process (because the test result is not simply the

presence or absence of the target organism) but is technically

feasible. High concentrations of target organism sequences

recognized through the amplification process are highly sug-

gestive of infection, but can one trust the absence of amplifi-

cation to be an indication of lack of infection if there was

no initial assessment of the specimen to ensure that it was of

adequate quality? Will it be necessary to include some host cell

marker to indicate that the specimen originated from the site of

infection and is not saliva? Although Gram stains of specimens

are critical for interpretation of culture results, requiring a Gram

stain of a specimen before performing a molecular assay would

defeat the purpose of having a rapid molecular test, especially

one that may ultimately be used outside of the clinical micro-

biology laboratory as a ‘‘point of care’’ test [10]. Once again, we

have identified a gap in our knowledge that is critical to assay

development. It is imperative, particularly for clinical trials of

novel molecular assays, that the end points that represent pos-

itive results for each bacterial species be established, especially

for organisms that can both colonize and infect the respiratory

tract. Quantification would not be required for organisms

such as M. tuberculosis or B. pertussis, because the presence of

these organism is synonymous with disease. The issue of

whether quantitative bacterial cultures would be necessary for

clinical trials must be answered. Appropriate cut-offs for

quantitative culture are debated, but several sources suggest

that the following numbers of organisms of a single bacterial

species indicate infection and not colonization: for expectorated

sputum, .1 x 107 colony forming units (CFU)/mL; for endo-

tracheal aspirates, .1 x 106 CFU/mL; for bronchoalveolar

lavage, .1 x 104 CFU/mL; and for protected brush specimen,

.1 x 103 CFU/mL [6, 32, 35]. It likely will be necessary to

evaluate the semiquantitative analysis of expectorated sputum

and quantitative methods (such as with BAL specimens) in

parallel to indicate whether a molecular amplification assay

could be used successfully on expectorated sputum specimens.

Because molecular assays are more sensitive than culture, one

may ask whether enrichment broths should be required to

maximize detection by culture during clinical trials. However,




ownloaded from


there are no defined enrichment broths for the majority of

bacterial respiratory pathogens; thus, this is not likely to be of

value.

ASSAY DEVELOPMENT—WHAT IS THE GOLD

STANDARD FOR BACTERIAL IDENTIFICATION?

One question that arises when using amplification methods

during both test development and clinical trials is whether the

amplification products truly represent the target organism

sequences. To establish this, it is often necessary to determine

the nucleic acid sequence of a sample of the amplification

products for the targeted pathogens. DNA sequencing is often

assumed to be completely accurate, showing 100% sensitivity

and specificity (particularly by non–molecular biologists).

However, because of a variety of technical factors, including the

quality of the sequencing template (ie, the amount of contam-

inating DNA from other sources), the selection and quality of

the primers used for amplification and sequencing, the robust-

ness of the base-calling software, and the method for compiling

the ‘‘consensus sequence’’ from multiple forward and reverse

reactions, sequencing does not always represent the ultimate

reference standard [36–38]. Nonetheless, companies must now

consider what measures of sequence quality (eg, using the

quality scores obtained by means of the ‘‘Phred’’ and ‘‘Phrap’’

computer programs [37, 39]) would constitute acceptable

standards if sequencing were included in clinical trial design. For

example, for individual sequencing runs, a length of sequence

that has,100 cumulative ‘‘Phred 20’’ quality score bases (where

the Phred software estimates the statistical likelihood of an in-

accurate base determination) is often considered to be un-

acceptable, because it represents a probability of error level of

1% [40]. On the other hand, using another software program,

a ‘‘Phrap’’ quality score of 40, which is the quality score for the

consensus sequence assembled from multiple reads of a DNA

sequence (equivalent to an error rate of�1 base for every 10,000

bases sequenced), is often considered acceptable—that is, of

publication quality (http://www.phrap.com/background.htm).

Such key parameters of quality have yet to be promulgated

by regulatory agencies, although sequencing data have been

included in multiple 510(k) submissions. Such guidelines would

be of value to diagnostic test developers as they contemplate

beginning clinical trials of novel diagnostic products.

SENSITIVITY AND SPECIFICITY, POSITIVE,

AND NEGATIVE PREDICTIVE VALUES

The key parameters that define the accuracy of a novel test,

compared with the accuracy of the reference method, include

sensitivity, specificity, and positive and negative predictive val-

ues. At present, there is no consensus as to which of these

parameters is most critical for potential molecular assays for

respiratory pathogens. Is a highly sensitive test that rarely misses

the presence of a given pathogen in a respiratory sample desir-

able, or is a test with a high negative predictive value indicating

the absence of a given pathogen of greater value? The former

scenario indicates that an antimicrobial agent is likely necessary,

whereas the latter indicates that antimicrobial agents can be

withheld. Although both high sensitivity and high specificity

rates are desirable (and are sometimes achievable in an assay),

the predictive values depend on disease prevalence and cannot

be manipulated by assay design. Thus, it is critical to define the

intended use of the assay before initiation of any clinical trials.

This is best accomplished in consultation with representatives of

the appropriate FDA center. Other issues that arise concerning

the design of clinical trials include the age ranges, ethnicity, and

clinical severity of the populations to be studied. Will trials need

to be conducted in both high prevalence and low prevalence

populations, stratified not only by age but also by gender and

even socioeconomic group? The broader and more diverse the

study population is, the larger the cost of conducting the trial.

Clearly, such clinical trials, especially if conducted in accordance

with the guidelines of a PMA, can become prohibitively

expensive rather quickly. Thus, the intended use of the assay is

perhaps one of the most critical decisions industry needs to

make before any development is initiated.

A POSSIBLE VAP ASSAY

To illustrate the above issues, a possible molecular amplification

assay for VAP is described. The intended use for the test is as

a diagnostic aid where the results of the test are considered in

association with other laboratory and clinical information to

establish a diagnosis of VAP. Whether the intended use includes

the ability of the test to guide anti-infective therapy has to be

considered carefully (eg, can this aspect of the test be proven in

a clinical trial?). The target organisms are S. aureus, P. aerugi-

nosa, Acinetobacter baumannii, a DNA sequence common to all

Enterobacteriaceae, and target sequences that are highly specific

for the following antimicrobial resistance determinants: mecA,

blaKPC, blaIMP, blaVIM, and blaOXA. The target specimens are

protected bronchial brush samples and endotracheal aspirates.

The reference methods include quantitative cultures on blood

chocolate, MacConkey, and CNA agar; a Gram stain of the

specimen; and a clinical diagnosis of VAP (fever, leukocytosis,

and purulent respiratory secretions) supported by radiologic

findings [32]. If a high degree of correlation between culture

results, Gram stain results, molecular test results, and clinical

diagnosis are achieved in alpha trials, beta trials could be con-

ducted to gather information on the potential cost effectiveness

(and physician acceptance) of the assay. If the assay results are

promising, a molecular test that ultimately could be performed




ownloaded from


on sputum samples from patients with HAP (rather than VAP)

may be feasible. Tests for CAP would require a different selec-

tion of targets.

SUMMARY

There are many challenges to developing molecular tests for

bacterial respiratory pathogens, particularly organisms that can

be both asymptomatic colonizers and overt pathogens of the

respiratory tract. Defining the reference method and the in-

tended use of the product before initiating clinical studies is

critical. Input from the FDA also is highly desirable. Although

expectorated sputum may be the clinician’s specimen of choice

because of ease of collection, the poor quality of such specimens

may require development of a unique host cellular target to

ensure specimen adequacy. Molecular amplification assays

will be costly to develop and clinical trials will be expensive, yet

the need is great. There are still many gaps in our understanding

of the interplay between colonization and infection. The key

question industry must answer is: ‘‘Is the juice worth the

squeeze?’’

Acknowledgments

I thank Drs Peter Dailey, Ellen Jo Baron, David Persing, Kerry Flom,

Michele Schoonmaker, AlanWortman, and Russel Enns for their input into

the development of this article.

Financial support. This work was supported by Cepheid.

Supplement sponsorship. This article was published as part of a supple-

ment entitled ‘‘Workshop on Molecular Diagnostics for Respiratory Tract

Infections.’’ The Food and Drug Administration and the Infectious Dis-

eases Society of America sponsored the workshop. AstraZeneca Pharma-

ceuticals, Bio Merieux, Inc., Cepheid, Gilead Sciences, Intelligent MDX,

Inc., Inverness Medical Innovations, and Roche Molecular Systems pro-

vided financial support solely for the purpose of publishing the supplement.

Potential conflicts of interest. F. C. T. is an employee of Cepheid,

a molecular diagnostics company.

References

1. Bartlett JG. Decline in microbial studies for patients with pulmonary

infections. Clin Infect Dis 2004; 39:170–2.

2. Bartlett JG. Diagnostic test for etiologic agents of community-acquired

pneumonia. Infect Dis Clin North Am 2004; 18:809–27.

3. Murdoch DR, O’Brien KL, Scott JA, et al. Breathing new life into

pneumonia diagnostics. J Clin Microbiol 2009; 47:3405–8.

4. Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases

Society of America/American Thoracic Society consensus guidelines on

the management of community-acquired pneumonia in adults. Clin

Infect Dis 2007; 44(suppl 2);S27–72.

5. Verghese A. Spiraling empiricism: when in doubt put on blindfold

and shoot. The Atlantic. http://correspondents.theatlantic.com/abra-

ham_verghese/2009/12/spiraling_empiricism_when_in_doubt_put_

on_blindfold_and_shoot.php. Accessed 14 January 2010.

6. American Thoracic Society. Guidelines for the management of adults

with hospital-acquired, ventilator-associated, and healthcare-associ-

ated pneumonia. Am J Respir Crit Care Med 2005; 171:388–416.

7. Rea-Neto A, Youssef NC, Tuche F, et al. Diagnosis of ventilator-

associated pneumonia: a systematic review of the literature. Crit Care

2008; 12:R56.

8. Woodhead M, Blasi F, Ewig S, et al. Guidelines for the management

of adult lower respiratory tract infections. Eur Respir J 2005;

26:1138–80.

9. Musher DM,Montoya R, Wanahita A. Diagnostic value of microscopic

examination of Gram-stained sputum and sputum cultures in patients

with bacteremic pneumococcal pneumonia. Clin Infect Dis 2004;

39:165–9.

10. Charles PG. Early diagnosis of lower respiratory tract infections

(point-of-care tests). Curr Opin Pulm Med 2008; 14:176–82.

11. Helb D, Jones M, Story E, et al. Rapid detection of Mycobacterium

tuberculosis and rifampin resistance using on-demand, near patient

technology. J Clin Microbiol 2009; 48(1);229–37.

12. Hindiyeh M, Hillyard DR, Carroll KC. Evaluation of the Prodesse

Hexaplex multiplex PCR assay for direct detection of seven respiratory

viruses in clinical specimens. Am J Clin Pathol 2001; 116:218–24.

13. Pabbaraju K, Tokaryk KL, Wong S, Fox JD. Comparison of the

Luminex xTAG respiratory viral panel with in-house nucleic acid

amplification tests for diagnosis of respiratory virus infections. J Clin

Microbiol 2008; 46:3056–62.

14. Bravo LT, Tuohy MJ, Ang C, et al. Pyrosequencing for rapid detection

of Mycobacterium tuberculosis resistance to rifampin, isoniazid, and

fluoroquinolones. J Clin Microbiol 2009; 47:3985–90.

15. Arnold C, Westland L, Mowat G, Underwood A, Magee J, Gharbia S.

Single-nucleotide polymorphism-based differentiation and drug

resistance detection in Mycobacterium tuberculosis from isolates or

directly from sputum. Clin Microbiol Infect 2005; 11:122–30.

16. Coiras MT, Lopez-Huertas MR, Lopez-Campos G, Aguilar JC,

Perez-Brena P. Oligonucleotide array for simultaneous detection of

respiratory viruses using a reverse-line blot hybridization assay. J Med

Virol 2005; 76:256–64.

17. Centers for Disease Control and Prevention. Updated guidelines for the

use of nucleic acid amplification tests in the diagnosis of tuberculosis.

MMWR Morb Mortal Wkly Rep 2009; 58:7–10.

18. World Health Organization. Global tuberculosis control: surveillance,

planning, financing. lobal report 2009 update. 2008. http://www.who.int/

tb/publications/global_report/2009/update/en/index.html. Accessed on

3 July 2010.

19. Liu Y, Weinberg MS, Ortega LS, Painter JA, Maloney SA. Overseas

screening for tuberculosis in US-bound immigrants and refugees.

N Engl J Med 2009; 360:2406–15.

20. Centers for Disease Control and Prevention. Trends in tuberculosis–

United States, 2008. MMWRMorb Mortal Wkly Rep 2009; 58:249–53.

21. Centers for Disease Control and Prevention. Plan to combat extensively

drug-resistant tuberculosis: recommendations of the Federal Tuber-

culosis Task Force. MMWR Recomm Rep 2009; 58:1–43.

22. Boehme CC, Nabeta P, Hillemann D, et al. Rapid molecular detection

of tuberculosis and rifampin resistance. N Engl J Med 2010;

363:1005–15.

23. Syre H, Myneedu VP, Arora VK, Grewal HM. Direct detection

of mycobacterial species in pulmonary specimens by two rapid

amplification tests, the Gen-probe amplified Mycobacterium

tuberculosis direct test and the genotype mycobacteria direct test. J Clin

Microbiol 2009; 47:3635–9.

24. Akpaka PE, Baboolal S, Clarke D, Francis L, Rastogi N. Evaluation of

methods for rapid detection of resistance to isoniazid and rifampin in

Mycobacterium tuberculosis isolates collected in the Caribbean. J Clin

Microbiol 2008; 46:3426–8.

25. Ling DI, Flores LL, Riley LW, Pai M. Commercial nucleic-acid

amplification tests for diagnosis of pulmonary tuberculosis in

respiratory specimens: meta-analysis and meta-regression. PLoS One

2008; 3:e1536.

26. Bradford PA, Bratu S, Urban C, et al. Emergence of carbapenem-

resistant Klebsiella species possessing the class A carbapenem-hydro-

lyzing KPC-2 and inhibitor-resistant TEM-30 beta-lactamases in New

York City. Clin Infect Dis 2004; 39:55–60.




ownloaded from

http://correspondents.theatlantic.com/abraham_verghese/2009/12/spiraling_empiricism_when_in_doubt_put_on_blindfold_and_shoot.php



http://www.who.int/tb/publications/global_report/2009/update/en/index.html

http://www.who.int/tb/publications/global_report/2009/update/en/index.html


27. Yigit H, Queenan AM, Anderson GJ, et al. Novel carbapenem-

hydrolyzing beta-lactamase, KPC-1, from a carbapenem-resistant

strain of Klebsiella pneumoniae. Antimicrob Agents Chemother 2001;

45:1151–61.

28. Bratu S, Landman D, Alam M, Tolentino E, Quale J. Detection of

KPC carbapenem-hydrolyzing enzymes in Enterobacter spp from

Brooklyn, New York. Antimicrob Agents Chemother 2005; 49:776–8.

29. Endimiani A, Hujer AM, Perez F, et al. Characterization of blaKPC-

containing Klebsiella pneumoniae isolates detected in different

institutions in the Eastern USA. J Antimicrob Chemother 2009;

63:427–37.

30. Queenan AM, Bush K. Carbapenemases: the versatile beta-lactamases.

Clin Microbiol Rev 2007; 20:440–58.

31. Quale J, Bratu S, Landman D, Heddurshetti R. Molecular epidemiology

and mechanisms of carbapenem resistance in Acinetobacter baumannii

endemic in New York City. Clin Infect Dis 2003; 37:214–20.

32. Lisboa T, Rello J. Diagnosis of ventilator-associated pneumonia: is

there a gold standard and a simple approach? Curr Opin Infect Dis

2008; 21:174–8.

33. Spellberg B, Guidos R, Gilbert D, et al. The epidemic of antibiotic-

resistant infections: a call to action for the medical community from

the Infectious Diseases Society of America. Clin Infect Dis 2008;

46:155–64.

34. Berton DC, Kalil AC, Cavalcanti M, Teixeira PJ. Quantitative versus

qualitative cultures of respiratory secretions for clinical outcomes in

patients with ventilator-associated pneumonia. Cochrane Database

Syst Rev. 2008; 4:1–29.

35. Torres A, Fabregas N, Ewig S, de la Bellacasa JP, Bauer TT, Ramirez J.

Sampling methods for ventilator-associated pneumonia: validation

using different histologic and microbiological references. Crit Care

Med 2000; 28:2799–804.

36. Bonfield JK, Staden R. The application of numerical estimates of base-

calling accuracy to DNA sequencing projects. Nucleic Acids Res 1995;

23:1406–10.

37. Richterich P. Estimation of errors in ‘‘raw’’ DNA sequences: a valida-

tion study. Genome Res 1998; 8:251–9.

38. Churchill GA, Waterman MS. The accuracy of DNA sequences: esti-

mating sequence quality. Genomics 1992; 14:89–98.

39. Ewing B, Green P. Base-calling of automated sequencer traces using

Phred. II. Error probabilities. Genome Res 1998; 8:186–94.

40. Patton SJ, Wallace AJ, Elles R. Benchmark for evaluating the quality

of DNA sequencing: proposal from an international external quality

assessment scheme. Clin Chem 2006; 52:728–36.

41. Carratala J, Garcia-Vidal C. What is healthcare-associated pneumonia

and how is it managed? Curr Opin Infect Dis 2008; 21:168–73.




ownloaded from


Documents

Clin Infect Dis. 2011 Tenover S338 45