QUALITY CONTROL AND QUALITY ASSURANCE FOR MOLECULAR

INFECTIOUS DISEASE LABS

Richard L. Hodinka, Ph.D. University of South Carolina School of Medicine Greenville

Greenville Health System, Greenville, SC hodinka@greenvillemed.sc.edu

Objectives

Describe pre-analytical, analytical, and post-analytical phases of QC/QA

Discuss guidelines for test requests and specimen collection, handling and processing

Explain lab design and best lab practices

Define reagent prep and use and instrument maintenance and calibration

Summarize proficiency testing, training and competency assessment

Discuss interpreting, recording and reporting test results

Quality Control – refers to the measures that must be included during each assay run to verify that the test is working properly

Quality Assurance – defined as the overall program used to ensure that the final results being reported by the laboratory are correct

Quality Assessment

Quality Assessment (Proficiency Testing) – means to determine the quality

of the results generated by the laboratory. Quality assessment is a challenge

to the effectiveness of the QC and QA programs that are in place.

Variables Affecting Quality of Results

Educational background and training of personnel

Selection and condition of specimens

Controls used in test runs

Reagents

Equipment

Interpretation, recording and reporting of results

Landscape of Laboratory Testing

Diagnostic lab testing single highest volume activity in

healthcare

Drives 70% of decision-making across medicine

12.8 billion tests performed each year in U.S.

Large scale analysis of 1.6 million results from 46 of 50

most commonly ordered laboratory tests

On average, 30% of all tests are probably unnecessary

Overutilization and underutilization are wide spread

Molecular ID Testing

Plays major role in clinical care of patients with infectious diseases

Identify specific pathogens

Determine and optimize appropriate therapy

Monitor response to therapy

Assess prognosis

Provide disease surveillance

Accurate and timely diagnosis required

Laboratory Testing Errors

Quality Use of Microbiology Tests

Using the right test

At the right time

For the right patient

With the right clinical condition

At the right price

With the right outcome

Molecular QC/QA

One of the largest stumbling blocks for traditional Clinical Microbiology Laboratories that start performing molecular testing

Molecular QC/QA more closely resembles Clinical Chemistry QC/QA

Newer CAP Microbiology Checklists have been refined and full sections now devoted to Molecular Microbiology

Many changes in QC/QA; many are new to traditional microbiologists

Molecular QC/QA Program

Written clearly defined program that ensures quality throughout the preanalytical, analytical, and post-analytical phases of testing

Capable of identifying systems problems and opportunities for system improvement

The laboratory must be able to develop plans for corrective/preventative action based upon data from its QC/QA system

Monitoring Testing Process

Importance of Quality Control

Ensure accuracy and reproducibility of lab test results used in patient care

Ensure integrity and confidence in test results

Provide safety in management of patients

Comply with local, regional, and national regulations and lab accreditation requirements

Maintain staff morale and reputation of lab

Types of Molecular Assays

Unmodified FDA-Cleared/Approved Tests

Modified FDA-Cleared/Approved Tests

Laboratory-Developed Tests

Qualitative versus Quantitative Tests

Pre-Analytical Phase

Addresses activities that occur prior to testing

Test requests/ordering

Patient identification

Specimen collection and transport

Specimen handling and processing

Rejection of specimens

Test Requests/Ordering

Develop well-designed order entry (EMR or manual) Allow for sufficient identification of patient and

ordering physician, the test requested, and pertinent clinical information

Should include date and time of specimen collection and specimen type; information on collector

Should accompany patient’s specimen to the laboratory

Ensure that clinical indications are appropriate for test(s) requested

Specimen Collection

Specimen collection, transport, handling and processing can have a significant effect on final test results; integrity of target nucleic acid sequence must be maintained

Establish clearly defines criteria for monitoring proper specimen collection, labeling, preservation, transportation, and storage of specimens

Establish educational programs

Develop firm (but fair) rejection policy

Guidelines for Specimens

Clearly defined method(s) of collection of specimens from all sources

Sample type and quantity of specimen

Preparation of collection site

Timing of collection during disease

Specimen quality/adequacy

Collection and transport devices

Identification of specimens

Appropriate transport conditions

Storage time, temperatures, and conditions

Specimens for RNA Detection

Specimens for RNA detection require special collection and handling conditions

RNA may be degraded by RNases

Use method that stabilizes RNA until specimen can be processed and stored appropriately

Analytical Phase

Written Procedure Manual Nucleic Acid Extraction and Storage Contamination Control

Laboratory Design Laboratory Practices

Testing Controls Test Performance Equipment Maintenance Personnel Competency Proficiency Testing Accreditation

Procedure Manual

Written instructions that contain sufficient detail that qualified laboratory personnel can perform tests consistently and accurately

Must include: Principle of Test Calibration

Clinical Significance Reference Ranges

Specimen Requirements Calculations

Reagents Needed Result Reporting

Procedural Steps Test Limitations

Quality Control References

Yearly review by Director or designee; review all new procedures and revisions prior to implementation

Electronic or paper procedures acceptable

Nucleic Acid Isolation

Adequate procedures for release and isolation of target nucleic acid

Keep process simple and minimize number of manipulations

Maintain integrity of target while inactivating or removing inhibitory or interfering substances

Yield, purity, intactness can be measured Ideally, isolated nucleic acid should be tested

immediately; if delay, store under appropriate conditions until testing can be completed

Nucleic Acid Isolation/Preparation

Contamination Control

During specimen collection and transport

During initial specimen handling and

processing by lab

During nucleic acid extraction

During amplification

During product detection

Contamination of reagents

Laboratory Design

Molecular space should be divided into at least 3 separate work areas: Area 1 – Reagent preparation area/room

Area 2 – An area/room for specimen processing/nucleic acid extraction

Area 3 – An area/room for amplification and detection

*Area 4 – For preparation of controls, calibrators, standards

Laboratory Practice 1

Dedicated supplies and reagents for each work area

Use plugged aerosol resistant pipette tips

Use closed systems or methods to control product carryover (e.g., uracil-N-glycosylase (UNG) to inactivate amplified product)

Use appropriate positive and negative controls

Meticulous cleaning is a must!

Laboratory Practice 2

Follow unidirectional flow from pre-PCR to post-PCR

Pulse-spin reagents, samples, controls; Add reagents to vials before samples

Use gloves and protective clothing which are dedicated to each area

Decontaminate pipettes and instruments and wipe work surfaces with 10% bleach; rinse with 70% ethanol

Meticulous cleaning is a must!

Laboratory Practice 3

Carefully open and close all tubes to minimize aerosolization of contents

Keep all non-essential tubes closed during sample addition

Order of preparation and loading of samples for PCR should be actual clinical specimens first, followed by positive and then negative controls

Hoods and/or glove boxes (with UV light) are often necessary for nucleic acid isolation and set up of amplification reactions

Meticulous cleaning is a must!

General Rules of Thumb

Traffic can only go from clean to dirty area

Objects can only go from clean to dirty area

Reagents and supplies needed in a clean area need to be prepared and stored in a clean area

Equipment used in clean area cannot be moved and used in dirty area

Personal safety equipment cannot move with you from dirty to clean area

Meticulous cleaning is a must!

General Rules of Thumb

Protocol books, worksheets, sample tubes, test data and results cannot move from dirty area to clean area

Can employ color coded lab coats, pipettes, safety glasses, racks, etc. to monitor traffic

Have respect for RNases if working with RNA

Meticulous cleaning is a must!

Reagents & Solutions

All test reagents and controls should be stored properly and in a manner which minimizes contamination and degradation

Prepare and divide into single-use aliquots Store in area that is separate from specimen

preparation and post amplification Do not store in frost-free freezers Dedicated equipment and supplies should be used Reagents can be premixed into single-use, master

mixes Meticulous cleaning is a must!

Reagents

Test new lots for purity, functionality, concentration, and contamination before use

Compare performance of new reagents to ones currently in use

Test new reagents against reference materials; monitor quality with use

Keep meticulous QC records

QC for Molecular ID Assays

Qualitative

Controls (match matrix) Amplification: Positive for each

analyte; Negative (reagent blank or specimen) and Sensitivity

Every run, establish range values, and monitor

Internal (inhibition) for extraction and amplification

Calibration verification (if use cutoff to distinguish positive/negative) – weak positive

Every 6 mo; after any major system component change and QC failure

Quantitative

Controls (match matrix) Amplification: Positives at ≥2

levels and Negative

Every run, establish range values, and monitor

Internal (inhibition) for extraction and amplification

Calibration verification (verify cutoff)

Analytical measurement range verification (low, mid, high) Every 6 mo; after any major

system component change and QC failure

Adapted from EM Burd 2010 Clin. Microbiol. Rev. 23:550-576

Daily QC

QC of Multiplex Tests

Ideally, include a positive control for each of the nucleic acid targets included in test for each run

Often impractical if number of targets is large

Multi-target control or rotate single target controls

Example of Multiplex Controls

ANALYTE M211 M212

Adenovirus Positive Negative

Coronavirus 229E Negative Positive

Coronavirus HKU1 Negative Positive

Coronavirus NL63 Negative Positive

Coronavirus OC43 Negative Positive

Human Metapneumovirus Positive Negative

Human Rhinovirus/ Enterovirus Entero 1 Positive Negative

Entero 2 Positive Negative

HRV1 Positive Negative

HRV2 Positive Negative

HRV3 Positive Negative

HRV4 Positive Negative

Influenza A H1-2009

FluA-H1-

2009

Positive Negative

Influenza A H3 FluA-H1-pan Positive Positive

FluA-H3 Positive Negative

FluA-pan1 Negative Positive

FluA-pan2 Positive Negative

Influenza B Negative Positive

Parainfluenza Virus 1 Positive Negative

Parainfluenza Virus 2 Negative Positive

Parainfluenza Virus 3 Negative Positive

Parainfluenza Virus 4 Positive Negative

Respiratory Syncytial Virus Negative Positive

Bordetella pertussis Negative Negative

Chlamydophila pneumoniae Negative Negative

Mycoplasma pneumoniae Negative Negative

M211 and M212 Main Molecular

Controls run periodically and with each

new lot. No need for negative control.

Assay contains internal RNA process

and second stage DNA PCR controls.

FilmArray RP Panel M210 (Maine Molecular Controls)

QC Acceptance Limits & Statistics

Inhibition Assessment

Distribution of QC Results

W. Greg Miller. 2011. Chapter 10, Quality Control in Henry’s 22nd ed.,

Elsevier, Philadelphia, PA

Types of Errors

Random Systematic Random & Systematic

BLUE = True Value Average

GREEN = Sample Value Average

Westgard QC Evaluation Rules

Used to define specific performance limits for a particular assay

Can detect both random and systematic errors Warning rules and mandatory rules have been

applied over the years Violation of warning rules triggers a review of

test protocol, reagent performance and equipment calibration

Violation of mandatory rules triggers rejection

Levey-Jennings Plots

https://www.westgard.com

Warning Rules

Mandatory Rules

Westgard Sigma Multirule QC

Primers/Probes - LDTs

Commercial Reference Materials

Quality control material may be difficult to find for less common pathogens

Number of companies now provide reference material for a growing list of pathogens Acrometrix

Advanced Biotechnologies

American Type Culture Collection (ATCC)

Main Molecular Quality Controls, Inc.

Seracare Life Sciences

Zeptometrix

International Standards

WHO, NIST (US) and NIBSC (UK) have been active in establishing IS for viruses

HIV-2, HIV-2, HAV, HBV, HCV, Parvovirus, HPV-16, HPV-18, CMV, and EBV

Primary (reference) standards are assigned quantity values accepted as “gold” standards

Provide means to normalized inter-laboratory and method to method variability

Secondary standards are larger pools of material that are quantified by direct comparison to the primary standards

Calibrators should be traceable to IS if available

Equipment & Instruments

Ensure equipment and instrument function

Establish system for monitoring

Detection of drift, instability, or malfunction before problem affects test results

Regular preventive maintenance and cleaning

If multiple instruments, functions must be check against each other every 6 months or after repair

Keep meticulous maintenance, service and repair records

Establish service contracts with vendor

Quality Assurance/Monitoring

A continuous process

Documents that a test which has already been verified/validated is repeatedly giving the expected results as test is performed over time

Confirms that test continues to perform according to laboratory’s requirements and its intended use

Quality Monitoring Process

Personnel competency assessment Meticulous record keeping Vigilance in laboratory Quality control monitoring Quality improvement Internal and external proficiency testing Correlation with clinical findings; interacting with

healthcare providers Trend analysis Storage of specimens for follow-up

testing/evaluations Integral part of a labs QA program

Quality Control Monitoring

Have system in place to continuously monitor performance of positive and negative controls, standards, and calibrators

Perform wipe testing – can reinforce need to disinfect the environment daily

Monitor prevalence rates of specific viral diseases

Training & Competency

Establish a complete program of employee training, verification, and competency

Verify personnel competency at least semiannually during first year and at least annually thereafter

Reverification necessary if change in method or instrumentation

Assay Training

Determine magnitude of training and skill level required

Each operator must demonstrate proficiency with the method; may require certification

Should include understanding of: Sample collection, handling and storage

Reagent handling and storage

Proper test protocol

How to interpret and report results

QC/QA for the system

Competency Assessment

Direct observation of routine patient test performance, including specimen handling, processing, testing, instrument maintenance, and function tests

Monitor test result recording and reporting

Review worksheets, QC records, PT results, and PM records

Wet testing with proficiency samples

Assessment of problem-solving skills

Proficiency Testing/EQA

Challenge for PT providers to keep up with pace of new test development and new technology in molecular diagnostics

Many laboratories need combinations of commercially-available and alternative proficiency testing programs

CAP Proficiency Tests

Alternative performance assessment

Integrate into the workload

Rotated among people at the bench

Corrective action on failed PT surveys

Proficiency Testing/EQA

General principles and rules of PT should be applied

PT samples should challenge the diagnostic scope of the test Range of genetic diversity (QUAL) and extremes of

measurement ranges (QUANT)

Samples should be relevant to current test targets and compatible with available technologies

Samples should free of “matrix” effects and reflect the range of specimens types tested

Source of nucleic acid can have a significant effect on participant performance

Alternative PT Assessment

Used when no commercial PT products available

May also be used to challenge pre-analytical and post-analytical processes not addressed by commercially available PT

Example - Quality Control for Molecular Diagnostics (QCMD)

Participate in ungraded/educational PT

Split sample analysis with other laboratories

Establish in-house PT Program

Use regional pools (can be purchased)

Clinical validation by chart review

Internal or External Split-Sample PT

Minimum of 3-5 samples per testing event

At least 2-3 testing events at equal intervals per year

Use full range of positive and negative specimens in panel

Samples should represent clinical specimens

Use of in-house specimens or manufactured reference materials

Corrective Actions

Mistakes will occur; avoid assigning blame

Clearly document what has happened

Thoroughly investigate, take immediate corrective action, institute preventive measures

Errors often result of larger process issues with reagents, personnel, equipment, workflow

Ultimate goal: DO NO HARM

Post-Analytical Phase

Laboratory Test Reports

Timeliness of Reporting

Correction of Errors

Patient Confidentiality

Recording & Reporting Results

Laboratory test reports should be clear, concise, accurate, and fully interpretive

Verify all results before reporting as final Monitor and report specimen (in)adequacy Final report should include a summary of test method,

result(s) and interpretation if appropriate Gene target (biological false positives and negatives) Amplification method Lower limit of detection ASR disclaimer if LDT Quantitative (AMR, LLQ vs. LLD, log10 conversion, and action

thresholds) Name and location of laboratory

Good practice to have reports reviewed by 2nd party

Interpretation & Reporting of Results

Significance of results must be evaluated with respect to agent, specimen site, and clinical situation

Interpretive criteria must include provisions for equivocal results

Reporting of results should include as much information as necessary for clinician to properly interpret the results

Interact with healthcare providers

Quality Management: Turnaround Times

Provide test results in a timely manner to have greatest impact on patient care and management

Have system to monitor TATs Are TATs appropriate for the intended

purpose of the test? Consultation with medical staff Batch testing versus on-demand Examples: HSV and enteroviruses in CSF;

respiratory viruses; viral load tests; MRSA and C. difficile

Quality Management: Statistics

Additional Items

Correction of Errors – Laboratory should have a system in place for timely review and correction of clerical and analytical errors

Confidentiality – Testing records and reports should be maintained in a manner that preserves patient privacy and confidentiality

Total Quality Management

Comprehensive organizational approach

Focus on continually improving quality and efficiency with which lab operates

Includes not only QC and QA, but other technical and administrative considerations that may indirectly influence quality and efficiency