A procedural approach for the identification of sources of uncertainty associated with GM quantification and real-time quantitative PCR measurements

Eur Food Res Technol (2007) 226:7–18DOI 10.1007/s00217-006-0502-y

ORIGINAL PAPER

A procedural approach for the identification of sourcesof uncertainty associated with GM quantificationand real-time quantitative PCR measurementsMalcolm Burns · Hernan Valdivia

Received: 13 June 2006 / Revised: 4 October 2006 / Accepted: 8 October 2006 / Published online: 15 November 2006C© Springer-Verlag 2006

Abstract Estimation of measurement uncertainty is im-portant as it provides objective evidence that an assay orinstrument is ‘fit for purpose’, and is a requirement for ac-creditation to ISO17025. Technological developments withinthe bioanalytical laboratory have progressed rapidly in thepast few years. However, the measurement science associ-ated with these technologies has not yet fully matured. Theprovision of a full measurement uncertainty budget is oftendifficult because of a lack of standardised approaches andthe constraints associated with time, finances and expertise.A procedural approach for the identification of factors thatcan account for measurement uncertainty in quantificationof genetically modified (GM) ingredients using real-timequantitative PCR is presented here. This example illustratesan approach for the identification of factors that can be ap-plied to related assays concerned with GM quantification,can be utilised as part of existing uncertainty approaches,and can provide the basis for making quantitative estimatesof uncertainty that will contribute towards a full measure-ment uncertainty budget.

Keywords Measurement uncertainty . GMOs . Real-timequantitative PCR

Introduction

Accreditation to ISO/IEC 17025 stipulates that it is a require-ment of testing laboratories to apply procedures for the es-timation of measurement uncertainty [1]. The expression of

M. Burns (�) · H. ValdiviaBio-Molecular Innovation, LGC Limited,Queens Road, Teddington, Middlesex, TW11 0LY, UKe-mail: [email protected]

uncertainty components typically takes the form of standarddeviations, and guidelines exist regarding the manipulationand combination of these variance components. Uncertain-ties can arise from several sources that must be evaluatedthrough statistical or other means. Approaches for the quan-tification of sources of uncertainty include using precisionstudies to evaluate random variation; estimation of system-atic variation within an assay; application of model fitting;use of published information; and sometimes, sole relianceon experience when no other information is available.

Many publications broadly support these guidelines andapproaches to measurement uncertainty (MU) estimation [2–5], and any result from an analytical laboratory must be givenwith an associated uncertainty estimate, if that result is toconform to quality regulations. The benefits of implement-ing MU estimation include its usefulness as part of methodvalidation, certification, and accreditation. Measurement un-certainty estimation enables the identification of factors thatcontribute to the variability associated with a reported result,and can also facilitate the qualification of an assay as being‘fit for purpose’ when it is shown to be operating withindefined performance criteria. Furthermore, uncertainty esti-mates can be used diagnostically by identifying where rou-tine measurement problems occur, and contribute to feasi-bility and risk assessment studies by identifying sources ofpotential variability, as well as being an outcome associatedwith method validation [6].

Whilst in principle, the application of MU estimationshould follow a simple and systematic approach to any as-say, the present reality is very different within the bioanalyt-ical measurement community. There is currently no singleapproach to uncertainty analysis that is adequate to fit allneeds, and the assessment of MU is assay-specific, requiringconsideration on a case-by-case basis. It has been previouslysuggested that a systematic approach to method validation be

Springer

8 Eur Food Res Technol (2007) 226:7–18

taken [5]. Such a modular method validation approach com-prises partitioning an entire assay into its component parts,or modules, which can then be validated independently andMU estimates combined. Currently, there are no systematicguidelines for the writing of scientific protocols, and this is akey issue that needs to be addressed within the bioanalyticalmeasurement community.

The use of replication and the estimation of a MU bud-get for an assay are recognised as being critical in principle.However, on the practical side, the implementation of theseis not always feasible, requiring time, money, and expertise.Although guidelines exist regarding MU principles, these areoften application-specific, or may not be detailed enough tobe applicable for all measurements that span the bioanalyt-ical community [7]. Additionally, with respect to scientificterminology, issues have been highlighted where contradic-tions occur in some of the definitions associated with uncer-tainty principles [8]. It has been reported that many resultscurrently produced within the bioanalytical field have verylarge, or even unknown, MU estimates associated with them[5]. Because of these difficulties, it is often considered betterby some laboratories to spend the time on different, moretangible outputs, for example production of a second set ofdata to meet customers requirements. Furthermore, it is alsorecognised that there is a need for end-users of data to betrained in the terminology of MU, in order for results to beinterpreted correctly. If this is not enabled, then additionalMU can occur in the results interpretation [9].

The measurement science associated with the bioanalyt-ical community is still in its developmental stages, and hasnot grown as rapidly as the technological advances that arecurrently being witnessed within the modern laboratory. Thispresents a problem as whilst the benefits of estimating MUassociated with an assay are appreciated, the actual processof applying uncertainty estimates with any confidence is notwell developed, and there is a recognised need for a standard-ised way of facilitating this in analytical laboratories [10].The process of estimating MU is assay-specific, and evenwithin the relatively well defined field of quantification ofgenetically modified (GM) ingredients in food samples, theplethora of platforms, techniques, and lack of standardisa-tion in handling and interpreting data further confound theapplication of MU principles [9].

The lack of published examples in the literature re-garding full MU budgets associated with GM analyticalprocedures further highlights the problems in this area[6, 11, 12]. This aspect is in constant development, andis being addressed at an international level through suchprojects as the Framework 6 EU Co-Extra programme(http://www.coextra.org/). One objective of this project isto address MU aspects related to GM analytical tech-niques, but currently there is a lack of solid results in thisfield.

Scientists have recognised the inherent problems andcomplications faced by the bioanalytical community in at-tempting a full MU budget associated with GM analyticalwork, and a ‘modular method’ validation approach has beenproposed [5]. This essentially splits up the GM estimationprocess into component systematic modules that can poten-tially be validated independently, and thus MU estimates canbe made for each separate module. Practical data to reflectthe validity of the modular method approach has yet to bepublished, but the approach serves to emphasise that the pro-cedure to make MU estimates associated with GM analyticaltechniques is not straight forward, and there is still a deficitof published techniques with an associated MU budget.

One aspect that helps ensure a more representative esti-mation of MU is a good experimental design that facilitatesreliable conclusions to be drawn. The use of an appropri-ate level of replication, for example through samples, days,repeat runs, etc., provides an estimate of sampling variancewhich is inherent and representative of any assay, and canbe used as the basis for comparison of effects. In general, anincreased replication factor facilitates increased confidencein the results as a better estimate of the population mean isderived.

The elucidation of factors that can contribute towards MUcan benefit from simple ‘cause and effect’ analyses. This toolis valuable for structuring uncertainty analysis, is a powerfulaid to experimental design, and is relatively simple to apply.An approach to cause and effect analysis would typicallyinvolve examining the data set, comparing the data with alllisted input effects, followed by combining all of the sig-nificant contributions. This can be facilitated by specifyingexactly what is being measured, and for each stage of themeasurement procedure, listing the sources of uncertaintyand adding contributory factors.

This paper presents a procedural approach for identifyingthe sources of uncertainty involved in GM quantification andreal-time quantitative polymerase chain reaction (RT-QPCR)measurements. The broad context of RT-QPCR [14] and thenumber of different real-time platforms currently available,make this a suitable system on which to model this proce-dural approach [15–17]. This approach utilises some of theconcepts inherent in the simple ‘cause and effect’ analysis,but extends this by using a systematic methodology and ap-plying this to the complicated procedures involved in the fieldof GM quantification. There is currently a lack of publishedinformation regarding potential sources of MU in the fieldof GM quantification and RT-QPCR. The results presentedhere illustrate a way of identifying factors that can causesignificant variability in GM quantification, facilitate the ap-plication of the approach to enable identification of factorscausing variability in similar GM quantification methods,and provide a basis with which to proceed for the objectivequantification of the uncertainty estimates.

Springer

Eur Food Res Technol (2007) 226:7–18 9

Materials and methods

Experimental assay

A specific example has been included in order to demonstratethe concept of the procedural approach. This example usedspecific laboratory apparatus, but still had many features incommon with other quantitative tests for GM ingredients.For illustrative purposes, this assay was based on the quan-tification of Roundup Ready r© genetically modified soya. ART-QPCR amplification technique that targets the amountof product arising from the wild-type soya lectin gene (en-dogenous probe) and the amount of product arising fromthe Roundup Ready Soya transgene (transgenic probe) wasutilised.

Through out the experimental assay, manufacturer’s in-structions and laboratory grade reagents were used to help en-sure standardisation of results and to minimise any extrane-ous variation. Amplification reactions were performed withthe TaqManTM PCR Core Reagent Kit (Applied Biosystems,UK). Duplex reactions containing both the endogenous andtransgenic primers/probes were used, in conjunction withdNTPs, AmpliTaqGold DNA polymerase, AmpErase uracilN-glycosylase (UNG), MgCl2 and TaqMan buffer A fromthe Core Reagent Kit. Reactions were run on the AppliedBiosystems 7900HT Fast Real-Time PCR System followingmanufacturer’s instructions and reaction conditions whichhave been previously described [11, 17].

The experimental data set consisted of one analysis platecontaining six replicates of five certified reference materials(CRMs) used as calibrants, and six replicates of one sampleunknown along with relevant controls. The CRMs consistedof 0, 0.1, 0.5, 1, 2, and 5% (m/m) Roundup Ready GMsoya, obtained from the European Commission, Institute forReference Materials and Measurements [18].

DNA extraction was carried out using a commer-cially available DNA extraction kit (Promega Wizard DNACleanup Kit, Promega, USA). DNA quantification was per-formed on the Nanodrop (ND-1000 Spectrophotometer), anda real-time quantitative PCR platform was used for the gen-eration of sample results (Applied Biosystems 7900HT FastReal-Time PCR System, Applied Biosystems, USA). A com-mercially available software program was used for the man-ual data manipulation (Microsoft r© Office Excel, 2003).

Procedural steps for GM quantification

The reporting of a result associated with quantitative GManalysis may be thought of as a series of procedural steps. Inorder to identify factors that can contribute towards MU in theend result, a procedural approach was taken so as to accountfor the methods and routes involved GM quantification. Thecurrent study focuses on the analytical steps involved in a

Table 1 Procedural approach for GM quantification using RT-QPCR

Procedural step Name

Sampling effects1 DNA extraction2 DNA quantification and dilution3 PCR setup4 Platform operation5 Software analysis6 Manual analysis

Interpretation of results

typical molecular laboratory as defined by the quantificationand RT-QPCR measurements (steps 1–6, in Table 1), andattempts to identify sources that can contribute MU to thisprocess.

Results and discussion

Procedural approaches for identification of sourcesof uncertainty

DNA extraction

Procedure The extraction of DNA typically involves theevaluation of a specific mass of the sample and calibranton a balance (Fig. 1). This is followed by addition of a spec-ified volume of extraction buffer to the sample, inclusive ofsuch reagents as guanidine thiocyanate and proteinase K. Inthis case, the sample is then vortexed for a set period of timebefore being incubated overnight at 56 ◦C in a hybridisationoven. The following day the sample is centrifuged and a setvolume of the supernatant removed into a new tube, and thiscentrifugation and removal step is repeated. A clean-up resinis then applied to the sample, and the sample pipetted intoa syringe barrel in the presence of 80% isopropanol and ap-plied to a vacuum manifold. The elute is transferred to a newtube and centrifuged before being re-suspended in 1 × TE(10 mM Tris–HCl, 1 mM EDTA, pH 7.5), pre-warmed in astandard PCR thermal cycler.

Sources of uncertainty The DNA extraction process is likelyto be a procedural step that can contribute a relatively highamount of MU to the GM estimation process, as it con-tributes to the quality of the DNA extracted. If the DNA isnot of sufficient quality, the downstream PCR efficiency maybe affected. The quality of DNA can be affected by its size,stability, level of degradation, structural integrity, and pres-ence of inhibitors and modifications, which may be inherentin the sample matrix [19]. Additionally, variability can arisefrom the differential amplification efficiency associated withthe endogenous and transgenic targets (for example, when

Springer


Target availability

Amount and quality of DNA

Equipment

Upstream procedural steps Consumables

Sampling effects

Extraction kit

Balance

Thermal cycler

Vortex

Micro-centrifuge

Vacuum manifold

Hybridisation oven

Pipette tip: liquid retention

Extraction buffers

Pipettes

Extraction efficiency oftransgene:endogenous ratio

Fig. 1 Flow diagram to showfactors affecting DNA extraction

in the presence of inhibitors) [14]. It is important to use ‘fitfor purpose’ techniques, such as fully validated methods, toensure that potential variability is minimised.

Published work has shown that the estimated GM contentof a sample unknown can be affected by the extent of pro-cessing associated with the matrix [5, 20, 21]. A review ofsome factors that may account for variability in GMO analy-sis has recently been published [22] and this has highlightedthe need for ‘matrix-matched’ materials, where the matrix ofthe calibrants match that of the sample unknowns that willbe tested. Whilst the DNA extraction efficiency may havean impact upon MU, evidence suggests this does not havemuch influence upon the DNA template quality [23]. As theestimated GM concentration is based upon the relative quan-tification of the transgene target to the endogenous target, ithas been postulated that the MU due to the total amount ofDNA extracted is likely to be small.

DNA isolation kits are available, that may use ingredientswith different binding efficiencies between kits. Manufac-turer’s instructions should typically list the percentage re-covery of the total amount of DNA, dependent upon elutionvolume and starting amount of DNA (e.g. for the example inthis paper, we would expect to recover 40 ng of DNA basedon a 30 µl elute volume used on 50 ng of starting DNA). Ifthe DNA isolation kits impact upon the selective binding ofdifferent size DNA fragments, further MU can occur [14].The effect of template degradation, and also of differentialdegradation of the endogenous and transgenic targets cannotbe ignored.

A further study looked at the effect of four commer-cially available DNA extraction kits and their effect uponthe amount of DNA extracted, the level of degradation, andthe effect on the GM quantification using RT-QPCR [24]. Itwas found that the amount of DNA extracted could vary up

to around 40-fold dependent upon the extraction kit and thesample source, and the level of degradation of the extractedDNA was different as assessed by gel electrophoresis. In ad-dition, statistical techniques were used to show that signifi-cant differences could occur between the calibration curvesbased on the different DNA extraction methods.

Measurement uncertainty arising from such aspects asso-ciated with DNA extraction is often large, but these factorsare often not within the direct control of the laboratory sci-entist.

Additional sources of uncertainty in the DNA extractionprocess can be evaluated through the use of certificates as-sociated with the equipment. Dependent upon the specificGM quantification assay used, different laboratory equip-ment may have been utilised. Regular servicing associatedwith the maintenance of laboratory equipment will includecalibration of the equipment, and this should provide a certifi-cate with a measurable value associated with the equipment’sperformance (for example, the Certificate of Calibration as-sociated with a balance may give the MU associated witha reading as 200 ± 0.016 or 50 ± 0.0015 ng). The servicecertificate associated with a pipette may include values forits maximum and minimum volumes, and associated stan-dard deviation and coefficient of variation. Additionally, MUestimates can be found from published material related tothe manufacturer’s certificate associated with new labora-tory equipment. The DNA extraction process is also reliantupon the use of pipettes; hence, it is important that theseare calibrated on a regular basis. Calibration certificates willtypically list the performance characteristics such as the co-efficient of variation (CV) of the pipettes associated with theupper and lower volumetric measurements (e.g. for a 100 µlpipette, the CV for a 20 µl volume may be stated as 0.3%,and for the 100 µl volume the CV may be 0.15%).

Springer


DNA amenable for analysis

Sample volume Reagents

Upstream procedural steps

Spectrophotometer

Sampling effects

Certificate

Liquid retention

Sample carryover

Replication factor

DNA extraction

Light source interference

Reference

sdH20

Consumables Pipettes

Tubes

Low retention tips

Certificate

Dynamic range

Sample homogeneity

Fig. 2 Flow diagram to showfactors affecting DNAquantification and dilution

DNA quantification and dilution

Procedure It is important to ensure that the sample is suffi-ciently homogenous before quantification, and that the spec-trophotometer has been cleaned to ensure no sample car-ryover between readings (Fig. 2). Based on manufacturer’sguidelines, a 1–1.5 µl drop is taken from the sample for mea-surement, and compared to a suitable reference blank in theform of either water or 1 × TE. The apparatus is shieldedfrom any strong external light sources that may interfere withthe measurement. In order to ensure accuracy, three replicatereadings of the same sample are taken. Liquid retention of thesample may occur, and the dynamic range of the spectropho-tometer is 2–3700 ng/µl of dsDNA. There is a reproducibilityof 0.2 mm pathlength to calculate the absorbance, and thespectrum wavelength range is 220–750 nm.

Sources of uncertainty Uncertainty can arise from the per-formance of the machine chosen to do the quantification, butprovided that the manufacturer’s guidelines are adhered to inits operation, and a sufficient replication factor is taken intoaccount, there should be reasonable confidence in the results.Calibration certificates or manufacturer’s instructions shouldgive estimates of the uncertainty associated with the instru-ment. For example, the reproducibility associated with theNanodrop ND-1000 Spectrophotometer in the sample rangeof 2–100 ng/µl is ± 2 ng/µl. Uncertainty can also arise fromthe use of high-retention plastic-ware in the laboratory, par-ticularly when conducting serial dilutions following DNAquantification. In order to minimise any potential MU fromsuch sources, it is advisable to utilise specific consumablessuch as low-retention pipette tips, and non-stick PCR tubes.If the quantity of DNA is above a threshold that permits am-plification in the PCR process, the actual amount of DNAas determined by the quantification process may only add

a small amount of uncertainty to the evaluation of the GMcontent of sample unknowns, as the amount of transgenetarget is normalised internally to the total amount of DNApresent in the sample [5]. It is likely that the quality of theDNA that is extracted from procedural step 1, will accountfor more MU than this procedural step.

PCR setup

Procedure A 96 well micro-titre plate is used for the RT-QPCR setup, and this typically can consist of up to six repli-cates for all of the six calibrants, the sample unknown, andthe appropriate controls (Fig. 3). To help ensure no cross-contamination, the PCR setup is conducted within a UVcabinet (260 nm) that will denature or break dsDNA andssDNA. A range of pipettes is used with low-retention filtertips, along with relevant consumables for liquid handling.Two sets of forward and reverse primers along with fluo-rescently labelled probes are used: one for the transgeneelement, and the other set for the endogenous target. AllPCR components are made into a ‘master mix’ that includesthe relevant primers/probe set, PCR buffer, sterile water, Taqpolymerase, magnesium, etc., and this is aliquoted into themicro-titre plate before finally adding the DNA template.Optical adhesive covers or plastic caps are then added to thetop of the samples in the plate.

Sources of uncertainty If a validated protocol is followed,criteria such as the sequence, length, GC content andmelting temperature of the primers will be fixed, and theprimers will have been tested for cross-reactivity and self-complimentarity. To give un-equivocal identification of thespecific GM transgene insertion into the genome, ‘event-specific’ primers are often used. These target the junctionsequence between the new DNA elements and the site of

Springer


Samples ready to run on RT-QPCR

platform

Primers and probes Equipment

Upstream procedural steps

Replication factor

Sampling effects

PCR blanks

Consumables

Quality – manufacturers report

DNA extraction


Low retention tips

Reagents

Tubes

Dilution

Centrifuge

Sample unknowns

Calibrants

UV cabinet

Pipettes

Extraction blanks

Plates / optical adhesive lids

Between plate variability

Fig. 3 Flow diagram to showfactors affecting PCR setup

insertion into the plant genome, thus affording high speci-ficity. Additionally, primers and probes should be synthe-sized from a manufacturer of good repute using criteria suchas high purification. To avoid primer degradation through re-peated ‘freeze–thaw’ actions, separate aliquots of the primerscan be made and stored at 4 ◦C until use. Further guidelinesto ensure that MU arising from the design of primers andprobes is minimised can be found elsewhere [25]. The speci-ficity associated with the primers is important, and the choiceof a validated protocol would help ensure the ‘fitness for pur-pose’ of the assay.

A recent study based on the validation of PCR methodsfor GMO analysis examined the reproducibility associatedwith using different target sequences for the PCR amplifi-cation (e.g. 35S promoter compared to a transgene-specificsequence) across three European food control laboratories.The results showed that the expanded MU associated withthe estimated GM concentration could vary between 20 and73% of the mean estimated value, dependent upon whichtarget analyte was used [26]. These experiments were basedon evaluating the reproducibility for different real-time PCRbased detection systems, but illustrated the magnitude of theboth the problem and the size of the MU associated withchanging detection systems.

There are a number of factors that could contribute MU tothe end result in this procedural step, but as long as standardoperating procedures, protocols, laboratory working instruc-tions, and other quality guidelines are strictly adhered to, theresults should be repeatable with a minimal impact upon theuncertainty budget. The choice of pipettes and pipette tips areimportant, as is adherence to strict working practices withinthe laboratory to ensure that there is no cross-contamination.Additionally, a suitably high level of replication will ensurethat the variability associated with the samples is represen-

tative of the original population. Recent work involving aninter-platform RT-QPCR comparison study showed that theuncertainty arising from using different PCR mixes can benearly twice that of the uncertainty from repeating the sameexperiment across different days, in terms of the measuredresponse from each platform [27].

Platform operation

Procedure For the RT-QPCR platform, the user needs to se-lect the appropriate 96- or 384-well block format (Fig. 4).Using the Setup tab in the software application, the detec-tion manager is used to create the correct detectors, which arethen added to the appropriate wells. This process is repeatedfor however many other dye layers are required. The sampleidentity, including name, task and quantity is added to theplate grid. The user needs to check that the correct passivereference dye has been chosen. From the instrument tab, thethermal profile conditions with an appropriate cycle numberneed to be verified. The user needs to check the auto incre-ment, ramp rate, and data collection points, as well as thecorrect volume of reaction before the analysis commences.

Manufacturer’s instructions regarding the platform oper-ation are followed throughout, and the plate seal needs to bechecked with the applicator, and the thermal adapter added.A typical run takes just under 2 h.

Sources of uncertainty There are a number of sources ofuncertainty associated with the platform operation proce-dural step. However, strict adherence to quality guidelinessuch as the platform manufacturer’s instructions, validatedprotocols, standard operating procedures, etc., should helpensure that the MU associated with this aspect is rel-atively small. Calibration certificates and manufacturer’s

Springer


System ready to carry out analysis

Detection properties Reaction volume

Upstreamprocedural steps

Thermal profile

Sampling effects

Auto increments

Data collection points

Manual operation

Sample set-up

DNA extraction

Test plate seal

Add thermal adapter

Open / close block

PC connection

Add detection layer to wells

Create detectors

Passive reference dye


PCR setup

Ramp rate

Cycling conditions

Fig. 4 Flow diagram to showfactors affecting platformoperation

instructions should have an estimate of the performance as-sociated with a particular RT-QPCR machine. The AppliedBiosystems 7900HT Fast Real-Time PCR System is the ex-ample used in this paper, and its specifications include atemperature accuracy of ± 0.25 ◦C, and has a ramp rate of± 1.6 ◦C/s in standard mode. A recent article was based onan inter-platform study using different PCR mixes across anumber of days on different real-time PCR platforms. Re-sults indicated that the MU arising from different real-timePCR platforms can account for over seven times as muchvariability as the day effect, in terms of the response variablefor the amount of DNA measured [27].

Software analysis

Procedure After the run has finished, the file is saved andtransferred to another computer for analysis with the cor-rectly installed software, via an appropriate media (e.g. net-work or USB memory stick) (Fig. 5). Analysis settings areverified (automatic or manual), and results viewed by select-ing the appropriate wells. A graph can be viewed to verifythat the amplification reaction has worked correctly, and it ispossible to change the response variable on the y-axis (deltaRn) from a linear to a log scale. Using the manual settings,the cycle threshold (Ct) value should be set as the mid pointof amplification on the amplification curve. The Ct thresholdvalue represents the cycle number above which any measur-able signal is assumed to originate from amplification of thetarget sequence, as opposed to any background interference.The exact protocol to follow is dependent upon the specificreal-time PCR platform used, but for the example illustratedin this paper, the manufacturer’s instructions suggest that thebaseline should be calculated by subtracting sufficient cy-cles from the first amplification cycle above the Ct threshold

value [28]. This process should be repeated for all detectorsused. Individual Ct values are generated for each of the re-action wells, and these Ct values are directly proportionalto the amount of target analyte in the sample DNA beinganalysed. The Ct values are then exported to another suitablesoftware package in a tab-delimited form for further manualanalysis.

Sources of uncertainty The final derivation of the Ct valueis dependent upon the manufacturer’s software, which canact as an unknown ‘black box’ as it is not clear how thevalues have been derived, and these values can be platform-specific. Whilst the mechanism behind how the Ct values arederived is not clear, it may not be an invalid assumption thatthe MU associated with this aspect may be fairly minimalbecause of the automated and repeatable process. However, itis still possible for manual settings for the threshold cycle andbaseline cycle to be used in the derivation of the Ct values,which can be more subjective in nature and potentially giverise to high levels of uncertainty. Again, dependent upon themethod of analysis and the automatic computations involved,the manual setting of the baseline and threshold cycle is likelyto have a variable contribution towards the overall MU ofthe assay. Strict adherence to the manufacturer’s guidelinesfor this will help ensure that MU arising from this area isminimised. Based on adherence to these quality assuranceaspects, it has been reported that the CV associated withderivations of the Ct value can be as low as 1.5–2% [14].

Manual analysis of results

Procedure The previous software analysis stage generatedthe individual Ct values associated with the samples. Theseresults can then be exported into another software program

Springer


Raw Ct values

Analysis settings

Upstream procedural steps PC

Sampling effects

Software version

Amplification plot

Automatic Ct: “Black box”

DNA extraction


Transformation ofresponse variable

Manual baseline setting

Platform operation

PCR setup

Manual Ct

Manual threshold setting

Fig. 5 Flow diagram to showfactors affecting softwareanalysis

Estimation of % GM content of sample

unknowns

Upstreamprocedural steps

Handling of duplex / singleplex data

Model fitting

DNA extraction

Type of model

Software version

Platform operation

PCR setup

Software analysis

Normalisation (Delta Ct)

Mean Ct value: confidence given by replicates

DNA quantification and dilutions

Sampling effects

Manual handling of results

“In-house” methods forreporting

Fig. 6 Flow diagram to showfactors affecting manualanalysis of results

for manual manipulation of the data, often in tab-delimitedtext format. There are a variety of approaches to use for themethod of quantification in the manual analysis of results,inclusive of separate calibration curves based on the mea-sured response from the transgene and endogenous targets,and use of delta delta Ct calculations [14, 25, 26, 29–31]. Inthe example used here, the data is manually manipulated sothat a delta Ct value is calculated for each sample or cali-brant, based on the mean of the Ct values associated with theendogenous target, subtracted from the mean of the Ct valuesassociated with the transgenic target (Fig. 6). This effectivelyfacilitates a normalisation of the transgene response, as thetotal amount of amplifiable DNA is taken into account.

This delta Ct is then used in the construction of a calibra-tion curve, where the logarithm of GM content associatedwith the calibrants (independent variable) is plotted againstthe delta Ct value (dependent variable). This transformation

of the x variable using logarithms helps normalise the dataset and facilitates a linear relationship between the x and yvariables. The best fitting model to explain the relationshipbetween the two variables should then be used, although asimple linear regression model is invariably applied. The re-sultant regression equation is then utilised to estimate the GMcontent of any sample unknowns, based on their associateddelta Ct values.

Sources of uncertainty Uncertainty can arise from the man-ual manipulation of the data when combining and comparingvalues, for example through translation errors, when refer-encing incorrect cells, when specifying a function to calcu-late the average, etc. Additionally, different versions of thesoftware should be validated to ensure that these give thesame reproducible results when manipulating the same setof data. The structure of the experiment, for example when

Springer


using both target reactions in the same well (duplex) or whentarget reactions are conducted in separate wells (singleplex),can also alter the way the data is compared [13]. Finally,it is usual to apply a simple linear regression model basedon minimising the sum of squares between the best fittingstraight line and the deviates to the data set. However, thismay not always be the best model to apply to the data, andother models may be more relevant, for example weightedregression [13]. This manual analysis of results as a pro-cedural step has the potential to add a lot of MU to the endresult, as currently there are no standardised protocols for thereporting of the GM content of sample unknowns, and it hasbeen reported that one major source of the uncertainty maylie with the construction of the calibration curve [14]. Veri-fication of the data analysis and handling can help identifytranslation errors or incorrect calculations, but different lab-oratories may take very different approaches to the manualmanipulation of data even with the same data set, and with-out harmonised guidelines for this data analysis, differentresults and interpretations may occur. Suggested approachesto help standardise data analysis have already been publishedand include approaches to using different regression modelsfor the best fitting calibration curve, the use of individualor mean values in the calibration curve construction, backtransformation of Ct values into concentrations, handling ofduplex and singleplex data, comparison of regression curvesand handling of PCR outlying values [13]. The techniques ofsingleplex and duplex PCR are further examined in relationto their application in GM quantification [32].

Equation for measurand: cause and effect diagram

The expression of the final result in terms of percentage GMof the sample unknown is based upon the six procedural stepsalready examined (Fig. 7).

The final percentage GM content of a sample unknownis dependent upon the regression equation derived from themodel used to fit the calibration curve. Typically, simplelinear regression is used to fit the calibration curve, andthis estimates the line of best fit, which minimises the sumof squares of the deviations between the observed and theestimated values of the dependent variable, given that a linearrelationship exists. This resultant regression equation is oneof a straight line where y = mx + c. Here y refers to the deltaCt associated with the sample unknown; m is the gradient ofthe calibration line (or the PCR efficiency) which is the deltaCt divided by the logarithm of the percentage GM of sampleunknowns; x is the log of the percentage GM of the sampleunknown; and c is the intercept on the y-axis (expressed interms of delta Ct).

In relation to this equation, the prime variable that re-quires elucidation is y, which is the delta Ct. The delta Ct iscalculated based on the mean delta Ct = (mean (transgene

Ct)) − (mean (endogenous Ct)) for the replicates associatedwith a sample.

The Ct value associated with a sample is dependent upona number of things, inclusive of the software ‘Black box’associated with a particular platform. Aside from this ‘Blackbox’, the Ct value can depend upon which real-time quan-titative PCR platform is used; if there is a manual settingassociated with the threshold value and baseline and whatvalue these take; the quality and specificity of the primersand probes; the fluorophores used in the reaction; the qual-ity and amount of DNA; the cycling conditions used in thePCR; the number of replicates involved which will dictatethe confidence and variability associated with a value; thequality of the PCR constituents, and manual data handlingaspects. An overview of different methods for relative andabsolute quantification using Ct calculations can be found in[14, 25, 26, 29–31]

Additionally, the gradient of the line (m) and the intercepton the y-axis (c) are variables that are dependent upon thequality of the calibration curve, which itself will be influ-enced by the accuracy of calibrants, both in terms of pre-cision and trueness. Thus, the use of CRMs that have beenwell characterised and whose GM content are known withinset limits of accuracy and with an associated MU estimate,are suitable standards to be used as calibrants [33]. For ex-ample, the 2% CRM standard used in this paper had a meanvalue of 2.0% and an expanded MU estimates of ± 0.3%based on mass per mass measurements. Furthermore, a suit-able level of replication associated with each calibrant willhelp ensure that the response variables are a confident andtrue representation of how the calibrants behave. Currently,the percentage GM content of sample unknowns can be ex-pressed in terms of mass per mass, although current EUrecommendations advocate the expression in terms of hap-loid genome copies, which would be facilitated by a copynumber approach [22, 34]. At the present time, there are noCRMs available which have been certified on a copy num-ber basis, so the use of calibrants that have not been charac-terised fully can add more MU to the end result. Publicationsbased on using a copy number approach using plasmid andgenomic DNA calibrants have shown that plasmids may pro-vide a suitable alternative to GM quantification studies, asthey compare favourably based on bias and precision studies[16, 35]. The variability that can be incurred by using dif-ferent data analysis approaches to Ct values, and the effectof using different sources of calibrants have been describedfurther [36].

The identification of factors that can give rise to MU us-ing the procedural approach gave some estimates as to howimportant each of the factors was. Combining this informa-tion with the mathematical derivation of the GM content ofthe sample unknown can give rise to a theoretical model thatidentifies those procedural steps that are likely to have a large

Springer


DNA quality

% GM = 10((y – c)/m)

Platform operation Manual analysis of results

User interpretation

DNA quantification

DNA extraction

Software analysis

PCR setup

Where: y = Delta Ct of sample unknown c = intercept of calibration curve on y-axis m = gradient of calibration curve

Software

Singleplex / duplex

Manual operation

Primers / probes

“Black box” effect

Quality of DNAAmount of DNA PCR constituents

Baseline setting

Sampling effects

Sample homogeneity / stability

Manufacturer’s protocol

Threshold cycle setting

Replication factor

Lack of standardisedguidelines

Best fitting modelPlatform choice

Validated protocol

Accuracy of calibrants

Extraction efficiencytransgene:endogenous

ratio

Fig. 7 Model to show factorsaffecting estimation of GMcontent. The mathematicalmodel for the derivation of theGM content of sampleunknowns is shown, along withthose factors that are consideredto be important in contributingmeasurement uncertainty to thisas identified in the upstreamprocedural steps

effect upon the end result. Using the assay illustrated in thisstudy, such a model is shown in Fig. 7.

Conclusion

The procedural approach to measurement uncertainty eval-uation allowed the identification of factors that have the po-tential to strongly influence the estimation of the percentageGM concentration of sample unknowns. The sampling pro-cedural step is important in ensuring the test sample usedfor analysis is representative of the original batch sample,so homogeneity, analyte ratios and sample integrity/stabilityare key issues [14, 22]. It is postulated that a lot of MU maybe attributed to this upstream sampling stage [5]. The lab-oratory procedural step for the DNA extraction is likely tohave a relatively large influence on the results, as this stagewill contribute towards the quality of the resultant templateDNA. Upstream steps based on the DNA matrix, integrityand processing, also affect the DNA quality, but it is un-likely the laboratory analyst will be able to directly controlthese aspects. Both the sampling and DNA extraction pro-cedural steps may contribute towards the relative abundanceof the wild-type and transgene elements in the extractedDNA sample, and can thus have a large impact upon the MU[14]. In contrast, the procedural step for the quantificationof the DNA can be argued as contributing relatively littleMU as an internal normalisation of the response variable istaken into account in the downstream analysis. Provided thatquality criteria are followed as provided by manufacturer’sinstructions, validated protocols and working instructions,there is likely to be only a moderate amount of MU arisingfrom the two procedural steps of the PCR setup and the RT-QPCR platform operation. Dependent upon the RT-QPCRplatform and its associated software, the manual settings of

the baseline and threshold cycle can add a variable amountof uncertainty to the expression of the end result. Addition-ally, in the manual analysis procedural step, verification ofdata handling and manipulation techniques will help ensurerepeatability within a laboratory, but without standardisedapproaches and guidelines for the derivation of GM contentfrom the data, there will continue to be differences betweenlaboratories conducting exactly the same assay. A final as-pect that can account for significant MU in the assay is theinterpretation of the result by the client. It is important thatthe end-user of the result be familiar with MU terminology toensure that the result is interpreted with its correct meaning.

Dependent upon the scope of the GM quantification studybeing undertaken, additional sources of uncertainty may alsoneed to be considered. In relation to inter-laboratory stud-ies, the MU associated with the GM analytical proceduremay actually be larger than the estimate of the GM con-centration itself [16]. Potential sources of uncertainty at thisscale may include the stability of the DNA calibrants andsample unknowns, and the variability between the differ-ent laboratories. Inter-laboratory studies showed that therecan be a 20-fold difference in the sensitivity associated withusing the 35S promoter detection system between laborato-ries [14]. Whilst it is reported that the CV associated withdelta Ct values can be as low as 1.5–2%, once all uncertain-ties have been taken into account the CV associated withthe estimated GM value can invariably be between 20 and25% [14]. These examples illustrate the potential magnitudeof both the MU and the problem that scientists involved inGM quantification may encounter, and also the use of theprocedural approach in helping identifying any contributorysources in the analytical laboratory.

Furthermore, whilst RT-QPCR appears to be the currentmethod of choice for the quantification of GM ingredients,alternative methods are also available. For example, double-

Springer


competitive PCR has the advantages of relatively quick as-sessment of samples and associated low cost, whilst suffer-ing from problems inherent with staining and densitometricanalysis [14]. With respect to RT-QPCR methods, alternativedetection chemistries such as Molecular BeaconsTM (Strata-gene) and hybridisation probes are also available. For meth-ods that are distinct from RT-QPCR or use alternative detec-tion chemistries, an assessment of other potential sources ofMU must be made in full.

The method shown in this paper illustrates a procedu-ral approach for helping identify factors that can contributeMU to GMO quantification when using RT-QPCR. As such,the example used is by no means exhaustive and is context-specific in that a selection of certain procedural steps had tobe taken, for example the specific RT-QPCR platform andDNA extraction technique. The limitation of this is that afull-scale project would look at all of the permutations as-sociated with GM quantification, such as the different DNAextraction techniques, RT-QPCR platforms, etc. As there iscurrently a lack of standardisation of assays used for GMquantification in the bioanalytical community, such an ap-proach is beyond the remit of this small study, which utilisesone particular assay only for illustrative purposes. However,the procedural approach taken should be equally applicableto other GM quantification assays, and the pertinent resultsand conclusions bear relevance to all areas of GM identifi-cation and quantification.

This paper reports how the procedural approach can beapplied to a specific GM quantification method in order tohelp identify sources that can account for significant variabil-ity in the end result. The example used in this paper servesto illustrate how the procedural approach can be applied toother GM quantification method in order to help identifyother such sources of MU. Published techniques, such as themodular method validation approach [5] would also benefitfrom the implementation of this procedural approach. Ad-ditionally, this approach can be used as a pre-requisite tomaking the actual MU calculations themselves as the nextstep in a MU budget would be to estimate the magnitudeof the effect of those factors, through such approaches asin-house method validation, or inter-laboratory trials.

Acknowledgements The authors gratefully acknowledge funding pro-vided by the Food Standards Agency (UK) through project E01059:“The applicability of using plasmid DNA as opposed to extracted ge-nomic DNA for quantitative PCR calibration”, and advice from Dr. NeilHarris (LGC Limited).

References

1. ISO/IEC 17025:1999 (1999) General requirements for the compe-tence of calibration and testing laboratories. ISO, Geneva

2. International Standards Organisation (1993) International vocabu-lary of basic and general terms in metrology. Blackwell Science.ISBN 9267017051

3. Ellison s, Rosslein M, Williams A (eds) (2002) EU-RACHEM/CITAC Guide CG 4: quantifying uncertainty in ana-lytical measurement, 2nd edn. ISBN 0948926155

4. International Organization for Standardisation (1995) Guide to theexpression of uncertainty in measurement. Geneva, Switzerland

5. Holst-Jensen A, Berdal KG (2004) J AOAC Int 87(4):927–936

6. Burns M (2004) Trends Anal Chem 23(5):393–3987. Kuselman I (1998) 3(3):131–1338. Alexandrov Y (2001) Anal Bioanal Chem 370(6):690–

6939. Burns M (2005) Europ Pharmaceut Contract 122–128

10. Kristiansen J (2001) Clin Chem Lab Med 39(10):920–93111. Burns M, Valdivia H, Harris N (2004) Anal Bioanal Chem

378(6):1616–162312. Burns M, Nixon G, Foy C, Harris N (2005) BMC Biotechnol 5:31.

DOI: 10.1186/1472-6750-5-3113. Burns M, Corbisier P, Wiseman G, Valdivia H, McDonald P,

Bowler P, Ohara K, Schimmel H, Charels D, Damant A, Har-ris N (2006) Eur Food Res Technol. DOI: 10.1007/s00217-006-0376-z

14. Wiseman G (2002) J AOAC Int 85:792–79615. Wurtz A, Bluth A, Zeltz P, Pfeifer C, Willmund R (1999) Food

Control 10:385–38916. Vaı̈tilingom M, Pijnenburg H, Gendre F, Brignon P (1999) J Agric

Food Chem 49:5261–526617. Terry C, Harris N (2001) Eur Food Res Technol 213:425–

43118. Trapmann S, Catalani P, Conneely P, Corbisier P, Gancberg D,

Hannes E, Le Guern L, Kramer G, Prokisch J, Robouch P, Schim-mel H, Zeleny R, Pauwels J (2002) The certification of refer-ence materials of dry-mixed soya powder with different mass frac-tions of Roundup ReadyTM Soya – IRMM-410S. ISBN 92-894-3725-1

19. Terry C, Harris N, Parkes H (2002) J AOAC Int 85:768–774

20. Corbisier P, Trapmann S, Gancberg D, Hannes L, Van IwaardenP, Berben G, Schimmel H, Emons H (2005) Anal Bioanal Chem383:282–290

21. Moreano F, Busch U, Engel K (2005) J Agric Food Chem 28:9971–9979

22. Trapmann S, Emons H (2005) Anal Bioanal Chem 381(1):72–74

23. Smith D, Maxwell P (2005) Food Control 53:9848–985924. Peano C, Samson M, Palmieri L, Gulli M, Marmiroli N (2004) J

Agric Food Chem 52(23):6962–696825. Applied Biosystems 7900HT Fast Real-Time PCR System and

7300/7500 Real-Time PCR Systems (2005) Real-time PCR sys-tems. Part no 4348358, Rev E 5/2005

26. Hubner P, Waiblinger H-U, Pietsch K, Brodmann P (2001) J AOACInt 84(6):1855–1864

27. Donald C, Qureshi F, Burns M, Holden M, Blasic Jr J, Wool-ford A (2005) BMC Biotechnol 5:15. DOI: 10.1186/1472-6750-5-15

28. Applied Biosystems (2002) Document 117GU05-01. Part no4370923, Rev A

29. Applied Biosystems 7900HT Fast Real-Time PCR System (2005)Relative quantitation using comparative Ct—getting started guide.http://docs.appliedbiosystems.com/pebiodocs/04348358.pdf

30. Livak K, Scmittgen T (2001) Methods 25:402-40831. Mavropoulou A, Koraki T, Ioannou P, Christopoulos T (2005) Anal

Chem 77(15):4785–4791

Springer


32. Alary R, Gautier M-F, Joudrier P (2002) Oleagineux, Corps Gras,Lipides 9(6):468–473

33. Trapmann S, Catalani P, Conneely P, Corbisier P, Gancberg D,Hannes E, Le Guern L, Kramer GN, Prokisch J, Robouch P, Schim-mel H, Zeleny R, Pauwels J (2002) The certification of referencematerials of dry-mixed soya powder with different mass fractionsof Roundup ReadyTM Soya – IRMM-410S. Report EUR 20273EN, European Commission, Joint Research Centre, Institute forReference Materials and Measurements (IRMMGeel (BE))

34. European Commission Recommendation (2004/787/EC) On tech-nical guidance for sampling and detection of genetically modifiedorganisms and material produced from genetically modified or-ganisms as or in products in the context of regulation (EC) no.1830/2003. Off J Eur Union L 348/18

35. Taverniers I, Windels P, Van Bockstaele E, De Loose M (2001) EurFood Res Technol 213:417–424

36. Skern R, Frost P, Nilsen F (2005) BMC Mol Biol 6:10. DOI:10.1186/1471-2199-6-10

Springer

Documents

A procedural approach for the identification of sources of uncertainty associated with GM quantification and real-time quantitative PCR measurements