THE RUNNING OF STANDARDS IN CLINICAL …jcp.bmj.com/content/jclinpath/5/4/305.full.pdfJ. clin. Path. (1952), 5, 305. THE RUNNING OF STANDARDS IN CLINICAL CHEMISTRY AND THE USE OF THE

J. clin. Path. (1952), 5, 305.

THE RUNNING OF STANDARDS IN CLINICALCHEMISTRY AND THE USE OF THE CONTROL CHART

BY

RICHARD J. HENRY AND MILTON SEGALOVEFrom tlhe Bio-Science Laboratories, Beverly Hills, California

(RECEIVED FOR PUBLICATION JANUARY 4, 1952)

The progress of clinical medicine has been con-siderably aided in the past few decades by theincreased number of laboratory tests availableand the improvement in their specificity andaccuracy. Nevertheless, the reliability of analy-tical results from clinical laboratories has beenquestioned by many individuals, and several inde-pendent surveys have more than justified thesuspicion (Belk and Sunderman, 1947; Shuey andCebel, 1949).

In these surveys blame has been placed on poorsupervision of personnel, poorly trained and in-sufficient personnel, poor equipment, poor choiceof methods available, and so on. The vastmajority of clinical chemical analyses are andprobably will continue to be performed by, andsupervised by, people not trained as chemists. Insuch cases laboratory staff follow methods inthe so-called "cook-book" fashion with onlysparse knowledge of the chemistry involved.Faced with this problem, it becomes the job ofthe clinical chemist to outline procedures indetail so as to make them as fool-proof as possible.

It is the purpose of this paper to discuss theimportance of running routine standards withclinical chemical analyses, the types of standardsthat can be employed, and the treatment of thedata accumulated so as to give reasonableassurance that the results being obtained arewithin a known permissible error. By such meansthe laboratory can strengthen the reliability of itsresults and the faith of the physician in them.

The Use of StandardsPhotometric analyses make up the bulk of quan-

titative clinical chemical analyses. The chemistconsiders the running of frequent standards withphotometric or spectrophotometric analyses rou-tine, but in the clinical laboratory this has seldombeen true since the advent of the photocolori-

x

meter. " Precalibrated " photometers are widelyadvertised to-day with the implication that the testsneed not be standardized by the buyer. Theacceptance of such precalibrations cannot be tooseverely condemned. By relying on precalibra-tions errors can occur as a result of making orbuying new reagents, the deterioration or contam-ination of old reagents, the use of incorrect lightfilters, changes in the characteristics of the photo-meter itself, etc. Standardizing each test when anew instrument is obtained is not enough, sincethe same errors can and do develop subsequent tothe standardization. Assuming the necessity ofrunning standards, there is the question of howfrequently they should be run. Ideally they shouldbe run with every set of determinations, or oncea day (Archibald, 1950). Some have advocatedrunning them twice a week (Levey and Jennings,1950). Certainly this is far better than not runningthem at all, but a definite risk is being taken in thelatter case, because several days may elapse beforeserious error is detected. The authors have seen arefrigerated biuret reagent deteriorate within48 hours to the extent that standards read 30%below what they should have read. Whether thisrisk is justified is a question which is answered bythe individual situation and from experiencegained with a particular test over a period of time.

Types of StandardsThe Pure Standard.-This is the usually re-

commended standard and consists of a solutionof the substance of known' purity with perhapsa preservative added - (e.g., benzoic acid in thecase of glucose standards). Any other type ofstandard must first be standardized against a purestandard. The advantages of using this type ofstandard are that the purity is usually known witha high degree of accuracy, and, with a few excep-tions, such as enzymes standards, such standards

on 23 May 2018 by guest. P

rotected by copyright.http://jcp.bm

j.com/

J Clin P

athol: first published as 10.1136/jcp.5.4.305 on 1 Novem

ber 1952. Dow

nloaded from

http://jcp.bmj.com/

RICHARD J. HENRY and MILTON SEGALOVE

are readily available. The disadvantages include(1) the danger of prejudicial handling, since theperson running the test knows beforehand whatthe results should be; (2) substances normallypresent in blood which may interfere with or aug-ment the colour are not present; and (3) purestandards usually are not run through the entireprocedure. With some exceptions, however, thereis no reason why pure standards cannot be runthrough the whole procedure, and, where possible,it is usually advisable. The danger of omittingcertain steps is rather obvious. For example, inthe determination of blood glucose the usual wayto run a standard is to treat the standard glucosesolution as if it were a blood protein-free filtrate.The reagents used in preparing the protein-freefiltrates of the unknowns are thus by-passed andare not controlled. Any contamination of thesereagents leading to alteration of results in the un-knowns would not be impressed on the results ofthe standard.

Pooled Blood or Serum.-This type of standardhas been suggested by Archibald (1950) and byLevey and Jennings (1950). The obvious advantageis that it can be introduced as a routine sample,thus eliminating the possibility of prejudicialhandling. Furthermore, it must be run throughthe entire procedure. The primary disadvantageis that it is not at present easily available to alllaboratories. Although it is quite probable thatmost of the chemical values are stable in bloodin the frozen state, relatively little actual data areavailable. Levey and Jennings found that speci-mens stored in the deep freeze were stable forurea, total protein, albumin, chloride, and CO,but not for globulin.

Internal Standard.-This procedure involves theaddition of a known " pure standard " to an aliquotof one of the unknown samples. Such a standardwould, of course, have to be run through the entireprocedure and is readily available. The internalstandard is usually utilized in analytical chemistrywhen there is the possibility of substances presentin the unknown which will interfere with thedevelopment of colour in the analysis. To checkfor such interference in every sample wouldrequire running an internal standard on everyunknown.

BlanksThe use of blanks on the materials used in the

tests is nearly as important as the standard. Whenpooled blood or serum is used as the standard,contamination of a reagent would result in anelevated value for the standard. A reagent blank,

therefore, would not be necessary to indicate thattrouble of some nature was present; it would, how-ever, indicate the source of the trouble, and if thecontamination were not gross the run might besalvaged. But, with the other types of standards,' pure standards " and " internal standards," re-agent blanks may be essential. For example, inthe analysis of phosphate by the method'of Fiskeand Subbarow (1925), the standard usuallyemployed is monopotassium phosphate in tri-chloracetic acid. The trichloracetic acid used formaking the protein-free filtrate of the serumsample, however, is not the same trichloraceticacid present in the standard solution. Contam-ination of this reagent or the water used in theunknown in conjunction with trichloracetic acidwould be missed unless a blank were run. In fact,single-distilled water on occasion can contain signi-ficant amounts of phosphate and ammonia.

It is suggested, further, that where the absorp-tion of a reagent blank is low, reagent blanks,standards, and unknowns all should be read versusdistilled water set at 100% transmission. In thisway they are always read against a blank of fixedoptical density.

Definitions of Accuracy, Precision, and ReliabilityBefore discussing the problems of replication

of standards and unknowns and the ways of hand-ling data, it is best to define the terms accuracy,precision, and reliability as commonly used to-dayin the statistical approach.Accuracy of a result consists of comparing the

observed result with the actual true value. If aknown pure standard is used, the accuracy of aprocedure frequently can be estimated by recoveryexperiments with internal standards. In manycases no true value can be determined and a strictdetermination of accuracy is not possible.

Precision is the degree of variation in resultsobtained by a method when the same sample isrun repeatedly: in other words, the reproducibilityof what is observed. The less variation observed,the greater the precision. Obviously, precisioncan be determined with a high degree of accuracyfor every test procedure.

The reliability of a test is the ability to maintainits accuracy and precision into the future. If atest has maintained a steady state of accuracy andprecision over a long period of time, then one canpredict these characteristics for the future withassurance and the test is said to be reliable.Reliability of the test (which includes the test,analysts, and equipment) can be established onlyby checks over a period of time.

306



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/

RUNNING OF STANDARDS IN CLINICAL CHEMISTRY

Control of Accuracy and Precision

The accuracy of a test is a characteristicoccasionally beyond the control of the analyst.Various methods for determining the same sub-stance may give different results due to differencesin specificity. Blood glucose determinations are agood example of this, since with some methods,for example that of Folin-Wu, reducing substancesother than glucose normally present in blood aredetermined as glucose. Such a test has pooraccuracy but may have good precision. This doesnot necessarily negate the value of the test,especially if the " interfering " substances do notvary markedly, since the clinician becomes accus-

tomed to the definite set of normal values given bythe test.

The control of precision requires more seriousconsideration. For example, if a report on a bloodglucose concentration is N40 mg.%, it is importantthat the clinicians have reasonable assuredness thatthe true value by the method used is not 110 or170 mg.%. "Reasonable assuredness" is com-monly defined in statistical analyses as " 95%confidence," i.e. the value can be reported with arange limit on each side, within which range thetrue value, by the method used, will be found 95times out of 100. More rigorous "confidencelimits" can, of course, be used. If it is found forthe glucose determination used in our example thatthe confidence limits are + 10% of the observedvalue, then the clinician can be given the " reason-

able assurance" in the form 140 mg.% ± 10%,or 140 mg.% ± 14 mg. %. Ideally each reportshould be accompanied by some indication of itsprecision, although routine reporting of clinicalchemical determinations with confidence limits isprobably impracticable. It is important, how-ever, that laboratories and clinicians becomeaware of them and think in terms of them.The precision, i.e. confidence limits, of an analy-

tical procedure can be determined by severalmethods. The most obvious is to run numerousreplicates, say 30, of a sample and analyse mathe-matically the dispersion of results obtained. Suchan analysis will be considered in detail in the sub-sequent section on control charts. Although thissolution of the problem is valid, it is not verypracticable in the routine operation of most clinicallaboratories. A much simpler approach merelyrequires running routine unknowns in duplicate atleast until about 30 duplicate analyses are accu-mulated. The duplication must be complete, i.e.,through the entire procedure. The differencebetween each pair, the range R, is calculated and

the arithmetic average or mean of the ranges, R,is calculated. The magnitude of the confidencelimit on each side of a single observation is thenestimated by multiplying R by 2.65.*Table I shows the calculation of the confidence

limit in the determination of glucose, using only10 pairs.

TABLE I

Blood Glucose R(mg.%)100}

861 894f120} 9

97)192J126}125_I

7}9 4104} 0

120} 12

84} 4

9319S 2

Total = 48

-48R=- =4-8

10Confidence limit= 2-65 (R)

=±127 mg.',

The confidence limit as calculated in Table I isin terms of mg. % which, when converted to percent error by referring it to the approximateaverage of the range of analyses used (100 mg.%)becomes + 12.7 %. In neither of the above methodsof estimating precision is the error contributed byvariation in reagent blanks included in the estimate.This will be dealt with in detail in a future pub-lication, where it will be shown that usually thereis only a small increase in the limits as calculatedabove.The first decision which must be made before

setting up any clinical chemical procedure is thedegree of precision required, i.e., how wide darethe confidence limits be. Blood glucose valueswill probably be satisfactory for clinical use,

* This is an application of the use of range for estimation ofstandarddeviation. A confidence limit of three standard deviations is obtainedfor the mean of pairs by multiplying R by 1.88. Multiplying this by1.4 gives the limit for a single observation, i.e. 2.65 (R). Justificationfor the use of three standard deviation confidence limits is discussedin the section on control charts.

307



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/


though perhaps not for research, if they are within10% of the correct value for the method used.Many clinical chemical determinations are satis-factory for clinical use if they have a precision inthis range. There are some determinations wherean error of this magnitude obviously might beserious. The normal range for serum sodium isapproximately 300 to 345 mg. per 100 ml. Thus avalue in the middle of this range with confidencelimits of +7% would blanket the entire range.Even for clinical use a result with a maximumerror of approximately 3 % is thus required.Similarly, a value in the middle of the normalrange of serum calcium, if taken as 9 to 11 mg.%with limits of + 10%, would include the wholerange.The next problem to consider is what can be

done to decrease the magnitude of the confidencelimits (or error) of a determination once it is foundthat the existing limits are too large. In design-ing the routine procedures of a laboratory onemust stay within the bounds of practicability andstill be assured of the accuracy and precisiondesired. Valuable time in the laboratory is wastedby complicating procedures to give greater pre-cision than is actually required. If a procedure,however, lacks the required precision, one obviousline of attack is to use more precise and carefultechnique in its performance or improve themethod itself, e.g., better control of variables, suchas temperature, which may affect the result.Another possible approach to decreasing the erroris replication, i.e., running the unknown sample induplicate, triplicate, etc. Statistics must be calledupon to answer just how much of an increase inprecision is to be expected (Simon, 1941). If E isthe observed confidence limit or error for a singledetermination and E- the maximum confidencelimit or error desired, then the number of repli-

cates, N, necessary to achieve this is (;)2.

To demonstrate the magnitude of this inCreasein precision with replication let us presume anerror of + 10% with a single sample. The errorfor various numbers of replicates can be foundby substituting in the transposed version of theformula E3=E/ /R. For one, two, three, four,and five replicates the error would be 10, 7, 5.8, 5,and 4.5% respectively. Thus the error decreasesinversely as the square root of the number ofreplications. If the replication necessary is greaterthan duplicates, or perhaps rarely triplicates, thismethod of increasing precision becomes totallyimpracticable in the clinical laboratory. If neitherof these approaches yields the required precision

or if they are impracticable, the method must bediscarded and a search made for a more precisemethod.What has been said for unknown samples also

holds to a certain extent for standards. Reagentblanks present the same problem, but in a muchattenuated form. In the vast majority of cases thelight absorption by the reagent blanks is very smallcompared with that absorbed by the standard andunknown, so that even a rather large relative errorin the reagent blank would not often materiallyalter the correction factor of the blank. It is anentirely different matter if the reagent blankabsorption is relatively high. As previously stated,the frequent running of reagent blanks is usuallyfor the purpose of checking the reagents for con-tamination,

Control ChartsAfter a test has been set up and standardized

and periodic standards run to check on the pro-cedure, the simplest way to keep track of theseresults is merely to note them in a notebookreserved for the purpose. As expected, the resultswill vary from day to day, and if one value issuddenly greatly out of line suspicion will bearoused. There is a simple way, however, to treatthese check data as they accumulate which has theadvantage of telling the analyst at a glance howmuch the checks can vary before suspicion iswarranted and action indicated. The use of thequality control chart will frequently predicttrouble before it actually happens (Mitchell, 1947;Wernimont, 1946).There are numerous ways in which these charts

can be set up, but only a few of the simplest willbe discussed, because they will serve the purposeadmirably in the clinical laboratory.Type I.-For this type single standards are run.

A sheet of graph paper is employed (it is con-' enient to have a loose-leaf notebook of graphpaper for quality control charts of the variousdeterminations) with an appropriate scale on theordinates for the observed result of the standard.This scale may be in terms of the final result, e.g.mg. %, but it is usually more convenient to use thedensity reading of the standard (corrected for re-agent blank if read versus water or solvent blank).The abscissae represent successive determinationsand the date is noted for each point placed on thegraph. The next step cannot be taken until 20such points have been plotted, so it is advisable torun standards frequently, perhaps once a day, at

308



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/


230

220

* L* I.

0 1~~~0021001 * * 0

2001~~ ~ ~~~ ~~~* *1*~~~~ S

I S~ &S

21 I

LOp 5& 0

V 1 2345678910111213141516171819 20212223242526272812 34 5 6 7 8 9 12 13

February MarchFiG. 1.-Type I control chart. Single standards.

least until this number has been reached. Thearithmetic average or mean, x, is then calculatedfrom these 20 values and represents the best esti-mate of the true value for the standard. Ahorizontal line is then drawn on the graph at thispoint as in Fig. 1.

The next step is to establish confidence limitsfor the standard values. These are the same con-fidence limits mentioned before and constitute arange of equal magnitude on both sides of themean, x, in which approximately 95 times out of100 the standard result should fall if no factorsare operating other than those operating duringthis period in which the 20 values were obtainied.These limits are calculated as follows:

If N=number of determinationsix=arithmetic mean of N determinations£= " sum of "

a= standard deviationx=a single determination

(x-x) =the deviation of a determination from the mean, x.

x= (x)

N

a= | (j-X)2N-1

Often the mathematical labour is simplified bythe use of another form of this equation:

a= /(2) - -NN-1

As an example the calculations for the confidencelimits of Fig. I are presented herewith:

TABLE I I

(x) A~~x-x) (x-x)

211 I I215 5 25207 3 9206 4 16220 10 100210 0 0202 8 64209 1 1213 3 9215 5 25214 4 16210 0 0200 10 100212 2 4210 0 0216 6 36202 8 64213 3 9209 1 1206 4 16

Z(x) =4200N =20

ny)

A(x--x)2=496'kx AJVVA

x =N -20 = 210

(x-X)2 496I = 5-1

3= 15-3

The confidence limits are drawn above andbelow the line drawn for the x equal to a distanceof 3a. If all goes well for many more determi-nations, say a total of 100, the x and s can bere-evaluated. It will be noted that if the variationof determinations about their mean representedthe so-called "normal " or Gaussian distribution(symmetrical bell-shaped distribution), the 95%confidence limits would be represented by + 2a.It has been emphasized, however, that distri-butions of this type are frequently not of the"normals" type (Mitchell, 1947; Clancey, 1947).Clancey (1947) analysed the variation of manychemical analyses and found the Gaussian dis-tribution to hold in a minority of cases.

L-

o00

-cIv

309

- - - - - - - - - - r- - - - - - - - - - -

I

47Mn



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/


There are always two dangers when using con-trol charts: (1) looking for trouble that does notexist, and (2) not looking for trouble that doesexist. Thus, the use of 3a as compared to 2a mayreduce the number of times that we look fortrouble when it does exist, and may increase thenumber of times that trouble exists and we do notlook for it. Taking these points into consideration,and especially when there is no a priori knowledgeof the nature of the distribution, it is believed thatthe best compromise, and certainly the safest,is to associate "reasonable assuredness" withconfidence limits of + 3a (Simon, 1941; Mitchell,1947), especially when 3a limits are within thedesired precision and we do not want to waste timelooking for trouble that does not exist. If, how-ever, 3u limits are too broad compared with theprecision required, 2a limits may be preferred ifwe are willing to spend the time to investigate outof limit points knowing that frequently in suchcases trouble does not actually exist.What should be done if a value falls outside the

confidence limits? First of all it must be remem-bered that these are only " reasonable assured-ness" confidence limits, and therefore approxi-mately one value in every 20 to 100 should falloutside, but not far outside. If a value falls slightlyoutside the limits, the procedure should be checkedcarefully for an assignable cause-new reagentused, new standard, possible break in technique,etc. If none of these are obvious and the possibleerror involved, if it is an error, is not critical, youmay wait to see what happens next time. If thesame thing happens next time, then there is causefor alarm, since t4ere is a good indication that thereis trouble. Many times trouble can be predictedbefore it actually happens. In Fig. 1 it is notedthat after February 26 there is a gradual drift in thedistribution in a downward direction, although notuntil March 13 is there a value falling outside thelower confidence limit. Obviously, trouble beganbrewing about February 26, but not until March13 can the test be considered out of control. Thistype of drift occurs when one reagent graduallydeteriorates, e.g., aminonaphtholsulphonic acid re-agent in the determination of phosphate.Type II.-In a Type I control chart there is a

certain amount of mathematical labour involved inthe calculation of a. This can be obviated ifsuccessive determinations are treated as pairs(Wernimont, 1946). The mean of each pair andthe differences between each pair (the range, or R)are plotted as in Fig. 2. The 95% confidencelimits for the x are i 1.88 (R), where the value ofR is obtained by averaging the range values, R. The

confidence limit for the range is 3.27 (R). (Onlyone limit need be considered, since the other limitis obviously 0.)

230

220

X=210

200

190

30

20

*

_ - c

*0* ~~~~~IIa

*

_~~~~~~~~~~~~~~~~~~~~

* 0

0

10 00 0

0 0* * 0

oI0 0

0 24681012141618202224262824681012February March

FIG. 2.-Type II control chart. Successive single standards paired.

Type III.-More information can be obtainedby using duplicate standards. The difference be-tween the pair of standards or the range, R, iscalculated and plotted each time as in Type II andthe confidence limits calculated as in Type II. Themain advantage of this procedure is that it ispossible to tell whether the variation between daysis significantly greater than the variation in any onerun. With many chemical procedures there is adefinite tendency for this to be so, signifying thepresence of insufficiently controlled variables, suchas temperature. Such a situation would be in-dicated when the range values are apparently incontrol but the averages of the duplicates out ofcontrol. This type of control chart has been sug-gested for use in the clinical laboratory by Leveyand Jennings (1950), who give some examples ofits use and how it detects errors which develop inprocedures. Whether the added labour in runningduplicate standards is worth while depends on theindividual situation.

&I

310

*

c



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/


No matter which type of control chthe value for the standard used in the i

of the unknowns is the mean x, not ththe standard obtained on the same daythe same run with the unknowns.

If unknown samples are run singlyshou!d be the same as that determinedards if a Type I control chart is usethe unknowns and standards are treateSimilarly, if unknowns are run in dierror of their average should be tU

320

"D 310'aV3

300

5

C)'a

.0

VS

* .0 0

.

* 0

.

O0a 0* 0

0 00.

to00

30 o o X .

to-

_0 _ _

2010

. .*

0 ~~~~0 0

* * *

.0* *0

0

0* 0

0 a 0

art is used, that determined for standards run in duplicatecalculations (Type III). This assumes that the erroi is theie value for same for unknowns as for standards and isor even in approximately correct if the per cent transmission

for both is in the range of 20 to 60% (opticaly, the error density 0.2 to 0.7). When pure standards are used.d for stan- and not run through the entire procedure the con-d, provided fidence limits set for the standards obviously>d the same. cannot be used as an estimate of the precision ofuplicate the unknowns.ie same as In a procedure such as the determination of

serum phosphatase the substrate blank, the reagentblank, and the range for standard, if a Type II orIII control chart is being used, can all be plottedtogether as in Fig. 3.There is no practical way, and in many*cases

it is impossible, to be sure that the result of any* * one determination is correct even with the running

* * 0 of standards and blanks. The procedures out-lined here, however, will tell in many cases theaccuracy to be expected by a procedure, theprecision in all cases, and the reliability asindicated by quality control charts.There are other factors of importance in setting

e up determinations which are outside the scope of* . this paper and have received adequate treatment

. elsewhere (Archibald, 1950; Ayres, 1949). Theseinclude the proper choice of concentration range(yielding final optical densities of 0.2 to 0.7), andthe establishment of whether or not the colour pro-duced follows Beer's law.

SummaryIt is considered absolutely essential to run

frequent standards in all colorimetric clinicallaboratory analyses. The various types of stan-

* dards that can be run are discussed, as well as the.*. * estimation and control of accuracy and precision,

and the role of duplication in the control ofprecision. The en)poyment of control charts isadvocated, since their use allows the laboratory tobe aware at all times of the state of control of thetest.

The authors wish to express their appreciation for* the advice given by Frank B. Cramer on some of thestatistical points.

REFERENCESArchibald, R. M. (1950). Analyt. Chem., 22, 639.

* t Ayres, G. H. (1949). Ibid., 21, 652.Belk, W. P., and Sunderman, F. W. (1947). Amer. J. clin. Path., 17,

853.Clancey, V. J. (1947). Nature, Lond., 159, 339.Fiske, C. H., and Subbarow, Y. (1925). -J. biol. Chem., 66, 375.

15 20 Levey, S., and Jennings, E. R. (1950). Amer. J. clin. Path., 20, 1059.Mitchell, J. A. (1947). Analvt. Chem., 19, 961.Shuey, H. E., and Cebel, J. (1949). Bull. U.S. Army med. Dep., 9,799.Simon, L. E. (1941). An Engineers' Manual of Statistical Methods.

substrate blanks New York.Wernimont, G. (1946). Indusir. Engng Chem. Anal. Ed., 18, 587.

2V

cocoC)

C)

C)

cC~

FIG.

0 1 5 10February

3.-Type III control chart with reagent andrecorded.

311

4

3

I

2

I



j.com/

J Clin P


ber 1952. Dow

nloaded from

http://jcp.bmj.com/

Documents

THE RUNNING OF STANDARDS IN CLINICAL …jcp.bmj.com/content/jclinpath/5/4/305.full.pdfJ. clin. Path. (1952), 5, 305. THE RUNNING OF STANDARDS IN CLINICAL CHEMISTRY AND THE USE OF THE