INTERNATIONAL JOURNAL OF LEPROSY^

Volume 59, Number IPrinted in the U.S.A.

Inter- and Intra-Laboratory Variation in theReporting of Skin Smears in Leprosy 12

Betsy Abraham and Annaiah Cariappa3

The need for monitoring the results ofskin-smear examinations obtained in lep-rosy control programs has been recognized(4 ' 5 ). Studies on the variation in the reportsof smears read by peripheral laboratories incontrol programs and re-read by base lab-oratories are few ( 1 H). Such informationwould define the limits of variation, andwould provide program managers necessaryinformation about the expected variation inresults during the course of monitoring skinsmears.

Smears collected in the field and read ata peripheral laboratory situated in the con-trol area of Schieffelin Leprosy Research andTraining Centre (SLR&TC), Karigiri, India,were re-read 43 km away at the base labo-ratory situated at SLR&TC. The inter- andintra-observer variations in the base labo-ratory were also studied in order to definevariation in the reporting of smears underideal conditions, and to provide referencevalues against which inter-laboratory vari-ation may be interpreted.

MATERIALS AND METHODSSkin smears were collected by two tech-

nicians under field conditions as part ofthe leprosy control activities of the Branchof Epidemiology and Leprosy Control,SLR&TC. These technicians had beentrained earlier in all aspects of skin smearsat the base laboratory. Their experience af-ter training ranged between 1 and 14 yearsapproximately.

Smears were taken from a minimum offour sites per patient: the right earlobe, theforehead, the chin, the gluteal region for

' Presented in part at the 15th Biennial Conferenceof the Indian Association of Leprologists, Visakha-patnam, Andhra Pradesh, India, November 1987.

= Received for publication on 7 May 1990; acceptedfor publication in revised form on 17 September 1990.

' B. Abraham, M.B.B.S., Ph.D., Research Fellow;A. Cariappa, M.B.B.S., M.D., Research Pathologist,Schieffelin Leprosy Research and Training Center,Karigiri 632106, India.

males and the front of the thigh for females.These four sites are called "routine sites."Smears from these four sites are taken fromall new patients, regardless of clinical clas-sification, as baseline information. In ad-dition, in suspected paucibacillary patients,smears are taken from active lesions; theseare called smears from selective sites. In thecontrol area, 80% to 90% of the patients areclinically classified as borderline tubercu-loid or tuberculoid leprosy.

Smears were fixed immediately, andstaining and grading were completed within48 hr by the same technicians at the pe-ripheral laboratory situated in the controlarea. The staining method used was themodified Ziehl-Neelsen method. Smearswere graded according to Ridley's logarith-mic scale (6). A minimum of 25 fields, whenpositive, and the entire smear when nega-tive, are examined in all cases.

Routinely, all smear slides collected dur-ing a month are submitted to the base lab-oratory at the end of the month. At the baselaboratory, 10% of the slides received eachmonth are randomly selected for cross-checking which is completed within a dayor two of receipt of the slides at the baselaboratory. For the purposes of this study,the results obtained during the period 1985-1989 (4 years) were analyzed. During thisperiod, a total of 7938 slides had been re-ceived and a total of 875 slides (approxi-mately 10%) had been re-read at the baselaboratory.

All slides were read blind at the base lab-oratory by two technicians who had be-tween 9 and 18 years of experience in thesmear technique. Results from the controlarea laboratory were entered alongside theresults from the base laboratory only afterthe latter laboratory had completed exam-ination. A report of 1+, when obtained byone technician at the base laboratory, wasalways counter-checked by the other.

Inter- and intra-observer variationsamong the two technicians at the base lab-

76

59, 1^Abraham and Cariappa: Variation in Reporting Skin Smears^77

TABLE 1. Inter-laboratory variations.

Results from^ Slides"

^ Smears"

Base lab Control area lab No. No.

Negative Negative 788 90% 3166 93%Positive Positive 48 5.5% 155 4.5%Negative Positive (I +) 30 3.5% 71 2%Positive (1+) Negative 9 1% 18 0.5%

" One slide denotes I patient." Four smears on 1 slide; sometimes 3 or less when only selective sites taken.

oratory were studied as follows: 54 individ-ual smears (11 slides) that had been col-lected routinely at the base hospital and werenot part of the slides received from the con-trol area, were given to both technicians,three times each on three different occa-sions. The smears were read under "ideal"conditions: Two smears from the batch of54 were given to each technician per day.These smears were always given at the be-ginning of the day, to exclude the possibilityof error due to fatigue. The original reportsranged from 0 to 6+ (0 = 13 smears; 1+ =7; 2+ = 6; 3+ = 17; 4+ = 7; 5+ = 2 and6+ = 2). The smears had been reported atthe base laboratory up to 2 weeks prior tothe experiment. The identification numberetched at one end of the slide was coveredwith a thick white card, and the slides werecoded by one of the authors before exami-nation. The code was broken at the end ofthe experiment and code numbers werematched with identification numbers. It wasensured that each smear was re-gradedwithin a month of its collection and stain-ing, in order to overcome the possible effectof fading of stains.

The value of kappa (K) ( 2), a measure ofagreement, was calculated for the results ofthe experiment on inter- and intra-observervariations.

RESULTSInter-laboratory variation. Table 1 shows

that 90% of the slides were reported nega-tive by both laboratories. Therefore, themajority of patients from whom smears werecollected were considered negative by bothlaboratories. Only 5.5% were reported pos-itive by both laboratories. A small percent-age (3.5%) was reported positive by the con-trol area laboratory and negative by the baselaboratory; in an even smaller percentage

(1%), the reverse was true; all except onereport showed a positivity of 1+. When in-dividual smears were compared, the per-centages were approximately similar (Table1).

When positive slides (those with one ormore smears positive) were checked foragreement, only 8.3% (Table 2) showedagreement. The most common finding wasa difference of 1+. A difference of 2+ wasrare. However, comparison of individualsmears in the same set of slides revealedthat there was 55% agreement. A differenceof 1+ was found in 43%. This would in-dicate that different laboratories may agreeon the grading of individual smears, but thatagreement on sets of four smears may berare.

Intra- and inter-observer variation in baselaboratory. Figures 1 and 2 display the in-tra-observer variations among the two tech-nicians routinely engaged in crosscheckingsmears. Each technician read the same smearthree times, and 54 individual smears wereread. When the results of all smears (posi-tive and negative) were considered, bothtechnicians could reproduce results with nodifference in grading most of the time: tech-nician 1 = 75% of the time, technician 2 =80% of the time (data not shown). The val-ues of kappa (K) calculated for technician 1were 0.84, 0.81, and 0.74 for the first vs

TABLE 2. Inter-laboratory variations inpositives.

Grading Slides^Smears

difference No.^No.

Nil 4 8.3% 85 55%1+ 40 83.3% 67 43%2+ 4 8.3% 3 2("i)

• 06• 6.

5•5•

4• 464. •tg:^•0

07,2•0

2• 2op

.°22'l•^- irr

do

• •

78^ International Journal of Leprosy^ 1991

^I AA. •

2•^3•^4•^5•

Bacterial Index2•^3•^4•

Bacterial Index5•^6•

FIG. I. Intra-observer variation of technician I: 54smears, each reported three times. Variation/agree-ment between reports I and 2 (x); reports 2 and 3 (0);reports 1 and 3 (•).

second, second vs third, and first vs thirdreports, respectively; for technician 2 thevalues were 0.91, 0.78, and 0.79, respec-tively. All values except one are greater than0.75. Values 0.75 are taken to representexcellent agreement beyond chance ( 2). Theintra-observer agreement for positive smearswas 80% for technician 1 and 83% fo sr tech-nician 2 (Table 3).

Inter-observer variation is seen in Figure3. Three sets of reports on 54 individual

6 •

5. O•

4. °— 2:$ •■•

.nt,t„re.Tio:^3. 00

tr;

2.

0.0 00

2•^3•^ 5•^6•

Bacterial Index

FIG. 2. Intra-observer variation of technician 2 (secFig. 1 legend).

FIG. 3. Inter-observer variation of technician 1 vs2: 54 smears, each reported three times by each tech-nician. Variation/agreement of report I (x); report 2(0); report 3 (•).

smears were plotted. Again, agreement wasfairly common: both technicians agreed 66%of the time (data not shown). The values ofK were 0.79, 0.73, and 0.76 for the first,second, and third sets of reports, respec-tively. The inter-observer agreement forpositive smears between technicians 1 and2 was 80% (Table 3).

Figures 1 to 3 also show that variationswere unrelated to the grade of the smears.

When positive slides instead of individ-ual positive smears were compared (Table3), technician 1 reproduced results 54% ofthe time and technician 2, 45.5% of the time.

TABLE 3. Infra- and inter-observer vari-ations in positives.a

Tech- Gradingdiffer- ^nician ence^No.

Inter-observer1 vs 2^Nil^4^36.4%

1+^7^63.6%

41/54 smears.h Mean of 3 reports.

Slides"^Smears"

^

No.^%

Intra-observer6^54.5%^33^80%5^45.5%^8^20%5^45.5%^34^83%6^54.5%^7^17%

^

33^80%8 20%

59, 1^Abraham and Cariappa: Variation in Reporting Skin Smears^79

Both technicians agreed 36.4% of the time;the remainder in each case varied by 1+.

DISCUSSIONThe aim of this paper is to define the

extent of variation present in the grading ofskin smears when the same smears are readby a laboratory in a control area and by abase laboratory. This variation was thencompared with the values obtained under"ideal" conditions from the study of inter-and intra-observer variation in the base lab-oratory.

There was hardly any variation betweenlaboratories in the reporting of negativeslides and smears. Vettom and Pritze de-scribe similar results ( 8). However, a fewslides reported positive (1+) by the baselaboratory were reported negative by thecontrol area laboratory (Table 1). It is im-portant to define this number, since smear-negative patients would receive a shorterduration of therapy compared to smear-positive patients. As this paper shows, only1% of the patients would have received in-appropriate therapy in the sample studied.

Inter-laboratory variation in the report-ing of positive slides was quite common.There was agreement in only 8.3% of pos-itive slides (Table 2). The extent of disagree-ment or variation was between 1+ and 2+;the most common variation was 1+. Theexperiment studying inter-observer varia-tion in the base laboratory shows that it ispossible to report positive slides with totalagreement between technicians in 36% ofthe slides (Table 3). The extent of the vari-ation in the remaining 64% was 1+. Whenindividual positive smears were compared,the agreement in the inter-laboratory studywas 55% (Table 2), and in the intra-labo-ratory study (inter- and intra-observer vari-ation) was 80% (Table 3). The inter-labo-ratory results suggest that completeagreement in the reporting of skin-smearslides is not a feasible proposition. This isreinforced by the finding that even under"ideal" conditions in the base laboratory,the intra-observer agreement in the report-ing of positive slides was only between 45%and 55% and the inter-observer agreementwas only 36% (Table 3). It appears, there-fore, that while almost total agreement maybe achieved when reporting negative slides,

it is prudent to expect a variation of 1+among positive slides.

Our results are different from those ob-tained by de Rijk, et al. ('). They observedvariations greater than 1 + in 21% of theirsample; some were 3+. Their results ofinter-observer variation also show a muchhigher variation than seen in our study. Eventhough comparison of both sets of results isnot strictly valid because of different sam-pling methods used, it is interesting to notethat variations greater than 1+ were notunusual in de Rijk, et al.'s study (').

In light of the results obtained by us andde Rijk, et al. it appears, as stated earlier,that a variation of 1 + in the reporting ofpositive slides is to be expected betweenlaboratories reporting skin smears in a fieldprogram.

Vettom and Pritze suggest that the rea-sons for this variation in the reporting ofskin smears may be a) the method of grad-ing, which counts only a limited number(25) of fields, and b) nonuniform distribu-tion of bacilli ( 8). A logical extension of thisargument would be to count all fields inpositive smears and to try to declump or-ganisms. However, such procedures wouldincrease the time, effort, and complexity ofthe technique to unacceptable levels, es-pecially under field conditions. The successof laboratory tests in any field program de-pends to a large extent on the degree of sim-plicity without compromising too much ofthe quality of reporting. While the scanningof all fields when the smear is positive maybe theoretically possible in regions wherepaucibacillary leprosy is common, it will beimpossible in areas where multibacillaryleprosy is endemic. Further, declumping ofAl. leprae is difficult even under the best oflaboratory conditions, and it is unrealisticto attempt it in the field. In any case, changesin the technique would require meticulousstandardization.

In retrospect, the redefinition of pauci-bacillary (PB) and multibacillary (MB) lep-rosy by the WHO ( 9) vis-a-vis the bacterialindex (BI) seems appropriate. Earlier, PBpatients were those with a BI of < 2+ atall sites ( 10). Our study shows that a varia-tion of 1+ is present in the reporting ofapproximately 92% of all positive slides.Hence, the recent WHO recommendation

80^ International Journal of Leprosy^ 1991

(9) that all smear-negative patients he clas-sified as PB and all smear-positive patientsas MB seems pragmatic. However, even withthis redefinition, the problem of misclassi-fication remains, although to a slightly lesserdegree. Two scenarios for misclassificationcan be described: a) A patient whose smearslide is reported as 1+ positive by the baselaboratory is reported as negative by thefield laboratory and is treated for PI3 lep-rosy. b) A patient whose smear slide is re-ported as negative by the base laboratory isreported as 1+ positive by the field labo-ratory and is treated with the MB regimen.Of these two examples, the former shouldhe of greater significance to program man-agers since the aim of a field program is therapid bacterial sterilization of the patientpopulation. In our study only 1% of patientswere reported 1+ positive by the base lab-oratory and negative by the field laboratory.We feel that this percentage may be ac-ceptable to managers of a field program. Onthe other hand, 3.5% of the patients werereported negative by the base laboratory and1+ positive by the field laboratory. In thisgroup the primary aim of bacterial steriliza-tion is met although it is a moot pointwhether time, money, materials and man-power have been wasted and patients over-treated. When the new definition of PB andMB leprosy ( 9) is applied to this study, thepercentage of misclassification is 4.5% (1%+ 3.5%). When the old definition (to) is ap-plied, 5% ofthe patients (44 out of 875 slides,Tables 1 and 2) would have been misclas-sified. In areas unlike ours where MB lep-rosy is common, the percentage of misclas-sification would be higher with the olddefinition. This is because, as shown in ourstudy, a variation of 1+ among positives isinherent in the technique.

Quality control (QC) of all aspects of theskin-smear technique is essential for thesuccessful implementation of leprosy con-trol programs. It must be remembered thatthe reporting of skin smears is the last eventin a long chain of events, which includes theselection of sites in patients and the variousaspects of collection, fixing, and staining.Therefore, any deviation in the techniquein any of these aspects would also be re-flected in the final report. (The reader isreferred to reviews that address these

issues''.) In the few published reports onQC of smears ( 1 • 8 ), different methods havebeen used to analyze the variation in re-porting. We feel that three simple guidelinesare all that are required by base laboratoriesto monitor the reporting of smears in pe-ripheral laboratories: a) positive slides mayvary only up to 1 +; b) negative slides mustagree; c) the percentage of slides reported as1+ positive in the base laboratory and asnegative in the control area laboratoryshould be low (approximately 1%).

SUMMARYThis paper defines the variations in the

reporting of skin smears between a base andfield laboratory in a leprosy control pro-gram. Ten percent of all slides read by thefield laboratory in a control area werere-read by the base laboratory. There wasalmost no variation in the reporting of neg-ative slides, but a variation of 1+ was pres-ent in approximately 92% of positive slides.Thus, there was agreement in approximate-ly 8% of positive slides. This paper also de-fines the variations in the reporting of pos-itive slides under "ideal" conditions bydescribing the results of a study on intra-and inter-observer variations among tech-nicians at the base laboratory. There wasbetween 45% and 55% agreement withinobservers and about 36% agreement be-tween observers. The results of both studiesare compared. Simple guidelines are de-rived to monitor the reporting of skin smearsin leprosy control programs.

RESUMENEste trabajo analiza las variaciones en los reportes

de los resultados de extendidos de linfa cutanea entreun laboratorio base y un laboratorio "de campo," enun programa de control de la lepra. El 10% de todaslas laminillas leidas por el laboratorio de campo en unarea controlada fueron releidas en el laboratorio base.Aunquc casi no hubieron variaciones en los reportesde las laminillas negativas, en aproximadamente el 92%de las laminillas positivas se encontrO una variaciOndc I +. Asi, la concordancia fue de aproximadamenteel 8% en las laminillas positivas. El trabajo tambiêndescribe las variaciones en los reportes de las laminillaspositivas bajo condiciones "ideales" al analizar los re-sultados de un estudio sobre las variaciones intra- einter-observadores, efectuado en el laboratorio base.La concordancia fue del 45% al 55% dentro de losobservadores y de aproximadamente el 36% entre los

59, 1^Abraham and Cariappa: Variation in Reporting Skin Smears^81

observadores. Los resultados de ambos estudios secompararon y de aqui se derivaron lineamientos sim-ples para valorar los reportes sobre los extendidos delinfa cutanea en los programas de control de la lepra.

RESUMECet article dêcrit les variations dans les resultats des

frottis cutanes rapportês par un laboratoire central etun laboratoire peripherique dans un programme delutte contre la lepre. Dix pourcents do toutes les lamespar le laboratoire peripherique d'une region furent re-lues au niveau du laboratoire central. II n'y avait pra-tiquement pas de difference dans les resultats rapportespour les lames negatives, mais une difffN-ence de 1+&tail presente pour environ 92% des lames positives.II y avait done accord pour 8% des lames positives.Cet article dêcrit egalement les variations dans la no-tification de lames positives dans des conditions"ideales" en detaillant les resultats d'une etude sur lavariabilite inter- et intra-observateurs parmi les tech-niciens du laboratoire central. 11 y avait un accord intra-observateurs variant de 45% a 55%, et un accord inter-observateurs d'environ 36%. Les resultats de ces deuxetudes sont compares. Des directives simples en sontdeduites pour contrOler les resultats des frottis cutanesrapportés au niveau des programmes de lutte contre lalepre.

Acknowledgments. The technicians at the base lab-oratory are Mr. K. Vilvanathan and Mrs. Shanthi Ye-supadam. The technicians at the laboratory in the con-trol area are Mr. S. John and Mr. Y. Inbanathan. Wethank Dr. K. Jesudasan, Head, Branch of Epidemiol-ogy and Leprosy Control, for permitting analysis of thedata and for helpful discussions. The encouragementgiven by Dr. C. J. G. Chacko, Head, Branch of Lab-oratories, is appreciated.

REFERENCES1. DE KIJK, A., NILSSON, T. and CHONDE, M. Quality

control of skin smear services in leprosy pro-

grammes: preliminary experience with inter-ob-server comparison in routine services. Lepr. Rev.56 (1985) 177-191.

2. FEINSTEIN, A. R. The measurement of intcrrateragreement. In: Clinical Epidemiology. The Archi-tecture of Clinical Research. New York: Igaku-Shoin/Saunders, 1985, pp. 212-236.

3. GEORGIEV, G. D. and McDouoALL, A. C. Thebacteriological examination of slit-skin smears inleprosy control programmes using multiple drugtherapy: a plea for radical changes in current op-erational methodology. Indian J. Lepr. 59 (1987)373-385.

4. GEORGIEV, G. D. and McDouoALL, A. C. Skinsmears and the bacterial index (131) in multipledrug therapy leprosy control programs: an unsat-isfactory and potentially hazardous state of affairs.Int. J. Lepr. 56 (1988) 101-104.

5. INDIAN DIRECTORATE GENERAL OF HEALTH SER-N./Ices, LEPROSY DIVISION. Report on second in-dependent evaluation of National Leprosy Erad-ication Programme. New Delhi: Ministry of Healthand Family Welfare, 1987, pp. 27-29.

6. RIDLEY, D. S. Bacterial indices. In: Leprosy inTheory and Practice. Cochrane, R. G., ed. Bristol:John Wright and Sons Ltd., 1959, pp. 371-372.

7. SEHGAL, V. N. and JOGINDER. Slit-skin smear inleprosy. Int. J. Dermatol. 29 (1990) 9-16.

8. VETTOM, L. and PRITZE, S. Reliability of skin smearresults: experience with quality control of skinsmears in different routine services in leprosy con-trol programmes. Lepr. Rev. 60 (1989) 187-196.

9. WHO EXPERT COMMITTEE. Sixth report. Geneva:World Health Organization, 1988, p. 15. Tech.Rep. Ser. 768.

10. WHO STUDY GROUP. Chemotherapy of leprosyfor control programmes. Geneva: World HealthOrganization, 1982, p. 240. Tech. Rep. Ser. 675.