Int. J. Epidemiol. 1996 VALADEZ 381 7

8/13/2019 Int. J. Epidemiol. 1996 VALADEZ 381 7

1/7

International JournalolEpidemiologyO International Epidflmtotoglcal Association1996Vol. 25, No.2PrintedInGreat Britain

Using Lot Quality AssuranceSamplingtoAssess Measurementsfor Growth Monitoringin aDeveloping Country's PrimaryHealth Care SystemJOSEPH VALADEZ.t LORI DIPRETE BR OW N,* WILLIAM VARGAS VAR GAS* AND DAVID MORLEYValadez J J(Department of Internationa) Health, Johns Hopkins University, School of Hygiene and Public Health,Baltimore,MD,USA), DiPrete BrownL, VergesW Vand Morley D.Using lot quality assurance samplingtoassessmeasurements for growth monitoring in a developing country's primary health care system. International JournalofEpidemiology1996; 25: 381-387.Background Local supervisors used lot quality assurance sampling (LQAS) during routine household visits toassess thetechnical qualityofCosta Rican community-based health workers (CHW): measuring and recording weightsofchildren,interpreting their growth trend, and providing nutntion educationtomothers.Method Supervisors sampled 10households in eachof 12 Health Areas 4-8hoursperarea).Nomore thantwoperformance errors were allowedforeach CH W. This LQAS decision rule resultedinjudgements withasensitivityandspecificityofabout 95 .Results Three categoriesofresultsarereported: 1) CHW adequately weighed children, calculated ages, identifiedchildren requiring nutritional services,andusedthegrowth chart. 2)They neededtoimprove re ferral, education,anddocumentation skills. 3) Thelack of system support to regularly provide growth cards, supplementary feedingtoidentified m alnounshed children, and other es sential materials may have discouraged some CHW resulting In themnotapplying their sk ills.Conclusions Supervisors regularly using LQAS should,by the sixth roundof supervision, identify at least 90 ofinadequately performing CHW . This paper dem onstratesthestrengthofLQAS, namely,to beused easilyby lowlevellocal health workerstoIdentify poorly functioning componentsofgrowth monitonng and promotion.Keywords growth m onitoring, primary health care, quality assurance, LQAS, nu trition, child survival, international health,community health, Latin Am erica

Although growth monitoring and promotion (GMP)have been very effective in small projects, GMP hasbeen less useful when applied generally. Some publichealth professionals suggest improvements to GMP canbe made by more scientific scrutiny, greater communityinvolvement, and more attention to underlying eco-nomic and epidemiological issues related to mal-nutrition. * Other practitioners question how frequentGMP should occur.2 Regardless of the outcomes of

* Department of International Health, Johns Hopkins UniversitySchoolof Hygiene and Public Health, Baltimore, MD, USA.* QA Project, University Research Corporation, Bethesda, MD,USA.* DepartementodeControl deCalidad, M inisteriodeSaluddeCostaRica, San Jose, Costa Rica. CentreforInternational Child Health, InstituteofChild Health,Uni-versityof London,UK.Reprint requeststo: DrJoseph J VaJadez, JHPIEGO, 1615ThamesStreet, Suite 200, Baltimore, MD 21231,USA.

these debates, it is fundamentally important to maintainhigh quality growth monitoring since inaccuratemeasurement and recording can exacerbate health prob-lems for the child and result in irrelevant socialpolicies.3** This paper addresses this issue by showinghow Costa Rica applied an industrial quality assurancetechnique (Lot Quality Assurance Sampling: LQAS) toidentify rapidly weaknesses in its own GMP system.

Growth monitoring and promotion is defined as:assessing the child s growth curve, and taking corrective

TaylorC E Mercer M A. Analysis of Causal Factors InfluencingChildhood MalnutritionasPartofCommunity Based Nutritional Care:UNICEF, unpublished material, 1992:35 and Morley D. ColloquiumonGrowth Monitoring for Child D evelopmen t: UNICEF, unpublishedmaterial, 1992:11.* JohnsonU. AConceptual A nalysisofGrowth Monitonng and Pro-motion: UNICEF, unpublished material, 1992:7.

381


2/7

382 INTERNATIONAL JOURNAL OF EPIDEMIOLOGYactions should it deviate from the standard. Ideally, themother should take such corrective actions. However,no matter who takes the remedial action, correctdiagnosis requires accurate measurement of growth ateach point that forms the curve. Because LQAS is rapidand easy to use, it is one which local supervisors canuse regular ly to manage community-based healthworkers (CHW) performing GMP.4 Previous applica-tions of LQAS assessed other primary health care(PHC) services: vaccination,5 oral rehydration therapy,*health education,** record keeping,5 and reproductivehealth.+ One study using LQAS for CHW supervisiondemonstrated that when used at three 6-month intervals,of 36 problematic activities originally detected in thevaccination system, 30 improved by a factor of at least80 , and three improved by 25 .6 These experiencesshowed that LQAS can be used by minimally trainedCHW in different cultures for assessing complex aswell as simple tasks in the health system.

The following sections describe an application ofLQAS to assess the quality of CHW GMP, and presentfindings relevant to health system management andtraining.

M E T H O D SLQAS PrinciplesLQAS uses the binomial formula to calculate smallsamples and to formulate decision criteria for clas-sifying CHW by their performance using a three-parttr iage system: adequate, inadequate, and very in-adequate service delivery. In earlier assessments ofPHC coverage the criteria used to assign a CHW toa triage system stratum were: (1) judge coverage asadequate if 80 or more of the target populatio nreceives the service, (2) judge it as inadequate ifbetween 50 and 80 are covered, and (3) judge it asvery inadequate if 50 or less of the target populationis covered.5

LQAS calculates a probability estimating whether aCHW reached a predetermined performance standard (e.g.80 or more of his target population received the PHC);it does so by analysing the number of children in a small

Valadez J J, Vargas W V, Sells M. Using LQAS to Assess OralRehydraiion Therapy in Costa Rica. Cambridge, MA: Harvard Institutefor International Development, unpublished material, 1989:11. Voladez J J, Weld L H, Vargas W V. The Quality of VaccinationEducation of Mothers by Community Health Workers in Costa Rica.Cambridge, MA: Harvard Institute for International Development,unpublished materials, 1990.* Valadez J J. Assessin g Reproductive Health S ervice Provider Skillsand Training Using LQAS. Baltimore, MD: JHPIEGO, unpublishedmaterials, 1995:12.

sample who have not received a particular service. Forexample, in a sample of 19 children if six or fewer havenot received the service, the CHW is classified as pro-viding adequate coverage. If more than six have notreceived the service the CHW 's performance is judgedas very inadequate.The exact coverage of a given CHW isless important than deciding whether coverage was ade-quate. In most instances of day to day health systemmanagement, a supervisor needs to detect extremes ofperformance to make rational decisions about resourcesallocation.

Because LQAS uses binomials, it can precisely iden-tify quality at either end of the continuum: adequate orvery inadequate. It is less sensitive to CHW within themiddle category. However, this limitation is not severesince the closer the quality is to either end of the triagesystem, the greater the likelihood that CHW will beclassified as adequate or very inadequate. The two ex-treme categories of CHW are the most important toidentify correctly so that: (1) resources can be directedto improve low quality services that otherwise couldheighten health risks in communities; and (2) resourcesare not needlessly spent on adequately performingC H W .

To use LQAS three initial decisions must be made:1. Define performance standards for the service deliv-ery unit under assessment using a three-part triagesystem.2. Decide the permissible classification error of theLQAS screening, for example, a sensitivity of 92 anda specificity of 93 .3. Develop a decision rule that stipulates the m aximumnumber of individuals who have not received the inter-vention allowed in the LQA sample. Any numbergreater than this threshold results in judgin g a C HW 'sperformance as very inadequate. In the above example,if six or fewer children do not receive a service, theCHW is judged as having performed adequately. Ifseven or more do not receive it, the CHW is judge d asvery inadequate.

The LQA sample size depends on all three decisions.Performance standards, the classification error, and thenumber of permissible performance errors are all inter-related. A complete discussion of LQAS theory and acomprehensive set of Tables for sample sizes rangingfrom five to 50, can be found elsewhere. 5 '7 '8Sample Size and Sampling MethodsSample sizes were selected using LQAS principles tomake two assessments: (1) the quality of GM P techniqueof individual CHW, and (2) the proportion of CHW


3/7

USING LQAST OASSESSC HWGROWTH MONITORING TECHNIQUE 38 3TABLE 1 LQAS classification probabilities of a CHW byobserving GMP for 10 children with two performance errorspermittedExpected level ofskill quality

0.950.900850800.750.700.650.600.550.500 45

ProbabilityofclassifyingCHWas adequate

0.990.930.820.680.530.380.260.170.100.0550.03

Probabilityofclassifying CHWas very inadequate

0.010.070.180.340.470.620.740 830.900.9450.97

with adequate GMP skills working throughout thenational programme.Sampling within a Health Area. A CHW is respons-ible for growth monitoring of all children under sixin approximately 500 households that form a HealthArea (HA).

Because this application of LQAS assesses thequality of GMP technique, the Ministry of Health(MOH) used higher performance standards than thosepresented in earlier examples about coverage of a po pu-lation with a service. Almost all of the time a CHWshould deliver services using proper technique. TheMOH defined the following triage system: adequateCHW sk ills if 3=95 of the time the CHW u sed thecorrect technique , 95 > inadequate > 50 , very in-adequate 5 0 . The desired sensitivity and specificitywas 95 . The resulting decision rule was: observe aCHW monitoring the growth of 10 children, if per-formance errors are observed in a component of GMPin more than two children then classify the CHW ashaving inadequate growth monitoring technique. Thissimple decision rule has a corresponding sensitivity of94.5 and a specificity of 99 .

This decisio n rule misclassifies about 1 of CH Wwith performance quality of 95 , and about 5 ofCHW with skill quality of 50 (Table 1). Perform ancebetwee n 95 and 50 is more likely to be classified asadequate as it appro aches 95 , and classified as veryinadequate as it approaches 50 . Thus, LQAS identi-fies the best and worst services with small error, butservices in the middle range of quality have higherclassification errors.

Sampling procedures within Health Areas. A trainedsupervisor using a checklist observed a CHW meas-uring the growth of the 10 children in each child'shome. Each supervisor randomly selected the first of15 households in a CHW 's HA (five extras in case somemothers or children were not at home); the remaining14 were homes located near to the first household. TheCHW then visited the corresponding households with asupervisor and monitored the growth of one child in thehousehold. The first household was chosen randomly toeliminate CHW preferences for easily accessible andcompliant mothers. The remaining 14 was a convenientsample due to the assumption that CHW performance isindependent of the location of the child's home. Anyresulting Hawthorne effect was assumed to benefit theassessment since if CHW over performed for the evalu-ators and still committed errors they surely would con-tinue to do so when not being observed.Selecting Health Areas throughout the country. Thesecond focus of the evaluation was to decide whether asufficient proportion of CHW adequately carried outGMP throughout the country. Therefore, a sample sizeof CHW and a decision rule had to be developed.Selecting the sample of CHW required the samedecisions to be made as when selecting the number ofhouseholds to observe. One constraint was that all sixCosta Rican regions had to be represented in thesamp le. A 95 : 50 triage system was chosen sincethe MOH decided that at least 95 of CHW shouldperform adequately for any given task. The MOHdecided upon a national sample of 12 CHW with a12:2 decision rule; these choices resulted in a sens-itivity and specificity of 98 . The sample was stratifiedby region and population density (rural/urban). TheMOH team randomly selected one rural and one urbanCHW in each of the six health regions.Summary of sample design. A two-stage LQA samplewas used. The first stage selected 12 CHW to assessoverall national performance. The second stage selectedhouseholds in an HA to assess a CHW monitoring thegrowth of 10 children.Evaluation InstrumentsThe evaluation team developed and pretested twoinstruments. The first one assessed the availability ofessential supplies (an accurate scale and a supply ofgrowth charts) at the HA, and registration of mal-nourished children. The supervisor provided growthcharts when necessary.

The second one was an observation checklist usedin the household. It distinguished separate tasks that


4/7

38 4 INTERNATIONAL JOURNAL OF EPIDEMIOLOGYcomprise GMP and aggregated them in five subsystemsthat must function correctly to learn the nutritionalstatus of a child based on weight-for-age. They are:weight calculation, age calculation, use of the growthchart, education of the mother, and documentation ofGMP data .

As the study assessed CHW GMP skills at the house-hold level, it did not consider issues such as motivationor time constraints that may affect a CHW's perform-ance. However, earlier studies in Costa Rica have con-sidered this issue.9

Data CollectionNurses and rural health supervisors (all of whom werefer to as supervisors) carried out data collection,accompanied by an observer from the MOH evaluationteam. Due to previous experience with a s imilarsupervision instrument the supervisors used the instru-ment easily with little instruction.3

Each su pervisor ob served each CHW with 10 chil-dren. During 5 weeks supervisors observed 12 CHWdeliver services in 120 households distributed acrossthe na tion, spen ding 48 hours with each CH W. T hetime varia tion w as mostly du e to the different travellingconditions within each HA.

RES U LTSQuality of Essential Supplies and RegistrationThe CHW require essential supplies to carry out regulargrowth monitoring. These include a functioning suspen-sion scale and an adequate supply of growth charts.Half the areas visited did not have a working scale ora supply of growth charts. However, most CHW hadat least one growth chart to use as a guide to classifychi ldren .

To follow-up and refer malnourished children, HA per-sonnel should register identified children. At a nationallevel the system for registering malnourished children wasinadequate; eight of 12 CHW did not maintain the system.Weighing the ChildThe MOH identified eight tasks (Table 2) included inthe process of weighing a child. Each can affect the ac-curacy of the weight measurement. At a national level,CHW performed only three components adequately:(I) remove shoes and heavy clothing, (2) read the scalefrom directly in front of the scale's face, and (3) accur-ately read the scale. Although the other componentswere not carried out adequately, results confirm theydid not affect the accuracy of the weight reading.Besides observing the weighing process, we also

TABLE 2 The GMP activities CHW should perform whileweighing the child

T a s k s

1. Position scale in a safe, well-lit place2. Set scale to zero3 . Remove shoes and heavy clothing4. Wait until child is still to read weight5 Read scale from directly in front of

scale face6. Read weight out loud7. Record weight immediately8. Weigh child twice9. Accurately read scale

InadequateCHW( n = 1 2 )

550324770

Nationalperformanceis adequate

(Ye sJNo)

NoNoYesNoYe sNoNoNoYe s

compared the supervisor's weight reading (used as thestandard) with the CHW's weight reading. As allmeasurem ents w ere within 0.1 kg, national perform-ance was judged adequate. When the CHW did notcalibrate the scale, the supervisor did so before measure-ments were taken.There are several possible explanations why in-adequate weighing procedures did not result in in-accurate weights. Performance standards may be toostrict; greater tolerance may be allowed without affect-

ing the weight measurement. Alternatively, the super-visors may have had similar readings to CHW becausethey were making similar mistakes. However, as super-visors were skilled in growth monitoring and becauseone of the evaluation team always observed them, itseems unlikely that this occurred.Age CalculationSupervisors judged each CHW's ability to calculateaccurately children's ages using date of birth. Theycompared a CHW's age calculations with his/her owncalculations to verify accuracy. Ages were judged asaccurate if the CHW age calculation was within onemonth of the supervisor calculation. All componentsof this subsystem were adequate in all 12 HA.Use of the Growth ChartSupervisors expected CHW to use a sex-specific NationalCenter for Health Statistics weight-for-age growth chart.As supervisors judged all 12 CHW as adequate, nationalperformance was also judged as adequate.

The CHW must also correctly plot current weightand age (one of 12 CHW plotted inad equately) and thenconnect the point to previous measures to chart the


5/7

USING LQAST OASSESSC HWGROWTH MONTTORJNG TECHNIQUE 38 5child's progress (five CHW were inadequate). Nationalperformance was adequate for the former activity andinadequate for the latter. In many HA the growth charthad no previous growth points to connect.

All CHW used the available data to classify childrencorrectly. This task was measured by comparing thesupe rvisor's classification w ith the CH W 's classifica-tion. This result, however, has to consider that somejudgements had no previously recorded growth pointsto use as a reference.Education of the MotherThe nutrition education that accompanies growthmonitoring is an important and difficult task to assess.The CHW are expected to inform all mothers of thenutritional status of their children and to use the growthchart as a visual aid during the explanation. Nation-al CHW performance for each of these componentswas not adequate (three of 12 CHW and eight of 12CHW were inadequate, respectively). Ideally, the fam-ily as a unit should receive education being sure toinclude older family members and family decisionmakers.

Beyond informing the mother of the child's nutri-tional status, CHW should refer malnourished or over-weight children to the health centre and supplementaryfeeding programmes where appropriate; they shouldalso educate mothers about: breastfeeding, feeding ofpreschoolers, prevention of diarrhoea, and hygiene.LQAS results show that all CHW need improvement.Of 38 malnou rished (n = 30) or overweight (n = 8)children, CHW referred children to the health centre, orverified that the child was followed-u p in only 10 cas es.Of the 30 malnourished children, CHW referred andgave information about supplementary feeding to nineof them.

Although some CHW may have been unaware oftheir duty to refer children, other considerations mayexplain referral problems. The CHW may have alreadydetected growth faltering on previous visits and dis-cussed this condition with mothers during those visits.A milk shortage during the time of the study affectedthe supplementary feeding programme, and CHW mayhave been hesitant to discuss milk supplements whichwere not available. The CHW expressed their dissatis-faction with the government nutrition services. Super-visors agreed that none of these issues excused lack ofreferral by CHW.

Discussions of diarrhoea prevention, oral rehydrationtherapy and breastfeeding were not satisfactorily per-formed. However, supervisors had instructed CHW toperform only GM P and nutritional assessments for thestudy. Therefore it is possible that during a regular

TABLE3 Documentation of growth monitoring informationTasks Inadequate NationalCHW performance(n = 12) is adequa te(Yes/No)1. Name of child2. Birth date3. Health record number4. Today's weight5. Today's age

13933

YesNoNoNoNo

household visit these issues would be discussed.Nevertheless, it is clear from these results that CHWdo not see diarrhoea prevention or breastfeeding as anintegral part of GMP.

The CHW are also expected to discuss the nutritionalneeds of preschoolers. They were mentioned 60 of thetime.

Education concerning personal hygiene and foodpreparation hygiene was inadequate. However, super-visors explained this deficiency as possibly due toCHW being reluctant to criticize hygiene in thepresence of a member of the evaluaton team.

Besides failure to inform and refer, many CHW didnot provide appropriate nutrition education when theyencountered a malnourished or overweight child. Incases where CHW conveyed health education, super-visors did note deficiencies in education skills. TheCHW did not routinely check to see that the mother hadretained nutrition information, and did not reinforce theeducational messages by repetition or example.DocumentationDocumentation of growth monitoring requires, at aminimum: name of the child, birth date, health recordnumber, weight and age measurements recorded on thegrowth chart. Although CHW did record the name ofthe child, national performance was inadequate forother documentation tasks (Table 3). Since age andweights were accurately measured, the biggest problemin this subsystem is that CHW failed to record theinformation.

Subsequent investigation revealed that this problemrelates to deficiencies in material supply. For the study,when CHW did not have growth charts; supervisorsgave them one for each child being monitored as theyhad brought their own supply. Because CHW did notusually have a supply of growth charts, they did not fillthem out regularly. Therefore, they were unaccustomedto do so.


6/7

38 6 INTERNATIONAL JOURNAL OF EPIDEMIOLOGY

^ 0

1I1f///

R\\ \

1\ 1

oo\ \ \\ \ \V

\\

A, \ J \\

PertIs JL

\

irmcndged

:e


7/7

USING LQAS TO ASSESS CHW GROWTH MONITORING TECHNIQUE 38 7These results demonstrate that regular detection ofGMP problems by community-based personnel isviable with LQAS methods. Its strength is the ability toidentify a poorly functioning end product. However,amelioration of problems may require corrective actioneither at the level of the CHW or at each higher level ofthe national programme.

ACKNOWLEDGEMENTSThe work upon which this paper is based was per-formed in part under a subagreement with the Centerfor Human Services under its Cooperative AgreementNo. DPE-5920-00-A-5056-00 with the US Agency forInternational Development. The original data collectionand data analysis occurred while Dr Valadez was afaculty member of the Office for Health, Harvard Insti-tute for International Development, Harvard University;final analyses were performed after Dr Valadez joinedthe faculty of the Department of International Health,The Johns Hopkins School of Hygiene and PublicHealth. Special recognition is given to Dr CarlosValerin (Director General of Health), without whosesupport this research would not have been possible. Wealso gratefully acknowledge the helpful suggestionsof Professor Andrew Tomkins of the Institute of ChildHealth who reveiwed a draft manuscript.

REFERENCES1Nabarro D, Chinnock P. Growth m onitoring: inappropriatepromotion of an appropriate technology.Soc Sci Med1988;26 :941-48.2Henry F, Briend A, Fauveau V. Child survival: should the strategybe redesigned? Experience from Bangladesh.H ealth PolicyPlan1990; 5: 226-34.3 Lotif M. Growth monitoring: a brief literature review of currentknowledge.Food and Nutr Roma)1988; 10: 3-10 .4Mohs E. Changing Health Paradigms in Costa Rica. In: Rohde J,Chatterjee M, Morley D (eds). Reaching Health for All.Delhi: Oxford University Press, 1993, pp. 378-95.'Valadez J J .Assessing Child Survival Programs in DevelopingCountries: Testing Lot Quality Assurance Sampling.Cambridge, MA: Harvard University Press, 1991, p. 247.(Harvard Series on Population and International Health).'Valadez J J, Weld L, Vargas W V. Monitoring Community HealthWorkers' Performance through Lot Quality AssuranceSampling.Am J Public Health1995; 85: 1165-66.7Lemeshow S, Stroh G Jr. Quality Assurance Sampling forEvaluating Health Parameters in Developing Countries.Surv Methodol1989; IS:71-81.8Dodge H F, Romig H G. Sampling Inspection Tables: Singleand Double Sampling. New Jersey: John Wiley, 1959,p.224.'Valadez J J, Vargas W V, DiPrete L. Supervision of primaryhealth care in Costa Rica: time well spent? Health PolicyPlan1990; 5: 118-25.

Revised version received September 1995)

Documents

Int. J. Epidemiol. 1996 VALADEZ 381 7