Implementing a Quality Assurance Program Using a Risk Assessment Tool on Dairy Operations

OUR INDUSTRY TODAY

1997 J Dairy Sci 80:777–787 777

Received December 5, 1995.Accepted July 25, 1996.1Department of Veterinary Science.2Department of Human Resources.

Implementing a Quality Assurance Program Using aRisk Assessment Tool on Dairy Operations

W. M. SISCHO,1 N. E. KIERNAN,2 C. M. BURNS,1

and L. I. BYLER1

The Pennsylvania State University, University Park 16802

ABSTRACT

Concerns and perceptions about antibiotic residuesin milk prompted the dairy industry to develop avoluntary program to support rational antibiotic useon dairy farms. One deficiency of this program is theinability of producers to identify easily the weak-nesses in antibiotic management in order to developcontrol plans. To overcome this deficiency, an educa-tional approach was designed. The program centeredon an on-farm risk assessment tool used by theproducer and an industry educator to determine thecurrent risk for residue violation. The risk assessmenttool was tested by 25 field personnel working withnortheastern milk receivers and 250 producers inseven states. The participants in the study identifieda lack of adequate treatment records as being thehighest risk factor for antibiotic residues, followed bydeficiencies in understanding how to use antibioticsand poor relationships between veterinarians andtheir clients. When field representatives utilized therisk assessment tool, for most producers, risk of an-tibiotic residue decreased by approximately 19%. Inparticular, more farms kept written records or morecomplete records. Finally, producers with reportedhistories of antibiotic residues were less likely to im-plement management changes to reduce the risk ofantibiotic residue.( Key words: quality assurance, antibiotic residues,food safety)

Abbreviation key: HACCP = hazard analysis criti-cal control points, MDBQAP = Milk and Dairy BeefQuality Assurance Program.

INTRODUCTION

Consumer and governmental concern over thepotential for the introduction of chemical con-taminants on the farm into the food supply is an

important issue affecting policy for the livestock in-dustry. Farm organizations have responded to thisconcern by developing innovative on-farm programsfor management and quality assurance. In particular,the dairy industry developed the Milk and Dairy BeefQuality Assurance Program ( MDBQAP) to promoterational protocols directed at managing and reducingthe use of antibiotics on the farm (6) .

The MDBQAP was originally designed as a volun-tary program for producers, but, in July 1992, therevised Pasteurized Milk Ordinance ( 1 ) required thatany producer found with a violative antibiotic inshipped milk would be required to participate in theMDBQAP. This ordinance changed the nature of theMDBQAP; the program was no longer voluntary. Thechange might explain in part the poor nationwideparticipation in the program. According to registra-tion records as of October 1995, including duplicateand required participation, only 6276 of approxi-mately 145,000 dairy producers in the US had com-pleted the program.

The MDBQAP was conceived as a Hazard AnalysisCritical Control Points ( HACCP) program (2) .There are several general components to any HACCPprogram. An HACCP program must 1) articulate thehazard, 2) identify the critical control points for thehazard, 3) quantify the limits of the hazard (i.e., theallowed product variability), 4) identify the risk forthe hazard on the premises, 5) implement proceduresto control risks that are specific to the premises, 6)assess the success of the procedures to control risk(i.e., implement a monitoring program), and 7) es-tablish a system to verify and document the im-plementation of the HACCP program (4) .

Evaluation of early implementations of theMDBQAP by representatives of milk receivers andveterinarians identified deficiencies in the MDBQAPprogram as an HACCP program. Although theMDBQAP clearly articulated the hazard (antibioticresidues) and provided critical control points to pre-vent the hazard [the 10 points in the booklet (3)], theMDBQAP failed to motivate producers to examinetheir own practices carefully. The MDBQAP did notprovide adequate tools for the producers to developplans to manage and monitor changes that were

Journal of Dairy Science Vol. 80, No. 4, 1997

SISCHO ET AL.778

specific to their own farms. Because the MDBQAPappeared to be a regulatory program, producers werenot convinced that practices used to avoid residueswere integral to successful dairy production and didnot constitute government interference in their enter-prise.

Overall, the MDBQAP did not readily allowproducers to characterize their own risk for antibioticresidue, did not result in a plan to overcome risk, and,ultimately, did not allow producers to evaluate theirown progress in lowering the risk of violation. Toovercome these deficiencies, researchers at The Penn-sylvania State University designed an alternativeeducational program to aid in the implementation ofthe MDBQAP. The program was built on the strengthof the MDBQAP, which articulated the hazards ofantibiotic residues and provided a general set of con-trol points. However, the new program extended theMDBQAP to focus producer awareness of on-farmpractices to avoid antibiotic residues. The core of thiseducational program was a tool for risk assessment tobe used by the producer and an industry educator onthe farm to determine the current risk of residueviolation. This study evaluated the success of theeducational program in influencing management asreflected by changes in on-farm practices to avoidantibiotic residues.

MATERIALS AND METHODS

Development of the Risk Assessment Tool

The risk assessment tool was designed to comple-ment the MDBQAP workbook, Milk and Dairy BeefResidue Prevention Protocol (3) . The risk assessmenttool was to be used to quantify and rank the risksassociated with antibiotic residue in the bulk tankaccording to the HACCP outlined in the MDBQAPworkbook. The tool would then identify on-farmstrengths and weaknesses for residue prevention,facilitate risk communication, and provide a means toassess on-farm progress in risk abatement.

The risk assessment tool was based on threesources of information that helped to quantify andrank on-farm management practices. The first wasthe MDBQAP workbook, which served as the basicstructure to rank risk and clearly outlined HACCP toprevent on-farm antibiotic residues. The secondsource was a refereed journal article by McEwen et al.(10). Those researchers (10) used observationalstudy methodology to contrast the management styleand knowledge of antibiotics among producers whohad violative levels of antibiotics in their milk againstthose who did not. The study of McEwen et al. (10)

was the most important source for quantifying riskbecause violators were contrasted against a controlgroup, and the strength of association between riskand management practices could be determined. Thefinal source of information to rank risk came fromstudies (7, 12) that used known cases of residueviolations to identify causes for the violations. Thosestudies (7, 12) were of less value because they lackedcontrol farms for comparison, and, therefore nostrength of association could be determined for therisk factors.

Of the 10 HACCP outlined in the MDBQAPproducer booklet, 8 could be evaluated on the farm byan objective observer. Consequently, the risk assess-ment tool was designed to address those 8 points. TheHACCP were: 1) herd health, which was evaluatedaccording to mastitis control and the reproductive anddry cow management on the farm; 2) the relation-ships between veterinarians and clients; 3) identifica-tion of treated cows; 4) treatment records and com-munication on the farm; 5) drug storage; 6) labelingof antibiotics; 7) on-farm screening for antibiotics;and 8) understanding the appropriate use of antibiot-ics. An additional section was added to assess theperception of the producer’s own risk as well as therisk of neighboring producers for antibiotic residuesin bulk tank milk.

Based on the information sources described previ-ously, each of the 8 management areas or HACCPwere ranked from first to last in importance for avoid-ing residues. Importance was defined as a deficiencythat could directly result in an antibiotic residue. Themanagement areas with the highest potential riskwere treatment records, relationships betweenveterinarians and clients, understanding the properuse of antibiotics, and the use of antibiotic screeningtests. The areas with the least risk were programsinvolving reproduction, dry cow management, andmastitis control. The assessment tool was designed sothat deficiencies in the high risk areas would beweighted to dominate the on-farm risk assessment.The areas from highest to lowest risk and the range ofrisk weights are presented in Table 1.

Program Implementation

The risk assessment tool was designed to be ad-ministered by any educator (dairy industry fieldrepresentatives, veterinarians, or county extensionagents) who was interested in implementing theMDBQAP. In this study, field representatives im-plemented the program. The representatives werechosen because their primary focus on the farm wasmilk safety and quality, and those representatives

OUR INDUSTRY TODAY 779


TABLE 1. The ranking and range of potential risk points for on-farm management or critical controlpoints associated with the risk of an accidental occurrence of antibiotics in bulk tank milk.

1Hazard Analysis Critical Control Point.

Range ofRanking HACCP1 risk points

1 Treatment records and communication to and between employees 0–352 Relationship between veterinarian and client 0–232 Understanding the use of antibiotics 0–202 Proper labeling of antibiotics 0–195 Use of on-farm screening tests 0–185 Identification of treated cows 0–207 Proper storage of antibiotics 0–177 Existence of reproductive and dry cow programs 0–179 Existence of an active mastitis control program 0–15

were the only industry contacts that all dairyproducers would have consistently had. The risk as-sessment tool contained questions directed to theproducer and observations made by the educator; anexample of one section of the tool is shown in Figure1.

Field supervisors from four participating milkreceivers were designated as field representatives inthe project. In turn, each field representative identi-fied a sample of 10 producers who would cooperate inthe study. The goal was to enroll 250 producers fromat least three states to participate in the program.Although the sample of both field representatives andproducers was likely biased, the randomization ofproducers to treatment and control groups allowedfair evaluation of the treatment effects within thestudy population.

Research Design

The project was designed as a clinical trial withtreatment and control farms. On treatment farms,field representatives and the producer jointly usedthe risk assessment tool to identify specific risks andthen to develop a plan to avoid risks that werespecific to that farm. The field representative andproducer surveyed the farm using the risk assessmenttool, discussed the results of the survey and the risksassociated with certain practices (emphasizing highrisk areas identified by the assessment), and formu-lated a risk abatement plan that was specific to thatherd. The important aspects of the treatment im-plementation were the joint agreement by theproducer and the field representative on the results ofthe survey and the development of a plan that ad-dressed one or two of the identified deficiencies. Thedevelopment of a herd plan was facilitated using asummary form filled out jointly by the field represen-tative and the producer (Figure 2).

On control farms, the field representatives used therisk assessment tool to identify on-farm risk but didnot explicitly review the assessment results with theproducer and only provided the MDBQAP booklet ( 3 )for the producer to review at his or her discretion. Thetreatment and control implementations were com-pared for the resulting changes in farm practices thatmight avoid antibiotic residue in bulk tank milk.

Three to 5 mo after the initial implementation, allfarms were revisited by the field representatives, andthe risk assessment tool was used to reevaluate themanagement practices on the farm. Because theproject was over, the results of this final evaluationwere shared with each producer, regardless of group,at this visit. All completed risk assessment tools werereturned to the project coordinators for data entryand summary.

Field Representative Training

All participating field representatives were in-volved in 2-d training sessions on the purpose of theproject, the need for consistency and neutrality in theimplementation, and the nature of risk. The trainingsessions were a combination of lectures on projectprotocol, discussion groups, trial use and peer critiqueof the risk assessment tool, on-farm visits to use thetool, and clarification and standardization of the useof the tool.

Program Evaluation andData Analysis

The program was evaluated in five stages: 1) inputof field representatives on the risk assessment tooland the training program, which was obtained afterthe training session using a focus group format; 2)farm implementation, which was evaluated and sum-marized in writing by the field representative; 3)


SISCHO ET AL.780

Figure 1. Example of the format used for the risk assessment tool. Answers within squares receive Risk Points.

quantitative program effectiveness, which was evalu-ated by comparing changes in on-farm risk betweenthe first and second visits; 4) focus group discussionsof producers for subjective evaluation of the value ofthe program, which was conducted following the finalfarm visit; and 5) focus group discussions of the fieldrepresentatives to reflect on their role in the program,which was also conducted following the final farmvisit.

Design and implementation of focus groups werebased on the methods described by Merton et al. (11)and Kruger (9) . The focus group meetings were heldin two different locations, and all field representativeswere invited. Nonparticipants were invited to com-plete a mail questionnaire that was designed andimplemented using the methods described by Dillman

(5) . This report presents only the results from evalu-ations 3 and 5, which are those related to programeffectiveness and the use of the risk assessment tool.

For quantitative analysis, the outcome of interestwas the change in practices on the farm to avoidantibiotic residues as surveyed between the first andsecond visits. This change was measured as both theabsolute and relative difference in risk points be-tween the two visits. A statistical model described therelative change in risk as a function of treatmentgroup, field representative, and milk receiver and wasdeveloped using analysis of covariance (8) .

For the analysis of qualitative data, the outcomesof interest were effect on treatment and control farms,the use of the risk assessment tool to discuss dairyquality, and the perceived reaction of the producer to



Figure 2. The form used to develop a focused, on-farm risk reduction plan for dairy operators in the treatment groups.

the risk assessment tool. The focus group discussionswere taped, and the tapes were transcribed. The tran-scripts were abstracted, and the comments were or-ganized to represent the array of positions about eachof the four evaluation topics.

RESULTS

A total of 25 field representatives from four milkreceivers covering seven states (Pennsylvania, NewJersey, Delaware, Maryland, West Virginia, Ohio,and New York) participated in the project. Thesefield representatives each recruited 10 producers for atotal of 250 producers. The training sessions for thefield representatives occurred during the first 2 wk ofJanuary 1993, and the first farm visits commenced bythe end of January.

Final herd visits were completed by mid-April; atotal of 23 field representatives completed both farmvisits and returned a complete set of risk assess-ments, which resulted in complete information from219 operations. Three focus groups for producers werescheduled between April 6 and 16, and two focusgroups for field representatives were scheduled in lateJune. Representatives who were unable to attend the

June meetings were mailed a questionnaire in July.All data used in the analysis was received by October1993.

Demographics of Participating Producers

Demographics of the participating producers arepresented in Table 2. The vast majority of producerswere herd owners (89%) and were male (96%). Ap-proximately 75% of the producers were <50 yr of age,74% were high school graduates, and 22% had eitherattended or graduated from college. Mean milkproduction was 26.1 kg/d per cow; median productionwas 27.3 kg/d per cow. Production ranged from 37.3 to13.2 kg/d per cow. Sixty-two percent of the participat-ing producers reported that they previously eitherhad had an antibiotic residue violation or haddumped bulk tank milk because of antibiotics.

On-Farm Practices to PreventAntibiotic Residues

First visit. The risk assessment tool was designedto weight management deficiencies differentially asgreater or lesser risks that might result in accidental


SISCHO ET AL.782

TABLE 2. Descriptive information from 250 dairy producers inseven states involved in the risk assessment project.1

1Not all producers answered each question.

Producer

(no.) ( % )Currently participating in DHIA testing 169 68Role on farmHerdsperson 15 6Herd manager 13 5Owner 220 88Unknown 2 1

GenderMale 239 96Female 10 4Unknown 1 0

Age, yr<20 1 020–34 90 3635–49 106 4250–64 51 2065–84 2 1

Educational backgroundAttended grade school 23 9Attended high school 26 10High school graduate 128 51Technical school 16 6Attended college 19 8College graduate 36 14Unknown 2 1

Antibiotic residue in bulk milk 154 62Mean milk production, kg/d per cow 26.1Standard deviation of milk production 4.3

antibiotic residues in bulk milk (Table 1). Thisprocess was expected to force the highly weightedmanagement areas to be considered as the high riskareas; however, data for the actual observed risks forthe 250 operations at the first visit were not com-pletely parallel to those of the weighting scheme (Ta-ble 3). On the first visit, poor treatment records andcommunication was by far the most commonly ob-served and greatest risk factor of on-farm manage-ment. Of the 250 farms (treatment and control) in-volved in the initial evaluation, 133 (53.2%; 95% CI;46.8 to 59.5%) had deficiencies in this area. Of therisks observed, 50% of the farms had no writtenrecords, had more than two people milking cows, orhad part-time milkers (Table 4). On 12% of thefarms, not all milkers could recognize a treated cow.Of the farms that had written records, only 40% ofthose records included the name of the drug used,only 37% included the time that milk was withheld,and only 17% included the name of an appropriatemilk screening test.

Of the remaining risk areas evaluated on the firstvisit, the most common deficiencies were understand-

ing antibiotic use, relationships between veterinari-ans and clients, use of cowside screening tests, andidentification of treated cows. Overall, mean valuesfor initial total risk were similar for treatment (54.2)and control (57.7) groups, and both groups had simi-lar risk profiles (Table 3).

Changes between first and second visits. Thesame risk areas that appeared to be important duringthe first visits were also important during the secondvisits, but fewer farms (44.7%; 95% CI, 38.0 to51.6%) reported treatment records and communica-tion as the most important risk. There was also somechange in the order of identified risks on the farms(Table 5). Twenty fewer treatment farms and 15fewer control farms reported that treatment recordsand communication was the greatest risk factor. Thebiggest change occurred in the number of farms thatkept written records. During the first visit, 50% of thefarms had no written record of treatments. By thesecond visit, <34% of the farms had no writtenrecords. The improvement was slightly better fortreatment farms than for control farms. The quality ofthe records also improved between the first and se-cond visits. More records included the name of theantibiotic used and its labeled withholding time (Ta-ble 4).

The mean risk during the second visit was 43.4points for treatment farms compared with 46.2 pointsfor the control farms. On average, the treatment andcontrol farms decreased the risk between the twovisits by approximately 11 points, which representeda 19% decrease in risk for both groups. The treatmentfarms decreased fairly evenly (9 to 11%) in fourareas: 1) treatment records and communication, 2)understanding antibiotic use, 3) antibiotic residuetests, and 4) identification of treated cows. The con-trol farms decreased the risk of understanding anti-biotic use by 19% and also decreased the risk of poorveterinary and client relationships (–13%) and iden-tification of treated cows (–12%) (Table 6).

Influences on changing risk. Least squaresanalysis of covariance of the relative change in riskfor the occurrence of an antibiotic residue is shown inTable 7. Seven variables were assessed in the model:1) the risk assessment treatment, 2) age of theproducer, 3) previous history of antibiotic residues inthe bulk tank, 4) educational background of theproducer, 5) milk receiver field representative work-ing with the producer, 6) importance of antibioticresidues, and 7) producer assessment of his or herlikelihood of violation because of antibiotic residues.Interactions between field representatives and groupwere also assessed in the model.



TABLE 3. Results of a risk assessment survey from 250 farms located in seven states taken during the initial visit showing the five mostcommonly reported risk areas on all participating farms.

195% exact binomial confidence interval for the percentage.2Treatment farms.3Total percentage is >100% because some farms identified more than one risk area as the greatest.

Medianobserved

Maximumobserved

Minimumobserved

Dairy farm risk points risk points risk points

Risk areaTotal farms

assessed 95% CI1 Trt2 Control Trt Control Trt Control Trt Control

(no.) ( % ) 3 (no.)Treatment records and com-

munication 133 53.2 46.8–59.5 68 65 12 12 29 30 0 1Understanding how to use an-

tibiotics 45 18.0 13.4–23.4 21 24 7 6 19 20 0 0Relationships between

veterinarians and clients 39 15.6 11.3–20.7 16 23 7 7 16 20 1 2Use of cowside screening tests

on the farm 36 14.4 10.3–19.4 20 16 8 7 17 17 2 1Identification of treated cows 29 11.6 7.9–16.2 14 15 5 5 20 20 0 0

TABLE 4. Results of a risk assessment survey from 250 farms located in seven states taken during theinitial visit showing the five most commonly reported risk areas on these farms.

1Treatment farms.

First visit Second visit

Risk area Trt1 Control Trt Control

(no.)Treatment records and communicationNo written treatment records 65 62 33 41All milkers would recognize a treated cow 110 112 105 107More than two people milking 64 59 51 49Part-time milkers on the farm 66 61 46 46Records include name of drug used 49 49 70 56Records include milk withholding time 44 49 71 66

Understanding how to use antibioticsAgree that increasing the dose will increase

the withholding time 110 105 107 106Belief that labeled dose is only a guideline 49 64 47 47Only one person treats cows 69 67 76 69

Use of cowside screening tests on the farmHave screened milk from a cow for antibiotics

in previous 12 mo 93 91 90 88Milk cows that have been treated last 65 67 58 62Use separate milking equipment for treated cows 40 34 52 40

Identification of treated cowsHad treated a cow with antibiotics in previous 90 d 119 112 100 100Treated cows were visibly marked 95 87 89 90

The relative change in risk was influenced by twovariables, field representative and antibiotic residue.Success of field representatives in influencing changevaried widely. The least squares means for relativedifference in risk for the 23 field representativesranged from –0.37 to 0.12. Least squares mean was<0 for 19 of the 23 representatives, suggesting that

the majority of the representatives influencedproducers to decrease their risk. Box and whiskerplots that demonstrate the variation of representativeinfluence on the outcome are shown in Figure 3.

The second most important variable affecting riskoutcome was antibiotic residues. Participants whohad no history of antibiotic residues demonstrated a


SISCHO ET AL.784

TABLE 5. Results of a risk assessment survey from 219 farms in seven states taken during the second visit showing the five mostcommonly reported risk areas on all participating farms.

1The 95% exact binomial confidence interval for the percentage.2Treatment farms.3Total percentage is >100% because some farms identified more than one risk area as the greatest.

Medianobserved

Maximumobserved

Minimumobserved

Total farmsassessed

Dairy farms risk points risk points risk points

Risk area 95% CI1 Trt2 Control Trt Control Trt Control Trt Control

(no.) ( % ) 3 (no.)Treatment records and

communication 98 45 38.0–51.6 48 50 7 9 28 25 0 0Understanding how to use

antibiotics 45 21 15.4–26.5 25 20 6 6 19 19 0 0Relationships betweenveterinarians and clients 36 16 11.8–22.0 21 15 6 6 18 20 0 0Use of cowside screening

tests on the farm 49 22 17.0–28.5 24 25 6 6 16 15 0 0Identification of treated cows 18 8 4.9–12.7 9 11 5 5 20 20 0 0

TABLE 6. Mean absolute change (MAC) and mean relative change (MRC) in risk between first and second visits for risk assessment.1

1Data are shown for 219 herds and are reported as MAC and MRC for the treatment group (n = 110), the controls (n = 109), and forboth groups combined.

Herd

All Treatment Control

Risk area MAC MRC MAC MRC MAC MRC

(points) ( % ) (points) ( % ) (points) ( % )Overall change in risk –11.19 –19 –10.9 –19 –11.48 –18Treatment records and communication –3.27 –7 –3.98 –9 –2.55 –4Understanding how to use antibiotics –1.69 –14 –1.53 –9 –1.85 –19Relationships between veterinarians and clients –1.44 –5 –0.96 2 –1.91 –13Use of cowside screening tests on the farm –1.24 –8 –1.24 –9 –1.25 –8Identification of treated cows –1.54 –12 –1.59 –11 –1.5 –12

greater mean decrease in risk (–0.135) between thetwo visits than did producers who had a history ofantibiotic residues (–0.077).

Qualitative Evaluationof the Risk Assessment Process

Ten of the 23 field representatives attended one ofthe focus group evaluations. Most field representa-tives thought that the implementation of the riskassessment program did result in altered practices,particularly in the identified high risk areas. Thefield representatives generally thought that the pro-gram, as presented to the producers in the controlgroup, did not result in much change, although someof the control producers did initiate programs to de-velop records and to mark treated cows. As a commu-nication device, the risk assessment tool was seen as

a valuable means to discuss antibiotic residues and todevelop a dialogue between the field representativesand producers in a nonregulatory setting. Negativecomments regarding the participation in the projectwere few and mostly revolved around the extra workload and the need to engage veterinarians in theprocess. Several of the representatives thought thatveterinarians needed to be more responsible forresidue programs.

DISCUSSION

The risk assessment tool was designed to comple-ment the MDBQAP by helping producers to identifyspecific risk practices and to focus educational andmanagement efforts. The weighting scale of thecategories was based on objective and subjective defi-ciencies on the farm. Under this scheme, the category



Figure 3. Box and whisker plots of the influence of field representatives on relative on-farm antibiotic risk reduction on treatment andcontrol farms between first and second visits.

with the most substantial risk for antibiotic residuewas poor treatment records and communication, fol-lowed by relationships among the veterinarian, client,and patient; understanding the use of antibiotics; andidentification of treated cows.

By far the most common risk assessed on all farmsduring the first visit was keeping treatment records.More than 50% of all farms at the start of the projectdid not keep a written record of treatments. Druglabeling was a primary on-farm risk on <1% of thefarms, which was not surprising because the regula-tory penalty for a producer and a veterinarian forimproper labeling of antibiotics is high. Both theveterinarian and, in particular, the milk receiverrepresentative have a great deal of expertise andknowledge in labeling regulation. In addition, thedairy industry has put great emphasis on ensuringproper labeling of antibiotics on the farm.

Overall, the treatment and control groups hadsimilar risk profiles and practices at the beginning ofthe project, indicating that the allocation process usedto assign farms to treatment and control groups wassuccessful. The exception was in the area of under-standing drug use. More control farms (64 of 125)than treatment farms (49 of 125) thought that labelson antibiotics were only a guideline for use of thedrugs (Table 4).

The primary goal of the project was to test thehypothesis that identification of on-farm risk andfollow-up visits to focus educational efforts to reducethe risk of antibiotic residue would be more effectivethan risk assessment with only a generalized ap-proach toward risk reduction as in the MDBQAPbooklet (3) . The educational effort tested was iden-tification of the primary risk of the farm, focuseddiscussion between the educator and producer on theidentified risk, and development of a plan to alleviatethat risk. Three to 5 mo after the initial visit, allfarms showed a mean decrease in the risk for antibi-otic residue of 19% (Table 6). Proportionately fewerfarms identified treatment records and identificationof treated cows as their primary risks, and the num-ber of producers that identified on-farm screening forantibiotics as their primary risk increased proportion-ately.

No difference was detected in the relative overallchange in risk between treatment and control opera-tions. In both groups, the number of farms that iden-tified treatment records as their primary riskdecreased. Producers in the treatment group weremore likely to keep a complete written record of treat-ment. The producers on the control farms improvedrelatively more in regard to understanding the use ofantibiotics, particularly in that labels on antibioticsare more than just guidelines.


SISCHO ET AL.786

TABLE 7. Least squares analysis of covariance model assessing influences on the relative change inrisk for the occurrence of an antibiotic residue.

1R2 = 0.232941; relative difference of the mean = –0.187.2Risk assessment treatment groups: field representative developed plan or did not develop a plan.3Age categories for producers: 35 yr of age, 35 to 50, and >50.4Producer’s history of at least one antibiotic residue in bulk tank milk: yes or no.5Producer’s belief that antibiotic use is an important issue to the dairy industry: yes or no.6Producer’s educational background: grade school, high school, or college background.7Producer’s belief that she or he will have an antibiotic residue in the coming year: not likely or

likely.

Source df SS MS F P

Model1 30 2.501 0.083 1.91 0.0050Error 189 8.238 0.043Group2 1 0.000 0.000 0.0 0.98Age3 2 0.052 0.026 0.60 0.55Antibiotic residue4 1 0.128 0.128 2.94 0.08Residue issue5 1 0.001 0.001 0.01 0.90Field representative 22 1.964 0.089 2.05 0.005Education6 2 0.015 0.008 0.18 0.838Likelihood of violation7 1 0.006 0.006 0.14 0.704

Three important observations emerged from themultivariate analysis of factors associated with therelative change between the first and second visits.First, as observed in the bivariate analysis, meandecrease in the risk for both groups was similar.Second, the impact of the field representative on ef-fecting change was very important. Third, a history ofantibiotic residues in the bulk tank made producersless likely to change.

The first observation suggests that the use of a riskassessment tool for antibiotic residues in a nonregula-tory setting could result in management change. Fol-lowing the risk assessment, change occurred, regard-less of the extra educational efforts associated withdirect involvement of the producer in the developmentof risk reduction plans. Regardless of implementationfollowing the initial risk assessment, producersdecreased the risk for antibiotic residue. The changesobserved, that is, increased use and quality of records,would probably not have occurred without producerparticipation in the project, although that possibilitycould not be specifically evaluated. The response thatwe observed for the control farms was likely due tothe Hawthorne effect; that is, the process of complet-ing the risk assessment tool to obtain baseline infor-mation resulted in change and demonstrated the ef-fectiveness of the risk assessment tool in focusingproducer attention on risk management practices.

Field representatives, as educators, were influen-tial in decreasing risk. More than 80% of the fieldrepresentatives in this project influenced on-farmmanagement, decreasing the risk of residue in the

bulk tank milk. This influence was independent ofwhether the farm was in the treatment or controlgroup and was independent of the use of the riskassessment tool by the field representatives. Theresults of the qualitative evaluations of the programby the field representatives suggested that the riskassessment tool gave representatives the opportunityto focus on residue prevention in their interactionswith the producers. In the focus group evaluations,many of the field representatives stated that the riskassessment tool changed the perception of theproducers regarding their own practices, causingthem to be open to suggestions.

Although mean change for risk was not differentbetween treatment and control farms, the representa-tives did support the concept of developing an actionplan for the treatment farms on the basis of the riskassessment. Comments emerging from the focusgroups matched quantitative results that indicated animprovement of the records kept on treatment farms.

The third observation was that producers whoreported a history of antibiotics in their bulk tank didnot decrease their risk as much as those who did notreport such a history, which provides a challenge tothe dairy industry. The data from our study suggestthat the group that might need the most help (i.e.,producers whose own management system has al-ready been shown to be flawed) might be the mostresistant to adapting measures that would avoidresidue. This group of producers represented >60% ofour sample and suggests that extra efforts are neces-sary to move the whole industry toward decreased



antibiotic residues. On the positive side, theproducers who have been successful in avoiding an-tibiotic residues responded well to the risk assess-ment approach and implemented managementchange aimed at prevention.

CONCLUSIONS

In our study, the producers and field representa-tives from milk receivers identified a lack of adequateon-farm treatment records for antibiotic residues asthe highest risk factor, followed by deficiencies inunderstanding how to use antibiotics, the relationshipbetween veterinarian and client, on-farm use of an-tibiotic residue tests, and identification of treatedcows. When field representatives jointly utilized aninteractive risk assessment tool with their producers,most producers responded by decreasing risk by ap-proximately 19%. In particular, more farms kept writ-ten records or kept more complete records than previ-ously. The field representatives were effective in theirrole as educators and as agents of change, promotinga reduction in the risk for antibiotic residues. Thedairy industry should take advantage of the opportu-nity to use on-farm representatives to implementresidue reduction programs. Field representatives caneffectively introduce residue programs and buildteams of producers and veterinarians to implementthem. Finally, in our study, producers with reportedhistories of antibiotics in the bulk tank were lesslikely to implement management changes to reducerisk of an antibiotic residue. This group of producersshould be targeted to encourage change, which wouldlikely entail time with the individual producer tointroduce programs, followed by specific on-farmvisits to encourage continued participation.

ACKNOWLEDGMENTS

The authors thank the 25 field representatives whodevoted their time and effort to make this project asuccess. This material was based on work supportedin part by the Extension Service, United StatesDepartment of Agriculture, under special projectnumber 92-EFSQ-1-4064 and material support fromAtlantic Dairy, Hershey Milk, Eastern MilkProducers, and Milk Marketing Inc.

REFERENCES

1 Anonymous. 1989. “Grade A” Pasteurized Milk Ordinance. USDep. Health Human Serv., Publ. Health Serv., FDA, Washing-ton, DC.

2 Bauman, H. E. 1974. The HACCP concept and microbiologicalhazard categories. Food Technol. 28:30.

3 Boeckman, S., and K. R. Carlson. 1992. Milk and Dairy BeefResidue Prevention Protocol, 1993 Producer Manual. Agri-Education Inc., Stratford, IA.

4 Cullor, J. S. 1995. Implementing the HACCP program on yourclient’s dairies. Vet. Med. 90:290.

5 Dillman, D. 1978. Mail and Telephone Surveys, the Total De-sign Method. John Wiley and Sons, New York, NY.

6 Herrick, J. B. 1991. Milk quality assurance and dairy practi-tioners. JAVMA 199:1268.

7 Kaneene, J. B., and A. S. Ahl. 1987. Drug residues in dairycattle industry: epidemiological evaluation of factors influenc-ing their occurrence. J. Dairy Sci. 70:2176.

8 Kleinbaum, D. G., L. L. Kupper, and K. E. Muller. 1988. Ap-plied Regression Analysis and Other Multivariable Methods.2nd ed. PWS-Kent, Boston, MA.

9 Krueger, R. A. 1988. Focus groups: A Practical Guide for Ap-plied Research. Sage Publ. Inc., Newbury Park, NJ.

10 McEwen, S. A., W. D. Black, and A. H. Meek. 1991. Antibioticresidue prevention methods, farm management, and occurrenceof antibiotic residues in milk. J. Dairy Sci. 74:2128.

11 Merton, R. K., M. Fisk, and P. L. Kendall. 1956. The FocusedInterview: A Manual of Problem and Procedures. The FreePress, Glencoe, IL.

12 Van Dresser, W. R., and J. R. Wilcke. 1989. Drug residues infood animals. JAVMA 194:1700.

Documents

Implementing a Quality Assurance Program Using a Risk Assessment Tool on Dairy Operations