FEATURE

A study of long-term learningoutcomes transfer—For OSHAoutreach train-the-trainercourses

Judy L.EnvironmContinuversity ocine, Cin(Tel.: +51708; e-

1074-9098

doi:10.101

The purpose of this study was to determine long-term (four months after the training interven-tion) transfer of training skills taught in OSHA Outreach trainer courses by University ofCincinnati (U.C.) instructors, as evidenced by a change in trainer behaviors to conform to thosetaught in U.C. train-the-trainer short courses. Participants were recruited from training coursesgiven by the Continuing Medical Education Office, Great Lakes Regional OTI Education Center(GLROEC). No exclusion criteria were used. During the course of the study, course participantswere asked at the end of the training on effective training techniques if they would be willing to bea part of the study. A small incentive was offered for participation. Those who consented toparticipate were also asked to write down one teaching behavior learned in the training that theyplanned to integrate into their own teaching behaviors for future training courses. At approxi-mately 4 months subsequent to the training, participants received a self-reporting questionnaireto complete and return. Forty to sixty participants were anticipated, with ages ranging from 25 to60 years, however, only 20 usable (those who had actually trained during the reporting period)responses were returned.

The study results were very positive and showed only one respondent answering ‘‘not at all’’ tothe first question: ‘‘Have you changed your training/planning for training behaviors as a result ofthe training you received . . ..’’ The respondent’s reason for this answer was: ‘‘My ProgramExceeds Current Elements.’’ All other respondents (19) listed changes they made in their trainingand/or planning for training behaviors. When responding to the last question about effects ontrainees as a result of changing their training behaviors, thirty-three positive effects, twoindiscernible effects, and three negative effects were reported.

By Judy L. Jarrell

PURPOSE

The purpose of this study was to deter-

mine to what extent there is long-term(four months after the training inter-vention) transfer of skills taught in anOSHA trainer course by University ofCincinnati instructors, as evidencedby a change in trainer behaviors to

Jarrell, Ed.D, is Director ofental Health Department,

ing Medical Education, Uni-f Cincinnati, College of Medi-cinnati OH 45267-0567, USA13 558 1729; fax.: +513 558mail: jarreljl@email.uc.edu).

/$30.00

6/j.chs.2004.05.018

conform to those taught in Universityof Cincinnati train-the-trainer shortcourses.

SIGNIFICANCE OF THE STUDY

Feedback from the participants in thisstudy may lead to the improvement oftraining curriculum (especially for therecommendation of appropriate train-ing formats) to assure maximum long-term training efficacy for future safetyand health training provided by theUniversity of Cincinnati ContinuingMedical Education office, Great LakesRegional OTI Education Center, Uni-versity of Cincinnati NIOSH Educa-tion & Research Center continuingeducation program and the OSHATraining Institute.

� Division of Chemical Health

BACKGROUND

Both the Occupational Safety andHealth Administration and theNational Institute for OccupationalSafety and Health have been encoura-ging their University-based trainingpartners to explore ways to determinethe efficacy of the training given tohealth and safety professionals in theseprograms. This study will build uponprevious studies completed by thePrincipal Investigator, Dr. Judy L. Jar-rell, which identified three compo-nents that affect transfer of learning:training effectiveness, trainee attitudes,and impediments to behavior changeon the job (‘‘Regulated MandatoryContinuing Education and Transfer/Implementation of Learning,’’ Univer-sity of Cincinnati, 19941). This study

and Safety of the American Chemical Society 7Elsevier Inc. All rights reserved.

will look at the training effectivenesscomponent. According to Borich(1992),2 Clark and Voogel (1985),3

Hallinger and Greenblatt (1990),4

and Perkins and Salomon (1988),5 cer-tain training behaviors (dependentupon the type of learning transferdesired) must be utilized for maximumlearning transfer to on-the-job beha-viors (evidenced, in this study, by train-ing behavior changes). Effectivetraining techniques for transfer in thisstudy are: (1) teaching to clearly stateddesired competencies; (2) varying thetraining formats, and visual media, (3)using credible instructors, (4) usingpositive reinforcement and feedback,(5) using a variety of examples, (6)planning time for student interactionand practice, and (7) engaging traineesin the learning process through ques-tioning, discussions, and workshops.

PROCEDURES USED

Participants were recruited from train-ing courses given by the ContinuingMedical Education office. Forty tosixty participants were anticipated,with ages ranging from 25 to 60 years,however, only 20 usable (those whohad actually trained during the report-ing period) responses were returned.

During the course of the study, thefollowing occurred:

Course participants were asked at theend of the training on effective trainingtechniques if they would be willing to bea part of this study. A small incentive(novelty calculator) was offered. Thosewho consented to participate were alsoasked to write down (not for collection)one teaching behavior learned in thetraining that they planned to institutein their own teaching behaviors forfuture training courses.

At approximately four (4) monthssubsequent to the training, participantswere sent a questionnaire to complete,and return to the Principal Investigator(Dr. Jarrell). A postage-free returnenvelope was included with the ques-tionnaire. There was no request forpersonal identifying information onthe questionnaire (in other words,responses were anonymous). This eva-luation method is based on Kirkpa-trick’s (1967)6 four steps to trainingevaluation (Step #3). The self-report

8

evaluation instrument was designedwith input from course trainees andERC/CE Advisory Committee memb-ers, and approved by the University ofCincinnati Institutional Review Board.

SETTINGS

Recruitment of participants wasthrough Continuing Medical Educa-tion/Great Lakes Regional OTI Edu-cation Center safety train-the-trainershort courses held in various locationsin the Greater Cincinnati and Colum-bus, Ohio areas.

RESULTS

The study results were very positiveand showed only one respondentanswering ‘‘not at all’’ to the first ques-tion: ‘‘Have you changed your train-ing/planning for training behaviors asa result of the training you received. . ..’’ That respondent’s reason forhis/her answer was: ‘‘My ProgramExceeds Current Elements.’’ All otherrespondents (19) listed changes theymade in their training and/or planningfor training behaviors:

All respondents, except the one whoanswered ‘‘not at all,’’ and one respon-dent who answered ‘‘extensively,’’answered ‘‘somewhat’’ to the firstquestion regarding how much theyhad changed their training behaviorsas a result of the U.C. training.

The 22 respondents listed anywherefrom ‘‘0’’ (2) to ‘‘10’’ (2) courses pre-sented during the reporting period.The two responses listing ‘‘0’’ trainingcourses presented were not consideredin the results, therefore, leaving 20 asthe ‘‘n’’ of the study results. Three-fourths of the respondents (15),reported completing 1–5 trainingofferings during the reporting period.One-fourth (5) reported completing 6–10 training offerings. Giving the fol-lowing values to the responses:

‘‘Somewhat’’ = 1

‘‘Extensively’’ = 2

The respondents reporting 6–10training offerings also reported a sig-nificantly higher extent of behaviorchanges (average 20 value points) thanthe respondents reporting 1–10 train-ing offerings (average 15 value points).

Chemical Health

[Note: With 18 possible categories forchange, at the ‘‘extensively’’ level, thetotal possible value points are 36.]

The most often cited barrier tochange (question #2) was ‘‘InsufficientManagement Support’’ (2), however,these two respondents classified thisbarrier as just somewhat. Anotherrespondent stated that ‘‘InsufficientTime’’ was a major barrier to changein training behaviors (Figure 2).

For question #3 (Figure 1), mostrespondents reporting the implementa-tion of the behaviors/techniquestaught in the trainer course, reportedthe extent of implementation to be‘‘somewhat’’:

� S

&

eventeen (85%) of the respondentsreported incorporating the conceptson the way adults learn into theplanning and presentation of theirtraining courses as either ‘‘some-what’’ (75%) or ‘‘extensively’’ (10%);

� S

ixteen (80%) reported that theymade an effort to find out as muchas possible about the people whowere to be in their training coursesas either ‘‘somewhat’’ (60%) or‘‘extensively’’ (20%);

� T

welve (60%) of the respondentsreported adding the establishing oftheir credibility to their trainingintroductions as either ‘‘somewhat’’(50%) or ‘‘extensively’’ (10%);

� S

ixteen (80%) reported adding ‘‘set-ting expectations’’ to their trainingintroductions as either ‘‘somewhat’’(70%) or ‘‘extensively’’ (10%);

� S

ixteen (80%) reported they imple-mented the technique of ‘‘establish-ing relevance of the training’’ to theirintroductions as either ‘‘somewhat’’(45%) or ‘‘extensively’’ (35%);

� F

ifteen (75%) of the respondentsreported they were now using‘‘attention-getters’’ in their introduc-tions as ‘‘somewhat’’ (45%) or‘‘extensively’’ (30%).

� O

ne respondent reported adding‘‘class experiences’’ extensively intothe introduction of his/her training.

� F

ourteen (70%) of the respondentsreported they added ‘‘variety’’ intotheir training presentations ‘‘some-what’’ (60%) or ‘‘extensively’’ (10%).

� S

ixteen (80%) reported working on‘‘clarity’’ of the training ‘‘somewhat’’(60%) or ‘‘extensively’’ (20%).

Safety, November/December 2004

Figure 1. Behavior Changes Reported by Respondents.

� F

Ch

ourteen (70%) of the respondentsadded planning for sufficient timeinto their training planning ‘‘some-what’’ (50%) or ‘‘extensively’’ (20%).

� F

ifteen (75%) reported ‘‘planning fortrainee success’’ ‘‘somewhat’’ (40%)or extensively (35%).

� T

en (50%) of the respondentsreported that they were now incor-porating adequate practice timewithin their training courses ‘‘some-what’’ (45%) or ‘‘extensively’’ (5%).

� O

ne respondent reported incorpor-ating ‘‘competition between groups’’into his/her training format.

� T

en (50%) respondents added ques-tioning techniques ‘‘somewhat’’(30%) or ‘‘extensively’’ (20%).

� T

en (50%) of the respondentsreported incorporating ‘‘informationprocessing’’ into their training for-mats ‘‘somewhat’’ (45%) or ‘‘exten-sively’’ (5%).

� E

ighteen (90%) of the respondentsreported working to improve theirvisual aids ‘‘somewhat’’ (55%) or‘‘extensively’’ (35%).

� F

ourteen (70%) of the respondentsincorporated voice variation intotheir presentation skills ‘‘somewhat’’(50%) or ‘‘extensively’’ (20%).

emical Health & Safety, November/Dece

� F

m

ourteen (70%) reported addingmore eye contact to their presenta-tion skills ‘‘somewhat’’ (40%) or‘‘extensively’’ (30%).

� T

hirteen (65%) of the respondentsreported working on gestures/move-ment in their presentation skills‘‘somewhat’’ (50%) or ‘‘extensively’’(15%).

� T

hirteen (65%) reported that theyhave incorporated new training aidsinto their presentations ‘‘somewhat’’(45%) or ‘‘extensively’’ (20%).

Sixty-five percent of the respondentsreported that one of the behaviors theyincorporated into their training beha-viors was the one they wrote down on apiece of paper at the conclusion of thetrain-the-trainer course as a behaviorthey felt they would incorporate orchange.

When responding to the last ques-tion about effects on trainees as a resultof changing their training behaviors,the respondents reported 38 techni-ques they incorporated, with 33 posi-tive effects, 2 indiscernible effects, and3 negative effects upon their trainees.

Other comments written by therespondents included:

ber 2004

� ‘‘

It was good for me to change a littleand grow with my presentationskills. I teach a similar class so Igot some new ideas to share withmy students.’’

� ‘‘

I received several comments aboutthe success of this training program.The ideas gained in the OTI classand from talking with other trainerswas a huge benefit. I also had thepleasure of sitting in on one ofM.H.’s classes, which helped megather new ideas and improve uponnew training techniques.’’

� ‘‘

You need to ask before & after,since many techniques were incor-porated before.’’

� ‘‘

I have been a salesman/motiva-tional speaker/lecturer for over tenyears. I would hope that these [pre-sentation] skills were pre-existing.’’

� ‘‘

I am currently setting up class #2.Class#1wasinJanuaryandIhavenotfollowed up closely enough to give adefinitive answer [referring to effecton trainees]. I will be better able tocomment as time passes and traineesare exposed to more challenges.’’

� ‘‘

I found construction worker(s) donot always like the ‘‘Hi Tech’’approach to training. The use of

9

10

actual equipment, both good andbad, work(ed) better than the com-puter slides. I also found the humanaspect of training work wells (sic).’’

� ‘‘

[I] Did in previous classes.’’ [Refer-ring to effective training techniques.]

� ‘‘

Having only completed one courseto date, I am excited about theresponse for future courses.’’

DISCUSSION

Overall, the respondents reported ahigh number of changed training beha-viors corresponding to those whichwere taught in the train-the-trainercourse they completed. The one personwho answered ‘‘Not at All’’ to the firstquestion regarding any changed train-ing behaviors stated the reason wasthat he/she was already utilizing thetraining techniques taught in the class.One training technique was not speci-fically listed in the self-report instru-ment, but was mentioned five times inthe written responses as an effectivetraining technique which had a posi-tive impact/effect on the trainees: thatof utilizing small group formats.

While respondents reported twelveinteractive techniques/formats as gar-nering positive effects from their trai-nees (question #5), the three behaviorswhich were reported changed by theleast number of respondents (10/50%),were all interactive techniques (prac-tice time, questioning techniques, andinformation processing activities).

Since only 22 of the possible 62respondents returned the question-naire, the possibility that many of thenon-respondents did not return thequestionnaire because they did notalter their training behaviors as a resultof the trainer course should be noted.

Figure 2. Tabulated Long-Term Training Outcomes Study Results Questionnaire.

IMPLICATIONS FOR FUTURESTUDIES

Reflecting upon the results generatedby this pilot study instrument, the addi-tion of two more columns for each ofthe effective training/planning techni-ques could be: ‘‘I used this techniqueprevious to your train-the-trainercourse’’ and ‘‘Effect on Trainees’’(where respondent would circle either‘‘positive,’’ ‘‘negative,’’ or ‘‘indiscern-

Chemical Health & Safety, November/December 2004

Fig. 2. (Continued ).

Chemical Health & Safety, November/December 2004

ible’’). The recognition that some of thetrainees may, indeed, already utilizethe effective training technique mightencourage him/her to write in morecomments. Adding the ‘‘Effect on Trai-nees’’ column would eliminate theneed to write out the behavior/effect(question #5), and, thus, render agreater number of responses. Also,‘‘Group Activities’’ should be addedas an effective training technique.

It would also be interesting to have acontrol group which was not asked towrite down a training behavior theyeach felt they would incorporate intotheir training to determine if the writ-ing of the behavior encouraged theincorporation of the behavior.

Two other questions could be incor-porated into the questionnaire: ‘‘Towhat extent did the course trainingtechniques instructor (the first day ofthe course) model the training techni-ques being taught in the course?’’ and‘‘To what extent do you feel theinstructor’s use of the training techni-ques contributed to your use of them inyour own training?’’ These questionscould give some information regardingthe importance of modeling the beha-viors which are being taught, to theirsubsequent incorporation into thetraining behaviors of the students.

Lastly, if a mechanism could beincorporated for encouraging thereturn of the questionnaire (at the timewhen it is requested) by all of the par-ticipants who indicated their willing-ness to do so, the results of the studycould be strengthened.

References1. Jarrell, J. L. Regulated Mandatory Continu-

ing Education and transfer/Implementation

of Learning; UMI; Ann Arbor, MI, 1995.2. Borich, G. D. Effective Teaching Methods;

Macmillan Publishing Company; NewYork, 1992.

3. Clark, R. E.; Voogel, A. Educ. Comm.

Tech. 1985, 33(2), 113.4. Hallinger, P.; Greenblatt, R. Planning &

Changing, 1990, 21, 195.5. Perkins, D. N.; Salomon, G. Educ.

Leadership, 1988, 46(1), 22.6. Kirkpatrick, D. L. T&D Handbook,

Craig, R. I.; Binel, L. R., Eds.;McGraw-Hill: New York, 1967, p. 87.[Note: also mentioned in subsequentissues of the Handbook.]

11