Really Trained or “Reportedly” ” Trained? What FDA Warning ...education-can be defined as the means by which we systematize the situations, conditions, tasks materials, and opportunities

Kristina Spitler Kristina Spitler Training ManagerTraining Manager

Almac Clinical ServicesAlmac Clinical ServicesAugust 22, 2007August 22, 2007

Really Trained orReally Trained or ““ReportedlyReportedly”” Trained?Trained?

What FDA Warning Letters Say AboutWhat FDA Warning Letters Say About the Effectiveness of Corporate Trainingthe Effectiveness of Corporate Training Programs and How to Develop Training Programs and How to Develop Training Programs Employees Want to AttendPrograms Employees Want to Attend

Copyright © 2007, Kristina Spitler – Almac Clinical Services


Training TodayTraining Today

Recruiting employeesRecruiting employees $28,000$28,000Learning Mgmt System Learning Mgmt System $125,000$125,000Training costs per personTraining costs per person $10,000$10,000


Employee who Employee who reallyreally knows what heknows what he’’s doings doing……

PRICELESS


There are some things There are some things money canmoney can’’t buyt buy

Effective TrainingEffective Training

For everything else, you better haveFor everything else, you better have……


Truth or ConsequencesTruth or Consequences

Do you want to know the truth? Do you want to know the truth? …….or suffer the consequences?.or suffer the consequences?


The TruthThe Truth……

FDA observations and warnings related to FDA observations and warnings related to training are becoming more prevalent training are becoming more prevalent

“…“…SOP training in some cases indicated to have SOP training in some cases indicated to have covered a very large number of written covered a very large number of written procedures in one dayprocedures in one day…”…”“…“…no training requirement for the Supervisorno training requirement for the Supervisor…”…”“…“…inadequate traininginadequate training…”…”“…“…employees are not given training in the employees are not given training in the particular operations they performparticular operations they perform…”…”“…“…failure to document trainingfailure to document training…”…”“…“…failure to establish procedures for identifying failure to establish procedures for identifying training needstraining needs…”…”


……and the Consequencesand the Consequences

In 2005, after a wellIn 2005, after a well--publicized case, publicized case, a firm ultimately closed its doors in a firm ultimately closed its doors in the wake of serious compliance the wake of serious compliance deficiencies at its New Jersey facilitydeficiencies at its New Jersey facility2002 consent decree stemmed from 2002 consent decree stemmed from 483 observations that training was 483 observations that training was inadequate, excessive training was inadequate, excessive training was held on the same date, andheld on the same date, anduntrained personnel wereuntrained personnel weretraining otherstraining others


““ReportedlyReportedly”” trained?trained?

extract from a 2005 FDA warning letterextract from a 2005 FDA warning letterhttp://www.fda.gov/cder/warn/2005/320-05-01.pdf

http://www.fda.gov/cder/warn/2005/320-05-01.pdf


““ReportedlyReportedly”” trained?trained?

extract from the same 2005 FDA warning letterextract from the same 2005 FDA warning letter




The Conclusion

“These observations indicate a serious deficiency in the

methods and adequacy of your training efforts.”

extract from the same 2005 FDA warning letterextract from the same 2005 FDA warning letter




extract from a 2004 FDA warning letterextract from a 2004 FDA warning letter

Even earlierEven earlier……..

http://www.fda.gov/foi/warning_letters/archive/g4851d.pdf

http://www.fda.gov/foi/warning_letters/archive/g4851d.pdf


If employees don’t transfer the skills and knowledge from the training to their jobs, you’re

risking patient safety.

The Bottom LineThe Bottom Line


Your Training FocusYour Training Focus

Two major areas on which to focus:Two major areas on which to focus:Utilizing effective training Utilizing effective training methodsmethodsEvaluating training adequacy and Evaluating training adequacy and effectivenesseffectiveness

Effective teaching may be the hardest job there is.

William Glasser


Training CAPA CycleTraining CAPA Cycle

Retraining = common corrective Retraining = common corrective actionactionEffective training = Excellent Effective training = Excellent preventive actionpreventive actionIdentifying the root causes of Identifying the root causes of ineffective trainingineffective training


Root Causes of Ineffective Root Causes of Ineffective TrainingTraining

MotiveMotiveMethodsMethodsIntensityIntensityEvaluationEvaluation


Cause #1Cause #1 -- MotiveMotive

Completion Completion –– Training is documentedTraining is documented


MotiveMotive

ORORCompetency Competency –– Employee truly Employee truly

understands the job understands the job and performs it correctlyand performs it correctly


The GitThe Git--RR--Done MentalityDone Mentality

Motive / CommitmentMotive / CommitmentproblemproblemDocumentation is the Documentation is the motivemotive

““self trainingself training””Read and signRead and signEE--learning trapslearning trapsUnrealistic Unrealistic requirementsrequirements

Copyright © 1999-2007 Git-R-Done Productions / Team Pozer Productions


Efficient vs. EffectiveEfficient vs. Effective

Who would you rather have pack your Who would you rather have pack your parachute parachute -- the the efficientefficient guy or the guy or the effectiveeffective guy?guy?Value effectivenessValue effectivenessover efficiencyover efficiency

Lives are at stakeLives are at stake


Cause #2Cause #2 -- MethodsMethods

How are training methods How are training methods determined?determined?What methods really work What methods really work -- with with adultsadultsLecture and Reading Lecture and Reading ……..oh boy!..oh boy!


““CoveringCovering”” the materialthe material

Memorization is what we resort to when

what we are learning makes no sense.

Anonymous


Retention RatesRetention Rates

Lecture Lecture 5%5%ReadingReading 10%10%AudiovisualsAudiovisuals 20%20%DemonstrationDemonstration 30%30%DiscussionDiscussion 50%50%Practice by doingPractice by doing 75%75%Teaching othersTeaching others 90%90%

ItIt’’s not what s not what you teach that you teach that

countscounts----itit’’s what s what they take they take

away.away.


Educational DomainsEducational Domains

Cognitive: Cognitive: mental skillsmental skillsPsychomotor: Psychomotor: physical skillsphysical skillsAffective: Affective: emotional growthemotional growth

KnowledgeKnowledge

SkillsSkills

AttitudeAttitude


What is learning?What is learning?

"In its broadest sense, learning can be defined as a "In its broadest sense, learning can be defined as a process of progressive change fromprocess of progressive change from ignorance to knowledgeignorance to knowledge, , from inability to competencefrom inability to competence, and from , and from indifference to understandingindifference to understanding

....In much the same manner, instruction....In much the same manner, instruction--or or educationeducation--can be defined as the means by which can be defined as the means by which we systematize the situations, conditions, tasks we systematize the situations, conditions, tasks materials, and opportunities by which learners materials, and opportunities by which learners acquire new or different ways of acquire new or different ways of thinking, feeling, and doingthinking, feeling, and doing." ."

Cameron Fincher, "Learning Theory and Research," in Cameron Fincher, "Learning Theory and Research," in Teaching and Learning in Teaching and Learning in the College Classroomthe College Classroom, edited by Kenneth A. Feldman and Michael Paulson, Ashe , edited by Kenneth A. Feldman and Michael Paulson, Ashe Reader Series, Needham, MA: Reader Series, Needham, MA: GinnGinn Press, 1994.Press, 1994.

http://riannanworld.typepad.com/my_weblog/images/learning_curve_1.JPG


Laws of LearningLaws of Learning

Law of ExerciseLaw of ExercisePeople learn by DOING!People learn by DOING!

Law of ReadinessLaw of ReadinessUnless people are ready, willing, and able Unless people are ready, willing, and able to learn, no learning will take place.to learn, no learning will take place.

Law of AssociationLaw of AssociationTie new information to something known, Tie new information to something known, whenever possible. whenever possible.


Audience AttentionAudience Attention

Minimize attention “troughs” by changing activities and methods often. Make sure that vital points are

communicated in multiple ways using various senses.

Ideal

% a

ttent

ion

Time

Actual

% a

ttent

ion

100% 100%

Time


Perceiving InformationPerceiving Information Learning StyleLearning Style

VisualVisualAuditoryAuditoryKinestheticKinestheticCombinationCombination


Prepare Inform Practice

Evaluate

The Training PipelineThe Training Pipeline


EE--learning learning –– Blessing or CurseBlessing or Curse

Technical Technical resistanceresistance““CheatingCheating”” the the systemsystemLimited Limited interaction and interaction and ability to ask ability to ask questionsquestions

CostCost--effectiveeffectiveTakes less timeTakes less timePracticalPracticalStandardizedStandardizedEngagingEngaging


Read and SignRead and Sign


Cause #3Cause #3 -- IntensityIntensity

Training with the wrong intensity is Training with the wrong intensity is closely linked to training with the closely linked to training with the wrong methodwrong methodAppropriate toAppropriate totask task intensityintensityAppropriate toAppropriate topersonnel personnel intensityintensity

© 1998-2007 AllPosters.com


TMI TMI –– Too Much InformationToo Much Information

““Information is gushing toward your Information is gushing toward your brain like a fire hose aimed at a brain like a fire hose aimed at a teacup.teacup.”” –– DilbertDilbert

“…“…employee had a trainingemployee had a trainingrecord showing training onrecord showing training on6767 written procedures in one day.written procedures in one day.””–– FDA 483FDA 483


Keep it simpleKeep it simple

The most valuable of all talents is The most valuable of all talents is that of never using two words when that of never using two words when one will do. one will do. ---- Thomas JeffersonThomas Jefferson

The most valuable of all talents is that of never The most valuable of all talents is that of never using two words when one will do. using two words when one will do. (minus 4 words)(minus 4 words)


Six Levels of LearningSix Levels of Learning

BloomBloom’’s Taxonomy s Taxonomy –– developed by developed by Benjamin Bloom in the 1950sBenjamin Bloom in the 1950sQualitatively expressing different kinds of Qualitatively expressing different kinds of thinkingthinking

Knowledge (remembering)Knowledge (remembering)Comprehension (understanding)Comprehension (understanding)Application (applying)Application (applying)Analysis (analyzing)Analysis (analyzing)Synthesis (creating)Synthesis (creating)Evaluation (evaluating)Evaluation (evaluating)


Proper intensity to task ratioProper intensity to task ratio

“…“…newly recruited personnel should newly recruited personnel should receive training receive training appropriateappropriate to the to the duties assigned to them.duties assigned to them.””““Continuing training should also be Continuing training should also be given, and itgiven, and it’’s s practical effectiveness practical effectiveness should beshould be periodically assessedperiodically assessed..””

EU Guidance on Good Manufacturing Practice EU Guidance on Good Manufacturing Practice -- Section 2.9Section 2.9


if your employees are if your employees are ““reallyreally””trained trained ……

How do you evaluateHow do you evaluate……


or or ““reportedlyreportedly”” trained?trained?


Cause #4Cause #4 -- EvaluationEvaluation

KirkpatrickKirkpatrick’’s Model s Model ––developed by Donald developed by Donald Kirkpatrick in 1959Kirkpatrick in 1959Info from each level Info from each level serves as a base for next serves as a base for next levellevelLevel 1:Level 1:ReactionsReactionsSmile sheetSmile sheet

React

ions

Lear

ningTr

ansf

er

Result

s


Measuring Reaction Measuring Reaction

When you are When you are in deep trouble, in deep trouble, say nothing, say nothing, and try to look and try to look like you know like you know what youwhat you’’re re doing!doing!

http://www.worth1000.com/register.asp


Measuring LearningMeasuring Learning

Level 2:Level 2:LearningLearningAssess extent Assess extent of knowledgeof knowledgeQuizzesQuizzesQuestionsQuestionsPrePre--test and test and postpost--testtest

Reactions

Learning

Transfer

Results


Hear, Understand, ApplyHear, Understand, Apply

Did they hear you correctly?Did they hear you correctly?Repeat backRepeat back

Do they understand?Do they understand?QuestionsQuestions

Can they apply the information?Can they apply the information?Demonstrate Demonstrate


Measuring TransferMeasuring Transfer

Level 3:Level 3:TransferTransferBehavior on Behavior on the jobthe jobError ratiosError ratiosReworkReworkComplaintsComplaintsObservationsObservations

Reactions

Learning

Transfer

Results


Ensuring knowledge transferEnsuring knowledge transfer

““When operating in a robust quality When operating in a robust quality system environment, it is important system environment, it is important that supervisory managers ensure that supervisory managers ensure that skills gained from training be that skills gained from training be incorporated into dayincorporated into day--toto--day day performanceperformance..””

Pharmaceutical Pharmaceutical cGMPscGMPs: Integrating : Integrating FDAFDA’’s Risks Risk--Based ApproachBased Approach


Measuring ResultsMeasuring Results

Level 4:Level 4:ResultsResultsBottom lineBottom lineQuantitativeQuantitativeTie to Tie to business business initiativesinitiatives Reactions

Learning

Transfer

Results


Measure to the right levelMeasure to the right level

Objectives drive measurement levelObjectives drive measurement levelDefine outcomesDefine outcomesDetermine intensityDetermine intensityBuild measurements on one anotherBuild measurements on one another


Continually Evaluating TrainingContinually Evaluating Training

“…“…continued training is critical to continued training is critical to ensure that the employees ensure that the employees remainremainproficientproficient in their operational in their operational functions and in their understanding functions and in their understanding of of cGMPcGMP regulations.regulations.””

Pharmaceutical Pharmaceutical cGMPscGMPs: Integrating : Integrating FDAFDA’’s Risks Risk--Based ApproachBased Approach


Evaluation from start to finishEvaluation from start to finish

““Under a quality system, Under a quality system, managers managers are expectedare expected to establish training to establish training programs that include the programs that include the following:following:

Evaluation of training needsEvaluation of training needsProvision of training to satisfy these needsProvision of training to satisfy these needsEvaluation of effectiveness of trainingEvaluation of effectiveness of trainingDocumentation of training and/or Documentation of training and/or rere--training.training.”” Pharmaceutical Pharmaceutical cGMPscGMPs: Integrating : Integrating

FDAFDA’’s Risks Risk--Based ApproachBased Approach


Recipe for SuccessRecipe for Success

Motive / CommitmentMotive / CommitmentMethodsMethodsIntensity Intensity EvaluationEvaluation

Effective TrainingEffective Training


Contact Info:Contact Info:Kristina Spitler Kristina Spitler –– Training ManagerTraining Manager

Almac Clinical ServicesAlmac Clinical [email protected]@almacgroup.com

919919--479479--88618861

mailto:[email protected]

Documents

Really Trained or “Reportedly” ” Trained? What FDA Warning ...education-can be defined as the means by which we systematize the situations, conditions, tasks materials, and opportunities