Upload
anita-felix-lopes
View
18
Download
0
Embed Size (px)
Citation preview
Evaluation of Training Program
Anita F. LopesBSN, MSN, RN
Objectives
• Overall Aim: The trainers are able to identify and implement the various methods of ‘Evaluation of the Training Program’
Specific Objectives
The group--Reviews the definition, purposes and types of Evaluation-Defines ‘Evaluation of the Training Program’ and Differentiates Evaluation of Training Program and Training evaluation-Reviews the various models for Evaluation for a Training program-Identifies the various aspects of Evaluation in a training program-Implements the Various Evaluation Methods/Tool
Definition: Evaluation
•Evaluation is the assessment of merit or worth-Scriven,1963
•Evaluation is a systematic determination of a subject's merit, worth and significance, using criteria governed by a set of standards.
-Wikipedia
TYPES
TRAINING PROGRAMME EVALUATION
TRAINING PROGRAMME EVALUATION• A systematic, rigorous, and meticulous application of
scientific methods to assess the design, implementation, improvement, or outcomes of a program.
• It is a resource-intensive process, frequently requiring resources, such as, evaluate expertise, labor, time, and a sizable budget.
• Training program evaluation is a continual and systematic process of assessing the value or potential value of a training program.
• Results of the evaluation are used to guide decision-making around various components of the training (e.g. instructional design, delivery, results) and its overall continuation, modification, or elimination.
DifferentiationTraining Program Evaluation
• Examines the entire program including the training component,
• Training program should be consistently monitored and evaluated to determine progress and areas of improvement.
• All processes within the program contribute to program success or failure, yet all processes are not "training".
• Knowledge, understanding and skill in program evaluation is necessary to effectively evaluate training and development programs
Training Evaluation• Assessment of the
training course, process, or component within a program.
Purpose of Evaluation• Assist an organization, program, project or any
other intervention or initiative to assess any aim, realisable concept/proposal, or any alternative, to help in decision-making; or
• To ascertain the degree of achievement or value in regard to the aim and objectives and results of any such action that has been completed.
• Enable reflection and assist in the identification of future change.
• “Determine the quality of a program by formulating a judgment" Marthe Hurteau, Sylvain Houle, Stéphanie Mongiat (2009).
MODELS OF EVALUATION
Various Models of Evaluation
• Daniel Stufflebeam's CIPP Model (Context, Input, Process, Product) The CIPP Model for evaluation is a comprehensive framework for
guiding formative and summative evaluations of programs, projects, personnel, products, institutions, and systems. This model was introduced by Daniel Stufflebeam in 1966 to guide mandated evaluations of U.S. federally funded projects.
• Robert Stake's Responsive Evaluation Model: Robert Stake (1975) coined the term responsive evaluation. Responsive evaluation distinguishes four generations in the historical development of evaluation: measurement, description, judgment and negotiation. ‘Measurement’ includes the collection of quantitative data.
Various Models of Evaluation• Robert Stake's Countenance Model: This model focuses on description
and judgment. Stake wrote that greater emphasis should be placed on description, and that judgment was actually the collection of data. Stake wrote about connections in education between antecedents, transactions and contingencies (outcomes). He also noted connections between intentions and observations, which he called congruence. Stake developed matrices for the notation of data for the evaluation
• CIRO (Context, Input, Reaction, Outcome): This model is based on four stages or types of evaluation – context, input, reactions and outcome – and is underpinned by a set of three questions, which, according to the authors, the trainer should always bear in mind.
• PERT (Program Evaluation and Review Technique): The Program (or Project) Evaluation and Review Technique, commonly
abbreviated PERT, is a statistical tool, used in project management that is designed to analyze and represent the tasks involved in completing a given project. First developed by the U.S. Navy in the 1950s, it is commonly used in conjunction with the critical path method (CPM).
Various Models of Evaluation• Michael Scriven's Goal-Free Evaluation Approach: This approach is premised on the assumption that
an evaluation should establish the value of a program by examining what it is doing rather than what it is trying to do.
• Provus' Discrepancy Model: Developed in 1969 this model provides information for program
assessment and program improvement. Under this model evaluation is defined as the comparison of an actual performance to the desired standard..
• Illuminative Evaluation Model: An illuminative evaluation is a custom built research strategy which lacks in formal statements of objectives, avoids (but does not exclude) statistical procedures, employs subjective methods, and is primarily interested in the informing function of an evaluation, rather than the more usual inspectoral or grading functions of an evaluation
Kirkpatrick's evaluation model
ASPECTS
Common Tools for Evaluation
• Rating Scales• Observation checklists• Questionnaires• Anecdotes of Observer
1) Evaluation Of Objectives• General/ Overall aim of the Training Program• Specific Objectives• Each session : Will have specific objectives• Specific Evaluation Areas: -Objectives must be set-up in line with the Goal
of the Training Program-Objectives must cover from the definition to
evaluation (feedback) of the specified training content
Tool for Evaluating Educational Objectives
2) Evaluating the Teaching content
• Sequence must be maintainedFor e.g. Definition to Nursing Care• Includes A.V aids• Relevant Examples• Demonstrations/ Re-demo• Feedbacks ( Teaching- Learning activity)• Assignments (Classroom teaching)
Tool for Evaluating the Teaching Content
• PRE TEST POST TEST• POST TEST ONLY
OBSERVATION CHECKLIST
3) EVALUATION OF AV AIDS
Tool for Evaluating A.V aid
• Subject related• Supplements Teaching content• Properly presented• Accurate• Variety • Handled appropriately
4) Evaluation of Speakers• Well Prepared with the content• Work-needs and people identification. • Involvement in training programme and evaluation
development. • Support of pre-event preparation and holding briefing
meetings with the learner. • Giving ongoing, and practical, support to the training
programme. • Holding a debriefing meeting with the learner on their
return to work to discuss, agree or help to modify and agree action for their action plan.
• Reviewing the progress of learning implementation. • Final review of implementation success and
assessment.
Role of speaker• Analyze the need of the group, then facilitate
learning.• To enable learning.• Plans and designs the training programmes.• Collaborator• Selects appropriate teaching technology and
methods.• Enables skill development.• Keeps themselves self abreast with the latest trends
and technology.• Conduct , organize and participate in various
workshops.
RATING
SCALE
Example of Tool for Evaluating Speaker
5) Selection of EvaluatorsThe American Evaluation Association has created a set of Guiding Principles for evaluators. The principles run as follows:
1) Systematic Inquiry: Evaluators conduct systematic, data-based inquiries about whatever is being evaluated.
Selection of Evaluators
2) Competence: Evaluators need the required expertise.3) Integrity/Honesty: Underscored by three principles: impartialityindependencetransparency
Selection of Evaluators4. Respect for People: Evaluators respect
the security, dignity and self-worth of the respondents, program participants, clients.
5. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of interests and values that may be related to the general and public welfare.
6) Teaching Learning Environment
Teaching Learning Environment
The environment has to be conducive in order for learning to take place
Must supplement the effectiveness of teaching
State of the Learners
Effectiveness of the Speaker
Tool for Evaluating the Tng-Lng Envt
Physical facilities : Seating arrangement, Visibility, Lighting, Washroom facilities
Safety measures
Communication equipments : Audible, A.V aids
Room temperature
7) Organization of the Workshop1. Identification and setting of objectives by the
organization 2. Planning, design and preparation of the training
programmes against the objectives 3. Pre-course identification of people with needs and
completion of the preparation required by the training programme
4. Provision of the agreed training programmes 5. Pre-course briefing meeting between learner and
line manager 6. Pre-course or start of programme identification of
learners' existing knowledge, skills and attitudes7. Interim validation as programme proceeds
Organization of the Workshop8. Assessment of terminal knowledge, skills, etc., and
completion of perceptions/change assessment 9. Completion of end-of-programme reactionnaire 10. Completion of end-of-programme Learning
Questionnaire or Key Objectives Learning Questionnaire 11. Completion of Action Plan 12. Post-course debriefing meeting between learner and
line manager 13. Line manager observation of implementation progress 14. Review meetings to discuss progress of implementation 15. Final implementation review meeting
Checklist on Evaluating the Organisation of Workshop
I. RegistrationII. Program Schedule providedIII. Tng-Lng Envt maintainedIV. Implemented as per scheduleV. Mets the requirement of DelegatesVI. Delegate’s Feedback
Need For Evaluation? workshop• Use of evaluation data meets these demands in various ways:
1. Planning: To assess needs, set priorities, direct allocation of resources, and guide policy
2. Analysis of Course/Program Effectiveness or Quality: To determine achievement of objective, identify strengths and weaknesses of a program/course, determine the cost-effectiveness of a program/course, and assess causes of success or failure
3. Direct Decision-making: To improve effectiveness, identify and facilitate needed change, and continue, expand, or terminate a program/course
4. Maintain Accountability: To stakeholders, funding sources, and the general public
SUMMARY