View
24
Download
1
Category
Preview:
Citation preview
Finance in Human Resource
1 Training Need Analysis
People often jump the gun by assuming that training is the best solution to performance
problems Before we make that assumption be sure training is the best solution by
conducting a training need analysis The danger of not conducting training need analysis is
that human resources and training professionals are the ones held accountable for both
success and failure of our programs and products Professionals from all types of
organizations recite epic tales of training programs that didnt solve the problem instructional
products that solved non problems or programs that resulted in entirely unexpected and
often unpleasant outcomes The game has changed Where once we enjoyed a niche
defined by familiar activities like offering classes and facilitating meetings now we are
expected to contribute to results to the bottom line to the business of the organization
Training need analysis identifies the whole problem and offers organizations systematic
solutions Performance analysis allow training professionals proactively partner with their
customers providing valuable service and meeting customized needs with data-driven
coordinated solution systems Good performance analysis maximizes training expenditures
ensuring a fertile environment for successful solutions Too many times within organizations
training is used to try and fix problems that may not be skills or knowledge related Similarly
providing training without having determined a need for can also be a waste of time and
money
The objective of any training program is to determine whether training is needed and if
needed to specify what that training should accomplish Mitchell (1993) describes training
needs analysis as an examination of the existing need for training within an organization In
other words it identifies performance areas or programs within an organization where
training should be applied A training needs analysis gathers information about present
practices and compares these practices to the desired way of doing business The difference
between where you are now and where you want to be defines where a training program
should concentrate its effort This information also establishes a baseline against which
training accomplishments can be measured If you know where you started it is easier to
determine if your training has been effective A training needs analysis may identify more
than one training need These needs should be prioritized and either placed into a formal
training plan or form a data base for future training Training Need is utilized to identify what
training workshops or activities should be provided to employees to improve their work
productivity Focus should be placed on needs as opposed to desires For example training
dollars would be better spent on a new employee in the sales department who needs to
learn Sales Skills for their job duties as opposed to learning Microsoft Publisher which the
employees wants but does not need As in this case is to train new employees to reduce
wastage and increase quality output
Four kinds of problems have been identified that would require training as below
systemic problems
organizational problems
motivational problems
and skill problems
systemic problems A systemic problem is related to the way in which a system works
whatever that way may be For example a organization had a backlog of executives
awaiting promotion This bottleneck caused a problem The movement of these executives
into Managerial positions was slow due to budget constraints placed on the Modernization
and Associated Restructuring (MAR) Training cannot solve systemic problems like this one
organizational problems Organizational problems lie in the way an office or program is
organized For example the MIC insists that heshe make all administrative and public
relations decisions for the office This approach tends to cause delays in some of the routine
administrative functions within the office particularly when the MIC is out of the office The
problem in this situation is organizational a lack of delegation of authority by the MIC In
general problems like this are not solved by training programs However a training seminar
on delegation for the MIC may be part of the solution
motivational problems Trainers are sometimes asked to motivate employees Employee
motivation is not a training function it is a supervisory function Also remember that true
motivation comes from within not from external factors Training may be part of the solution
to a motivation and morale problem but it alone will not create motivation The supervisors
and MIC needs to be trained first to gain skills in motivating others
skills problems Skills problems fall into the domain of training The aim of
training is to change the way a trainee does something ie create a change
in hisher knowledge or skill level Identifying what skills need to be changed is
a primary concern of needs analysis This kind of problem is a training issue
One way to review manufacturing operations for potential training topics is to examine
these operations in terms of five categories of training needs listed by Leslie Rae
(1992)
1048707 newcomers to an organization
1048707 a change in work within an office
1048707 improvement of a poor performer
1048707 developmental requirements
1048707 consideration of a potential promotee
Each of these categories have been examined individually and defined what each
category refers to
1048707 newcomers to a organization When a new employee arrives at an office heshe needs
to become acquainted with the duties and responsibilities of hisher new position This may
require a formal training program for personnel new to an organization or it may be as
simple as a good orientation for someone transferring in from another part of the
organization In any case dont overlook the need for training as part of a new employee
orientation program
1048707 a change in work within an office Change is a constant part of any business Some
changes evolve slowly while other changes arrive suddenly These changes may include
things such as a formal business re-organization new tasks new equipment or new
software These types of change require training and will probably be the biggest source for
new or continuing training requirements
1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect
performance by an employee requires a change in the way an employee does something
Training may be the vehicle used to make this change easier Identification of the need for
this type of training is not always easy
1048707 developmental requirements Many employees seek tasks or knowledge that go well
beyond their basic job requirements These desires are referred to as their self-actualization
or self-development needs Supervisors can often help motivate employees by helping them
satisfy these needs through training
1048707 consideration of a potential promotee In some organizations training is provided as
part of an internal advancement program This training encourages personnel to seek higher
levels of authority and responsibility as well as provide some of the skills needed for these
higher positions Many needs assessments are available for use in different employment
contexts Sources that can help you determine which needs analysis is appropriate for your
situation are described below
1048707 Context Analysis An analysis of the business needs or other reasons the training is
desired The important questions being answered by this analysis are who decided that
training should be conducted why a training program is seen as the recommended solution
to a business problem what the history of the organization has been with regard to
employee training and other management interventions
1048707 User Analysis Analysis dealing with potential participants and instructorstrainers
involved in the process The important questions being answered by this analysis are who
will receive the training and their level of existing knowledge on the subject what is their
learning style and who will conduct the training
1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and
the requirements for performing the work Also known as a task analysis or job analysis this
analysis seeks to specify the main duties and skill level required This helps ensure that the
training which is developed will include relevant links to the content of the job
1048707 Content Analysis Analysis of documents laws procedures used on the job This
analysis answers questions about what knowledge or information is used on this job This
information comes from manuals documents or regulations It is important that the content
of the training does not conflict or contradict job requirements An experienced worker can
assist (as a subject matter expert) in determining the appropriate content
1048707 Training Suitability Analysis Analysis of whether training is the desired solution
Training is one of several solutions to employment problems However it may not always be
the best solution It is important to determine if training will be effective in its usage
1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective
training results in a return of value to the organization that is greater than the initial
investment to produce or administer the training
Mager (1984) describes a number of procedures that can assist a trainer in determining
whether deficiencies exist Five of these procedures are briefly described below Only the
first two however will routinely help a trainer during the needs analysis process
a Performance Analysis
The performance analysis is used to determine why people arent doing something they
are supposed to be doing or why they are doing something they shouldnt be doing This
analysis looks for the difference between actual performance and a desired performance
b Critical Incident Analysis
Critical incident analysis tries to answer the question What isnt happening according to
expectation This process attempts to identify deviations from normal performance with an
objective of returning things to normal operation This type of approach might be used to
focus on the cause of a series of accidents
c Task Analysis
Task analysis breaks down large tasks into small step-by-step components so that each
component can be taught as a separate step This approach is useful for teaching people
how to operate a piece of equipment such as a computer Each step is fairly simple and
easily learned but must be combined with other steps to accomplish the primary task This
process is not a needs analysis method
d Target Population Analysis
The objective of target population analysis is to not teach people what they already know
With careful examination of the characteristics (abilities education interests biases
experience) of those for whom the instruction is intended it is possible to select instruction
for any individual by subtracting what the individual knows from what the individual needs to
know The bottom line match the training to the needs of the individual This process is
more important in training design and development than in needs analysis
e Learning Outcome Analysis
The function of Learning Outcome Analysis is to define the indefinable to make tangible the
intangible - to help you say what you mean by your important but abstract goals to identify
the main performances that go to make up the meaning of the goal and Learning
Outcomes For example how do you measure someone who should develop proper
customer attitude This analysis is a tool for design and development of training material
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
productivity Focus should be placed on needs as opposed to desires For example training
dollars would be better spent on a new employee in the sales department who needs to
learn Sales Skills for their job duties as opposed to learning Microsoft Publisher which the
employees wants but does not need As in this case is to train new employees to reduce
wastage and increase quality output
Four kinds of problems have been identified that would require training as below
systemic problems
organizational problems
motivational problems
and skill problems
systemic problems A systemic problem is related to the way in which a system works
whatever that way may be For example a organization had a backlog of executives
awaiting promotion This bottleneck caused a problem The movement of these executives
into Managerial positions was slow due to budget constraints placed on the Modernization
and Associated Restructuring (MAR) Training cannot solve systemic problems like this one
organizational problems Organizational problems lie in the way an office or program is
organized For example the MIC insists that heshe make all administrative and public
relations decisions for the office This approach tends to cause delays in some of the routine
administrative functions within the office particularly when the MIC is out of the office The
problem in this situation is organizational a lack of delegation of authority by the MIC In
general problems like this are not solved by training programs However a training seminar
on delegation for the MIC may be part of the solution
motivational problems Trainers are sometimes asked to motivate employees Employee
motivation is not a training function it is a supervisory function Also remember that true
motivation comes from within not from external factors Training may be part of the solution
to a motivation and morale problem but it alone will not create motivation The supervisors
and MIC needs to be trained first to gain skills in motivating others
skills problems Skills problems fall into the domain of training The aim of
training is to change the way a trainee does something ie create a change
in hisher knowledge or skill level Identifying what skills need to be changed is
a primary concern of needs analysis This kind of problem is a training issue
One way to review manufacturing operations for potential training topics is to examine
these operations in terms of five categories of training needs listed by Leslie Rae
(1992)
1048707 newcomers to an organization
1048707 a change in work within an office
1048707 improvement of a poor performer
1048707 developmental requirements
1048707 consideration of a potential promotee
Each of these categories have been examined individually and defined what each
category refers to
1048707 newcomers to a organization When a new employee arrives at an office heshe needs
to become acquainted with the duties and responsibilities of hisher new position This may
require a formal training program for personnel new to an organization or it may be as
simple as a good orientation for someone transferring in from another part of the
organization In any case dont overlook the need for training as part of a new employee
orientation program
1048707 a change in work within an office Change is a constant part of any business Some
changes evolve slowly while other changes arrive suddenly These changes may include
things such as a formal business re-organization new tasks new equipment or new
software These types of change require training and will probably be the biggest source for
new or continuing training requirements
1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect
performance by an employee requires a change in the way an employee does something
Training may be the vehicle used to make this change easier Identification of the need for
this type of training is not always easy
1048707 developmental requirements Many employees seek tasks or knowledge that go well
beyond their basic job requirements These desires are referred to as their self-actualization
or self-development needs Supervisors can often help motivate employees by helping them
satisfy these needs through training
1048707 consideration of a potential promotee In some organizations training is provided as
part of an internal advancement program This training encourages personnel to seek higher
levels of authority and responsibility as well as provide some of the skills needed for these
higher positions Many needs assessments are available for use in different employment
contexts Sources that can help you determine which needs analysis is appropriate for your
situation are described below
1048707 Context Analysis An analysis of the business needs or other reasons the training is
desired The important questions being answered by this analysis are who decided that
training should be conducted why a training program is seen as the recommended solution
to a business problem what the history of the organization has been with regard to
employee training and other management interventions
1048707 User Analysis Analysis dealing with potential participants and instructorstrainers
involved in the process The important questions being answered by this analysis are who
will receive the training and their level of existing knowledge on the subject what is their
learning style and who will conduct the training
1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and
the requirements for performing the work Also known as a task analysis or job analysis this
analysis seeks to specify the main duties and skill level required This helps ensure that the
training which is developed will include relevant links to the content of the job
1048707 Content Analysis Analysis of documents laws procedures used on the job This
analysis answers questions about what knowledge or information is used on this job This
information comes from manuals documents or regulations It is important that the content
of the training does not conflict or contradict job requirements An experienced worker can
assist (as a subject matter expert) in determining the appropriate content
1048707 Training Suitability Analysis Analysis of whether training is the desired solution
Training is one of several solutions to employment problems However it may not always be
the best solution It is important to determine if training will be effective in its usage
1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective
training results in a return of value to the organization that is greater than the initial
investment to produce or administer the training
Mager (1984) describes a number of procedures that can assist a trainer in determining
whether deficiencies exist Five of these procedures are briefly described below Only the
first two however will routinely help a trainer during the needs analysis process
a Performance Analysis
The performance analysis is used to determine why people arent doing something they
are supposed to be doing or why they are doing something they shouldnt be doing This
analysis looks for the difference between actual performance and a desired performance
b Critical Incident Analysis
Critical incident analysis tries to answer the question What isnt happening according to
expectation This process attempts to identify deviations from normal performance with an
objective of returning things to normal operation This type of approach might be used to
focus on the cause of a series of accidents
c Task Analysis
Task analysis breaks down large tasks into small step-by-step components so that each
component can be taught as a separate step This approach is useful for teaching people
how to operate a piece of equipment such as a computer Each step is fairly simple and
easily learned but must be combined with other steps to accomplish the primary task This
process is not a needs analysis method
d Target Population Analysis
The objective of target population analysis is to not teach people what they already know
With careful examination of the characteristics (abilities education interests biases
experience) of those for whom the instruction is intended it is possible to select instruction
for any individual by subtracting what the individual knows from what the individual needs to
know The bottom line match the training to the needs of the individual This process is
more important in training design and development than in needs analysis
e Learning Outcome Analysis
The function of Learning Outcome Analysis is to define the indefinable to make tangible the
intangible - to help you say what you mean by your important but abstract goals to identify
the main performances that go to make up the meaning of the goal and Learning
Outcomes For example how do you measure someone who should develop proper
customer attitude This analysis is a tool for design and development of training material
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
One way to review manufacturing operations for potential training topics is to examine
these operations in terms of five categories of training needs listed by Leslie Rae
(1992)
1048707 newcomers to an organization
1048707 a change in work within an office
1048707 improvement of a poor performer
1048707 developmental requirements
1048707 consideration of a potential promotee
Each of these categories have been examined individually and defined what each
category refers to
1048707 newcomers to a organization When a new employee arrives at an office heshe needs
to become acquainted with the duties and responsibilities of hisher new position This may
require a formal training program for personnel new to an organization or it may be as
simple as a good orientation for someone transferring in from another part of the
organization In any case dont overlook the need for training as part of a new employee
orientation program
1048707 a change in work within an office Change is a constant part of any business Some
changes evolve slowly while other changes arrive suddenly These changes may include
things such as a formal business re-organization new tasks new equipment or new
software These types of change require training and will probably be the biggest source for
new or continuing training requirements
1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect
performance by an employee requires a change in the way an employee does something
Training may be the vehicle used to make this change easier Identification of the need for
this type of training is not always easy
1048707 developmental requirements Many employees seek tasks or knowledge that go well
beyond their basic job requirements These desires are referred to as their self-actualization
or self-development needs Supervisors can often help motivate employees by helping them
satisfy these needs through training
1048707 consideration of a potential promotee In some organizations training is provided as
part of an internal advancement program This training encourages personnel to seek higher
levels of authority and responsibility as well as provide some of the skills needed for these
higher positions Many needs assessments are available for use in different employment
contexts Sources that can help you determine which needs analysis is appropriate for your
situation are described below
1048707 Context Analysis An analysis of the business needs or other reasons the training is
desired The important questions being answered by this analysis are who decided that
training should be conducted why a training program is seen as the recommended solution
to a business problem what the history of the organization has been with regard to
employee training and other management interventions
1048707 User Analysis Analysis dealing with potential participants and instructorstrainers
involved in the process The important questions being answered by this analysis are who
will receive the training and their level of existing knowledge on the subject what is their
learning style and who will conduct the training
1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and
the requirements for performing the work Also known as a task analysis or job analysis this
analysis seeks to specify the main duties and skill level required This helps ensure that the
training which is developed will include relevant links to the content of the job
1048707 Content Analysis Analysis of documents laws procedures used on the job This
analysis answers questions about what knowledge or information is used on this job This
information comes from manuals documents or regulations It is important that the content
of the training does not conflict or contradict job requirements An experienced worker can
assist (as a subject matter expert) in determining the appropriate content
1048707 Training Suitability Analysis Analysis of whether training is the desired solution
Training is one of several solutions to employment problems However it may not always be
the best solution It is important to determine if training will be effective in its usage
1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective
training results in a return of value to the organization that is greater than the initial
investment to produce or administer the training
Mager (1984) describes a number of procedures that can assist a trainer in determining
whether deficiencies exist Five of these procedures are briefly described below Only the
first two however will routinely help a trainer during the needs analysis process
a Performance Analysis
The performance analysis is used to determine why people arent doing something they
are supposed to be doing or why they are doing something they shouldnt be doing This
analysis looks for the difference between actual performance and a desired performance
b Critical Incident Analysis
Critical incident analysis tries to answer the question What isnt happening according to
expectation This process attempts to identify deviations from normal performance with an
objective of returning things to normal operation This type of approach might be used to
focus on the cause of a series of accidents
c Task Analysis
Task analysis breaks down large tasks into small step-by-step components so that each
component can be taught as a separate step This approach is useful for teaching people
how to operate a piece of equipment such as a computer Each step is fairly simple and
easily learned but must be combined with other steps to accomplish the primary task This
process is not a needs analysis method
d Target Population Analysis
The objective of target population analysis is to not teach people what they already know
With careful examination of the characteristics (abilities education interests biases
experience) of those for whom the instruction is intended it is possible to select instruction
for any individual by subtracting what the individual knows from what the individual needs to
know The bottom line match the training to the needs of the individual This process is
more important in training design and development than in needs analysis
e Learning Outcome Analysis
The function of Learning Outcome Analysis is to define the indefinable to make tangible the
intangible - to help you say what you mean by your important but abstract goals to identify
the main performances that go to make up the meaning of the goal and Learning
Outcomes For example how do you measure someone who should develop proper
customer attitude This analysis is a tool for design and development of training material
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
1048707 Context Analysis An analysis of the business needs or other reasons the training is
desired The important questions being answered by this analysis are who decided that
training should be conducted why a training program is seen as the recommended solution
to a business problem what the history of the organization has been with regard to
employee training and other management interventions
1048707 User Analysis Analysis dealing with potential participants and instructorstrainers
involved in the process The important questions being answered by this analysis are who
will receive the training and their level of existing knowledge on the subject what is their
learning style and who will conduct the training
1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and
the requirements for performing the work Also known as a task analysis or job analysis this
analysis seeks to specify the main duties and skill level required This helps ensure that the
training which is developed will include relevant links to the content of the job
1048707 Content Analysis Analysis of documents laws procedures used on the job This
analysis answers questions about what knowledge or information is used on this job This
information comes from manuals documents or regulations It is important that the content
of the training does not conflict or contradict job requirements An experienced worker can
assist (as a subject matter expert) in determining the appropriate content
1048707 Training Suitability Analysis Analysis of whether training is the desired solution
Training is one of several solutions to employment problems However it may not always be
the best solution It is important to determine if training will be effective in its usage
1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective
training results in a return of value to the organization that is greater than the initial
investment to produce or administer the training
Mager (1984) describes a number of procedures that can assist a trainer in determining
whether deficiencies exist Five of these procedures are briefly described below Only the
first two however will routinely help a trainer during the needs analysis process
a Performance Analysis
The performance analysis is used to determine why people arent doing something they
are supposed to be doing or why they are doing something they shouldnt be doing This
analysis looks for the difference between actual performance and a desired performance
b Critical Incident Analysis
Critical incident analysis tries to answer the question What isnt happening according to
expectation This process attempts to identify deviations from normal performance with an
objective of returning things to normal operation This type of approach might be used to
focus on the cause of a series of accidents
c Task Analysis
Task analysis breaks down large tasks into small step-by-step components so that each
component can be taught as a separate step This approach is useful for teaching people
how to operate a piece of equipment such as a computer Each step is fairly simple and
easily learned but must be combined with other steps to accomplish the primary task This
process is not a needs analysis method
d Target Population Analysis
The objective of target population analysis is to not teach people what they already know
With careful examination of the characteristics (abilities education interests biases
experience) of those for whom the instruction is intended it is possible to select instruction
for any individual by subtracting what the individual knows from what the individual needs to
know The bottom line match the training to the needs of the individual This process is
more important in training design and development than in needs analysis
e Learning Outcome Analysis
The function of Learning Outcome Analysis is to define the indefinable to make tangible the
intangible - to help you say what you mean by your important but abstract goals to identify
the main performances that go to make up the meaning of the goal and Learning
Outcomes For example how do you measure someone who should develop proper
customer attitude This analysis is a tool for design and development of training material
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
b Critical Incident Analysis
Critical incident analysis tries to answer the question What isnt happening according to
expectation This process attempts to identify deviations from normal performance with an
objective of returning things to normal operation This type of approach might be used to
focus on the cause of a series of accidents
c Task Analysis
Task analysis breaks down large tasks into small step-by-step components so that each
component can be taught as a separate step This approach is useful for teaching people
how to operate a piece of equipment such as a computer Each step is fairly simple and
easily learned but must be combined with other steps to accomplish the primary task This
process is not a needs analysis method
d Target Population Analysis
The objective of target population analysis is to not teach people what they already know
With careful examination of the characteristics (abilities education interests biases
experience) of those for whom the instruction is intended it is possible to select instruction
for any individual by subtracting what the individual knows from what the individual needs to
know The bottom line match the training to the needs of the individual This process is
more important in training design and development than in needs analysis
e Learning Outcome Analysis
The function of Learning Outcome Analysis is to define the indefinable to make tangible the
intangible - to help you say what you mean by your important but abstract goals to identify
the main performances that go to make up the meaning of the goal and Learning
Outcomes For example how do you measure someone who should develop proper
customer attitude This analysis is a tool for design and development of training material
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
2 Training Objectives
Training objectives are one of the most important parts of training program While some
people think of training objective as a waste of valuable time The counter argument here is
that resources are always limited and the training objectives actually lead the design of
training It provides the clear guidelines and develops the training program in less time
because objectives focus specifically on needs It helps in adhering to a plan
Training objective tell the trainee that is expected out of him at the end of the training
program Training objectives are of great significance from a number of stakeholder
perspectives
1 Trainer
2 Trainee
3 Designer
4 Evaluator
Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training
Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise
Secondly it helps in increase in concentration which is the crucial factor to make the training
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention
Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training
Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately
Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants
3 Training Delivery
Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language
Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following
Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training
Blended training solutions
Provide the skills knowledge and support when and where theyre needed
Take advantage of many learning methods
Reinforce knowledge and sustain performance over a longer period of time than
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
traditional training solutions
Below are the individual elements that may comprise a blended training solution
Possible Elements for a Blended Training Solution
Web-Based Training (WBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training available on the Internet
Employees need to access just in time and just enough training at the time of the need
Employees may be working on varying platforms (Windows Macintosh and Unix)
Allows easy access anytime and virtually anywhere
Allows simple update to content
May use a variety of multimedia effects to draw the user in
May be linked to resources outside of the course
Requires computer and Internet access
Requires self-motivation to complete the training
Multimedia-based Training (MBT)
Definition ExamplesUses Pros Cons
Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)
Employees are geographically dispersed or otherwise unable to attend scheduled training
Employees may be unable to access the Internet
Allows easy access to training on a desktop or laptop
Does not need access to the Web
Is not able to take advantage of the power of the Web
Online Help
Definition ExamplesUses Pros Cons
Quick and immediate access to information about a specific task delivered to a user at the users request
Users need quick access to information or a quick refresher to get the job done
Users need a quick cue tip or prompt when they roll the mouse over a screen area
Users need an online tutorial that can be attached to the application
Allows user to get help and keep working
Allows limited detail
Distance Learning
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
Definition ExamplesUses Pros Cons
An instructor-led approach where the instructor and participant are separated by place or time
An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor
Employees are scattered geographically
Schedules prevent employees from attending face-to-face training
Avoids costly travel for geographically scattered employees
Avoids the need to be physically in a classroom
Provides some interaction between the instructor and participants
Allows limited interaction with other participants and the instructor
Reference Documentation
Definition ExamplesUses Pros Cons
Factual or procedural information that supports a person doing a particular job after initial learning has occurred
Information includes job aids charts posters user manuals and reference guides
Helps sustain learning
Serves as ongoing reference
Allows users easy access to structured information
No interaction
Face-to-Face Training
Definition ExamplesUses Pros Cons
An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location
Participants benefit from practice and feedback
Subject matter requires a classroom or laboratory situation
Allows participant and instructor to carry on detailed conversations about unclear points
Requires participants to travel to a certain location during a particular time period
As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line
The training should be planned phase by phase to ensure that production is not interrupted
4 Training Evaluation
A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback
is essential for people to know how they are progressing and also evaluation is crucial to
the learners confidence too And since peoples commitment to learning relies so heavily on
confidence and a belief that the learning is achievable the way that tests and assessments
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
are designed and managed and results presented back to the learners is a very important
part of the learning and development process
People can be switched off the whole idea of learning and development very quickly if
they receive only negative critical test results and feedback Always look for positives in
negative results Encourage and support - dont criticize without adding some positives and
certainly never focus on failure or thats just what you will produce
This is a much overlooked factor in all sorts of evaluation and testing and since this element
is not typically included within evaluation and assessment tools the point is emphasised
point loud and clear here
Evaluation is not just for the trainer or teacher or organisation or policy-makers -
evaluation is absolutely vital for the learner too which is perhaps the most important
reason of all for evaluating people properly fairly and with as much encouragement as the
situation allows
Most of the specific content and tools below for workplace training evaluation is based on the
work of Leslie Rae an expert and author on the evaluation of learning and training
programmes and this contribution is greatly appreciated W Leslie Rae has written over 30
books on training and the evaluation of learning - he is an expert in his field His guide to the
effective evaluation of training and learning training courses and learning programmes is a
useful set of rules and techniques for all trainers and HR professionals
This training evaluation guide is augmented by an excellent set of follow up tools created by
Leslie Rae which includes
bull Evaluation of Learning Questionnaire (LQ) - ready to use
bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
bull Action Plan template and instruction - ready to use
bull Four separate and different
The four levels of Kirkpatricks evaluation model essentially measure
reaction of student - what they thought and felt about the training
learning - the resulting increase in knowledge or capability
behaviour - extent of behaviour and capability improvement and implementationapplication
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
results - the effects on the business or environment resulting from the trainees performance
Kirkpatricks four levels of training evaluation in detail
evaluation level and type
evaluation description and characteristics
examples of evaluation tools and methods
relevance and practicability
1 Reaction
Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example
Did the trainees like and enjoy the training
Did they consider the training relevant
Was it a good use of their time
Did they like the venue the style timing domestics etc
Level of participation
Ease and comfort of experience
Level of effort required to make the most of the learning
Perceived practicability and potential for applying the learning
Typically happy sheets
Feedback forms based on subjective personal reaction to the training experience
Verbal reaction which can be noted and analysed
Post-training surveys or questionnaires
Online evaluation or grading by delegates
Subsequent verbal or written reports given by delegates to managers back at their jobs
Can be done immediately the training ends
Very easy to obtain reaction feedback
Feedback is not expensive to gather or to analyse for groups
Important to know that people were not upset or disappointed
Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same
2 Learning
Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before
Typically assessments or tests before and after the training
Interview or observation
Relatively simple to set up but more investment and thought required than
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
to after the learning experience
Did the trainees learn what what intended to be taught
Did the trainee experience what was intended for them to experience
What is the extent of advancement or change in the trainees after the training in the direction or area that was intended
can be used before and after although this is time-consuming and can be inconsistent
Methods of assessment need to be closely related to the aims of the learning
Measurement and analysis is possible and easy on a group scale
Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment
Hard-copy electronic online or interview style assessments are all possible
reaction evaluation
Highly relevant and clear-cut for certain training such as quantifiable or technical skills
Less easy for more complex learning such as attitudinal development which is famously difficult to assess
Cost escalates if systems are poorly designed which increases work required to measure and analyse
3 Behaviour
Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation
Did the trainees put their learning into effect when back on the job
Were the relevant skills and knowledge used
Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles
Was the change in behaviour and new level of knowledge sustained
Would the trainee be
Observation and interview over time are required to assess change relevance of change and sustainability of change
Arbitrary snapshot assessments are not reliable because people change in different ways at different times
Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool
Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements
The opinion of the trainee which is a relevant indicator is also subjective
Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation
Simple quick response systems unlikely to be adequate
Cooperation and skill of observers typically line-managers are important factors and difficult to control
Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning
Evaluation of implementation and application is an extremely important
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
able to transfer their learning to another person
Is the trainee aware of their change in behaviour knowledge skill level
and unreliable and so needs to be measured in a consistent defined way
360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees
Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria
Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols
Self-assessment can be useful using carefully designed criteria and measurements
assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging
Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below
4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test
Measures would typically be business or organisational key performance indicators such as
Volumes values percentages timescales return on investment and other
It is possible that many of these measures are already in place via normal management systems and reporting
The challenge is to identify which and how relate to to the trainees input and influence
Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured
Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc
This process overlays normal good management practice - it simply needs linking to the training input
Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training
For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training
Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results
For us to obtain a correct result we will assume that the productions operators are able to
read and write and comment truthfully on how the training was conducted They must be
convinced that their output is important for management to improve the working conditions
(b) Return on Investment for the training conducted
HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training
The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
There is resistance to measure the ROI in training and the following are some of the questions asked in the industry
How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI
The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course
The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner
The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace
The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions
Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers
Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management
The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts
ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as
[(monetary benefits ndash cost of the training) cost of the training] x 100
If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc
Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality
Duration of training 33 hrs
Estimated employee numbers 100Period over which benefits are calculated 12 months
Costs
Design and development RM5600
Printing RM4744
Administration RM8000
Floor Space RM25000
Materials RM5000
Trainer Fees RM10500
Salary RM20000
Evaluation RM872
Total cost RM79716
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
Benefits
Labour savings RM45000
Productivity increases RM85000
Other cost savings RM65250
Other income generation RM0
Total benefits RM195250
Return on investment 145
Payback period 14 months
Hence the ROI is calculated as per below
(RM1077321 ndash RM79716 RM79716) X 100
ROI on Training = 145
The above is calculated on the assumption that
1 All the 100 new employees attend the training
2 All the 100 new employees are able to understand and apply the training methods
3 The employees are motivated to attend the training
4 That the management is willing to spend RM79716 on training to achieve the desired benefits
It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps
Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of
units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes
Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions
Recommended