25
Finance in Human Resource 1. Training Need Analysis People often "jump the gun" by assuming that training is the best solution to performance problems. Before we make that assumption, be sure training is the best solution by conducting a training need analysis. The danger of not conducting training need analysis is that human resources and training professionals are the ones held accountable for both success and failure of our programs and products. Professionals from all types of organizations recite epic tales of training programs that didn't solve the problem, instructional products that solved non problems, or programs that resulted in entirely unexpected and often unpleasant outcomes. The game has changed. Where once we enjoyed a niche defined by familiar activities, like offering classes and facilitating meetings, now we are expected to contribute to results, to the bottom line, to the business of the organization. Training need analysis identifies the whole problem, and offers organizations systematic solutions. Performance analysis allow training professionals proactively partner with their customers, providing valuable service and meeting customized needs with data- driven, coordinated solution systems. Good performance analysis maximizes training expenditures, ensuring a fertile environment for successful solutions. Too many times within organizations training is used to try and fix problems that may not be skills or knowledge related. Similarly, providing training without having determined a need for can also be a waste of time and money.

Finance in Human Resource

Embed Size (px)

Citation preview

Page 1: Finance in Human Resource

Finance in Human Resource

1 Training Need Analysis

People often jump the gun by assuming that training is the best solution to performance

problems Before we make that assumption be sure training is the best solution by

conducting a training need analysis The danger of not conducting training need analysis is

that human resources and training professionals are the ones held accountable for both

success and failure of our programs and products Professionals from all types of

organizations recite epic tales of training programs that didnt solve the problem instructional

products that solved non problems or programs that resulted in entirely unexpected and

often unpleasant outcomes The game has changed Where once we enjoyed a niche

defined by familiar activities like offering classes and facilitating meetings now we are

expected to contribute to results to the bottom line to the business of the organization

Training need analysis identifies the whole problem and offers organizations systematic

solutions Performance analysis allow training professionals proactively partner with their

customers providing valuable service and meeting customized needs with data-driven

coordinated solution systems Good performance analysis maximizes training expenditures

ensuring a fertile environment for successful solutions Too many times within organizations

training is used to try and fix problems that may not be skills or knowledge related Similarly

providing training without having determined a need for can also be a waste of time and

money

The objective of any training program is to determine whether training is needed and if

needed to specify what that training should accomplish Mitchell (1993) describes training

needs analysis as an examination of the existing need for training within an organization In

other words it identifies performance areas or programs within an organization where

training should be applied A training needs analysis gathers information about present

practices and compares these practices to the desired way of doing business The difference

between where you are now and where you want to be defines where a training program

should concentrate its effort This information also establishes a baseline against which

training accomplishments can be measured If you know where you started it is easier to

determine if your training has been effective A training needs analysis may identify more

than one training need These needs should be prioritized and either placed into a formal

training plan or form a data base for future training Training Need is utilized to identify what

training workshops or activities should be provided to employees to improve their work

productivity Focus should be placed on needs as opposed to desires For example training

dollars would be better spent on a new employee in the sales department who needs to

learn Sales Skills for their job duties as opposed to learning Microsoft Publisher which the

employees wants but does not need As in this case is to train new employees to reduce

wastage and increase quality output

Four kinds of problems have been identified that would require training as below

systemic problems

organizational problems

motivational problems

and skill problems

systemic problems A systemic problem is related to the way in which a system works

whatever that way may be For example a organization had a backlog of executives

awaiting promotion This bottleneck caused a problem The movement of these executives

into Managerial positions was slow due to budget constraints placed on the Modernization

and Associated Restructuring (MAR) Training cannot solve systemic problems like this one

organizational problems Organizational problems lie in the way an office or program is

organized For example the MIC insists that heshe make all administrative and public

relations decisions for the office This approach tends to cause delays in some of the routine

administrative functions within the office particularly when the MIC is out of the office The

problem in this situation is organizational a lack of delegation of authority by the MIC In

general problems like this are not solved by training programs However a training seminar

on delegation for the MIC may be part of the solution

motivational problems Trainers are sometimes asked to motivate employees Employee

motivation is not a training function it is a supervisory function Also remember that true

motivation comes from within not from external factors Training may be part of the solution

to a motivation and morale problem but it alone will not create motivation The supervisors

and MIC needs to be trained first to gain skills in motivating others

skills problems Skills problems fall into the domain of training The aim of

training is to change the way a trainee does something ie create a change

in hisher knowledge or skill level Identifying what skills need to be changed is

a primary concern of needs analysis This kind of problem is a training issue

One way to review manufacturing operations for potential training topics is to examine

these operations in terms of five categories of training needs listed by Leslie Rae

(1992)

1048707 newcomers to an organization

1048707 a change in work within an office

1048707 improvement of a poor performer

1048707 developmental requirements

1048707 consideration of a potential promotee

Each of these categories have been examined individually and defined what each

category refers to

1048707 newcomers to a organization When a new employee arrives at an office heshe needs

to become acquainted with the duties and responsibilities of hisher new position This may

require a formal training program for personnel new to an organization or it may be as

simple as a good orientation for someone transferring in from another part of the

organization In any case dont overlook the need for training as part of a new employee

orientation program

1048707 a change in work within an office Change is a constant part of any business Some

changes evolve slowly while other changes arrive suddenly These changes may include

things such as a formal business re-organization new tasks new equipment or new

software These types of change require training and will probably be the biggest source for

new or continuing training requirements

1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect

performance by an employee requires a change in the way an employee does something

Training may be the vehicle used to make this change easier Identification of the need for

this type of training is not always easy

1048707 developmental requirements Many employees seek tasks or knowledge that go well

beyond their basic job requirements These desires are referred to as their self-actualization

or self-development needs Supervisors can often help motivate employees by helping them

satisfy these needs through training

1048707 consideration of a potential promotee In some organizations training is provided as

part of an internal advancement program This training encourages personnel to seek higher

levels of authority and responsibility as well as provide some of the skills needed for these

higher positions Many needs assessments are available for use in different employment

contexts Sources that can help you determine which needs analysis is appropriate for your

situation are described below

1048707 Context Analysis An analysis of the business needs or other reasons the training is

desired The important questions being answered by this analysis are who decided that

training should be conducted why a training program is seen as the recommended solution

to a business problem what the history of the organization has been with regard to

employee training and other management interventions

1048707 User Analysis Analysis dealing with potential participants and instructorstrainers

involved in the process The important questions being answered by this analysis are who

will receive the training and their level of existing knowledge on the subject what is their

learning style and who will conduct the training

1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and

the requirements for performing the work Also known as a task analysis or job analysis this

analysis seeks to specify the main duties and skill level required This helps ensure that the

training which is developed will include relevant links to the content of the job

1048707 Content Analysis Analysis of documents laws procedures used on the job This

analysis answers questions about what knowledge or information is used on this job This

information comes from manuals documents or regulations It is important that the content

of the training does not conflict or contradict job requirements An experienced worker can

assist (as a subject matter expert) in determining the appropriate content

1048707 Training Suitability Analysis Analysis of whether training is the desired solution

Training is one of several solutions to employment problems However it may not always be

the best solution It is important to determine if training will be effective in its usage

1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective

training results in a return of value to the organization that is greater than the initial

investment to produce or administer the training

Mager (1984) describes a number of procedures that can assist a trainer in determining

whether deficiencies exist Five of these procedures are briefly described below Only the

first two however will routinely help a trainer during the needs analysis process

a Performance Analysis

The performance analysis is used to determine why people arent doing something they

are supposed to be doing or why they are doing something they shouldnt be doing This

analysis looks for the difference between actual performance and a desired performance

b Critical Incident Analysis

Critical incident analysis tries to answer the question What isnt happening according to

expectation This process attempts to identify deviations from normal performance with an

objective of returning things to normal operation This type of approach might be used to

focus on the cause of a series of accidents

c Task Analysis

Task analysis breaks down large tasks into small step-by-step components so that each

component can be taught as a separate step This approach is useful for teaching people

how to operate a piece of equipment such as a computer Each step is fairly simple and

easily learned but must be combined with other steps to accomplish the primary task This

process is not a needs analysis method

d Target Population Analysis

The objective of target population analysis is to not teach people what they already know

With careful examination of the characteristics (abilities education interests biases

experience) of those for whom the instruction is intended it is possible to select instruction

for any individual by subtracting what the individual knows from what the individual needs to

know The bottom line match the training to the needs of the individual This process is

more important in training design and development than in needs analysis

e Learning Outcome Analysis

The function of Learning Outcome Analysis is to define the indefinable to make tangible the

intangible - to help you say what you mean by your important but abstract goals to identify

the main performances that go to make up the meaning of the goal and Learning

Outcomes For example how do you measure someone who should develop proper

customer attitude This analysis is a tool for design and development of training material

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 2: Finance in Human Resource

productivity Focus should be placed on needs as opposed to desires For example training

dollars would be better spent on a new employee in the sales department who needs to

learn Sales Skills for their job duties as opposed to learning Microsoft Publisher which the

employees wants but does not need As in this case is to train new employees to reduce

wastage and increase quality output

Four kinds of problems have been identified that would require training as below

systemic problems

organizational problems

motivational problems

and skill problems

systemic problems A systemic problem is related to the way in which a system works

whatever that way may be For example a organization had a backlog of executives

awaiting promotion This bottleneck caused a problem The movement of these executives

into Managerial positions was slow due to budget constraints placed on the Modernization

and Associated Restructuring (MAR) Training cannot solve systemic problems like this one

organizational problems Organizational problems lie in the way an office or program is

organized For example the MIC insists that heshe make all administrative and public

relations decisions for the office This approach tends to cause delays in some of the routine

administrative functions within the office particularly when the MIC is out of the office The

problem in this situation is organizational a lack of delegation of authority by the MIC In

general problems like this are not solved by training programs However a training seminar

on delegation for the MIC may be part of the solution

motivational problems Trainers are sometimes asked to motivate employees Employee

motivation is not a training function it is a supervisory function Also remember that true

motivation comes from within not from external factors Training may be part of the solution

to a motivation and morale problem but it alone will not create motivation The supervisors

and MIC needs to be trained first to gain skills in motivating others

skills problems Skills problems fall into the domain of training The aim of

training is to change the way a trainee does something ie create a change

in hisher knowledge or skill level Identifying what skills need to be changed is

a primary concern of needs analysis This kind of problem is a training issue

One way to review manufacturing operations for potential training topics is to examine

these operations in terms of five categories of training needs listed by Leslie Rae

(1992)

1048707 newcomers to an organization

1048707 a change in work within an office

1048707 improvement of a poor performer

1048707 developmental requirements

1048707 consideration of a potential promotee

Each of these categories have been examined individually and defined what each

category refers to

1048707 newcomers to a organization When a new employee arrives at an office heshe needs

to become acquainted with the duties and responsibilities of hisher new position This may

require a formal training program for personnel new to an organization or it may be as

simple as a good orientation for someone transferring in from another part of the

organization In any case dont overlook the need for training as part of a new employee

orientation program

1048707 a change in work within an office Change is a constant part of any business Some

changes evolve slowly while other changes arrive suddenly These changes may include

things such as a formal business re-organization new tasks new equipment or new

software These types of change require training and will probably be the biggest source for

new or continuing training requirements

1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect

performance by an employee requires a change in the way an employee does something

Training may be the vehicle used to make this change easier Identification of the need for

this type of training is not always easy

1048707 developmental requirements Many employees seek tasks or knowledge that go well

beyond their basic job requirements These desires are referred to as their self-actualization

or self-development needs Supervisors can often help motivate employees by helping them

satisfy these needs through training

1048707 consideration of a potential promotee In some organizations training is provided as

part of an internal advancement program This training encourages personnel to seek higher

levels of authority and responsibility as well as provide some of the skills needed for these

higher positions Many needs assessments are available for use in different employment

contexts Sources that can help you determine which needs analysis is appropriate for your

situation are described below

1048707 Context Analysis An analysis of the business needs or other reasons the training is

desired The important questions being answered by this analysis are who decided that

training should be conducted why a training program is seen as the recommended solution

to a business problem what the history of the organization has been with regard to

employee training and other management interventions

1048707 User Analysis Analysis dealing with potential participants and instructorstrainers

involved in the process The important questions being answered by this analysis are who

will receive the training and their level of existing knowledge on the subject what is their

learning style and who will conduct the training

1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and

the requirements for performing the work Also known as a task analysis or job analysis this

analysis seeks to specify the main duties and skill level required This helps ensure that the

training which is developed will include relevant links to the content of the job

1048707 Content Analysis Analysis of documents laws procedures used on the job This

analysis answers questions about what knowledge or information is used on this job This

information comes from manuals documents or regulations It is important that the content

of the training does not conflict or contradict job requirements An experienced worker can

assist (as a subject matter expert) in determining the appropriate content

1048707 Training Suitability Analysis Analysis of whether training is the desired solution

Training is one of several solutions to employment problems However it may not always be

the best solution It is important to determine if training will be effective in its usage

1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective

training results in a return of value to the organization that is greater than the initial

investment to produce or administer the training

Mager (1984) describes a number of procedures that can assist a trainer in determining

whether deficiencies exist Five of these procedures are briefly described below Only the

first two however will routinely help a trainer during the needs analysis process

a Performance Analysis

The performance analysis is used to determine why people arent doing something they

are supposed to be doing or why they are doing something they shouldnt be doing This

analysis looks for the difference between actual performance and a desired performance

b Critical Incident Analysis

Critical incident analysis tries to answer the question What isnt happening according to

expectation This process attempts to identify deviations from normal performance with an

objective of returning things to normal operation This type of approach might be used to

focus on the cause of a series of accidents

c Task Analysis

Task analysis breaks down large tasks into small step-by-step components so that each

component can be taught as a separate step This approach is useful for teaching people

how to operate a piece of equipment such as a computer Each step is fairly simple and

easily learned but must be combined with other steps to accomplish the primary task This

process is not a needs analysis method

d Target Population Analysis

The objective of target population analysis is to not teach people what they already know

With careful examination of the characteristics (abilities education interests biases

experience) of those for whom the instruction is intended it is possible to select instruction

for any individual by subtracting what the individual knows from what the individual needs to

know The bottom line match the training to the needs of the individual This process is

more important in training design and development than in needs analysis

e Learning Outcome Analysis

The function of Learning Outcome Analysis is to define the indefinable to make tangible the

intangible - to help you say what you mean by your important but abstract goals to identify

the main performances that go to make up the meaning of the goal and Learning

Outcomes For example how do you measure someone who should develop proper

customer attitude This analysis is a tool for design and development of training material

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 3: Finance in Human Resource

One way to review manufacturing operations for potential training topics is to examine

these operations in terms of five categories of training needs listed by Leslie Rae

(1992)

1048707 newcomers to an organization

1048707 a change in work within an office

1048707 improvement of a poor performer

1048707 developmental requirements

1048707 consideration of a potential promotee

Each of these categories have been examined individually and defined what each

category refers to

1048707 newcomers to a organization When a new employee arrives at an office heshe needs

to become acquainted with the duties and responsibilities of hisher new position This may

require a formal training program for personnel new to an organization or it may be as

simple as a good orientation for someone transferring in from another part of the

organization In any case dont overlook the need for training as part of a new employee

orientation program

1048707 a change in work within an office Change is a constant part of any business Some

changes evolve slowly while other changes arrive suddenly These changes may include

things such as a formal business re-organization new tasks new equipment or new

software These types of change require training and will probably be the biggest source for

new or continuing training requirements

1048707 improvement of a poor performer Occasionally inefficient ineffective or incorrect

performance by an employee requires a change in the way an employee does something

Training may be the vehicle used to make this change easier Identification of the need for

this type of training is not always easy

1048707 developmental requirements Many employees seek tasks or knowledge that go well

beyond their basic job requirements These desires are referred to as their self-actualization

or self-development needs Supervisors can often help motivate employees by helping them

satisfy these needs through training

1048707 consideration of a potential promotee In some organizations training is provided as

part of an internal advancement program This training encourages personnel to seek higher

levels of authority and responsibility as well as provide some of the skills needed for these

higher positions Many needs assessments are available for use in different employment

contexts Sources that can help you determine which needs analysis is appropriate for your

situation are described below

1048707 Context Analysis An analysis of the business needs or other reasons the training is

desired The important questions being answered by this analysis are who decided that

training should be conducted why a training program is seen as the recommended solution

to a business problem what the history of the organization has been with regard to

employee training and other management interventions

1048707 User Analysis Analysis dealing with potential participants and instructorstrainers

involved in the process The important questions being answered by this analysis are who

will receive the training and their level of existing knowledge on the subject what is their

learning style and who will conduct the training

1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and

the requirements for performing the work Also known as a task analysis or job analysis this

analysis seeks to specify the main duties and skill level required This helps ensure that the

training which is developed will include relevant links to the content of the job

1048707 Content Analysis Analysis of documents laws procedures used on the job This

analysis answers questions about what knowledge or information is used on this job This

information comes from manuals documents or regulations It is important that the content

of the training does not conflict or contradict job requirements An experienced worker can

assist (as a subject matter expert) in determining the appropriate content

1048707 Training Suitability Analysis Analysis of whether training is the desired solution

Training is one of several solutions to employment problems However it may not always be

the best solution It is important to determine if training will be effective in its usage

1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective

training results in a return of value to the organization that is greater than the initial

investment to produce or administer the training

Mager (1984) describes a number of procedures that can assist a trainer in determining

whether deficiencies exist Five of these procedures are briefly described below Only the

first two however will routinely help a trainer during the needs analysis process

a Performance Analysis

The performance analysis is used to determine why people arent doing something they

are supposed to be doing or why they are doing something they shouldnt be doing This

analysis looks for the difference between actual performance and a desired performance

b Critical Incident Analysis

Critical incident analysis tries to answer the question What isnt happening according to

expectation This process attempts to identify deviations from normal performance with an

objective of returning things to normal operation This type of approach might be used to

focus on the cause of a series of accidents

c Task Analysis

Task analysis breaks down large tasks into small step-by-step components so that each

component can be taught as a separate step This approach is useful for teaching people

how to operate a piece of equipment such as a computer Each step is fairly simple and

easily learned but must be combined with other steps to accomplish the primary task This

process is not a needs analysis method

d Target Population Analysis

The objective of target population analysis is to not teach people what they already know

With careful examination of the characteristics (abilities education interests biases

experience) of those for whom the instruction is intended it is possible to select instruction

for any individual by subtracting what the individual knows from what the individual needs to

know The bottom line match the training to the needs of the individual This process is

more important in training design and development than in needs analysis

e Learning Outcome Analysis

The function of Learning Outcome Analysis is to define the indefinable to make tangible the

intangible - to help you say what you mean by your important but abstract goals to identify

the main performances that go to make up the meaning of the goal and Learning

Outcomes For example how do you measure someone who should develop proper

customer attitude This analysis is a tool for design and development of training material

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 4: Finance in Human Resource

1048707 Context Analysis An analysis of the business needs or other reasons the training is

desired The important questions being answered by this analysis are who decided that

training should be conducted why a training program is seen as the recommended solution

to a business problem what the history of the organization has been with regard to

employee training and other management interventions

1048707 User Analysis Analysis dealing with potential participants and instructorstrainers

involved in the process The important questions being answered by this analysis are who

will receive the training and their level of existing knowledge on the subject what is their

learning style and who will conduct the training

1048707 Work analysis Analysis of the tasks being performed This is an analysis of the job and

the requirements for performing the work Also known as a task analysis or job analysis this

analysis seeks to specify the main duties and skill level required This helps ensure that the

training which is developed will include relevant links to the content of the job

1048707 Content Analysis Analysis of documents laws procedures used on the job This

analysis answers questions about what knowledge or information is used on this job This

information comes from manuals documents or regulations It is important that the content

of the training does not conflict or contradict job requirements An experienced worker can

assist (as a subject matter expert) in determining the appropriate content

1048707 Training Suitability Analysis Analysis of whether training is the desired solution

Training is one of several solutions to employment problems However it may not always be

the best solution It is important to determine if training will be effective in its usage

1048707 Cost-Benefit Analysis Analysis of the return on investment (ROI) of training Effective

training results in a return of value to the organization that is greater than the initial

investment to produce or administer the training

Mager (1984) describes a number of procedures that can assist a trainer in determining

whether deficiencies exist Five of these procedures are briefly described below Only the

first two however will routinely help a trainer during the needs analysis process

a Performance Analysis

The performance analysis is used to determine why people arent doing something they

are supposed to be doing or why they are doing something they shouldnt be doing This

analysis looks for the difference between actual performance and a desired performance

b Critical Incident Analysis

Critical incident analysis tries to answer the question What isnt happening according to

expectation This process attempts to identify deviations from normal performance with an

objective of returning things to normal operation This type of approach might be used to

focus on the cause of a series of accidents

c Task Analysis

Task analysis breaks down large tasks into small step-by-step components so that each

component can be taught as a separate step This approach is useful for teaching people

how to operate a piece of equipment such as a computer Each step is fairly simple and

easily learned but must be combined with other steps to accomplish the primary task This

process is not a needs analysis method

d Target Population Analysis

The objective of target population analysis is to not teach people what they already know

With careful examination of the characteristics (abilities education interests biases

experience) of those for whom the instruction is intended it is possible to select instruction

for any individual by subtracting what the individual knows from what the individual needs to

know The bottom line match the training to the needs of the individual This process is

more important in training design and development than in needs analysis

e Learning Outcome Analysis

The function of Learning Outcome Analysis is to define the indefinable to make tangible the

intangible - to help you say what you mean by your important but abstract goals to identify

the main performances that go to make up the meaning of the goal and Learning

Outcomes For example how do you measure someone who should develop proper

customer attitude This analysis is a tool for design and development of training material

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 5: Finance in Human Resource

b Critical Incident Analysis

Critical incident analysis tries to answer the question What isnt happening according to

expectation This process attempts to identify deviations from normal performance with an

objective of returning things to normal operation This type of approach might be used to

focus on the cause of a series of accidents

c Task Analysis

Task analysis breaks down large tasks into small step-by-step components so that each

component can be taught as a separate step This approach is useful for teaching people

how to operate a piece of equipment such as a computer Each step is fairly simple and

easily learned but must be combined with other steps to accomplish the primary task This

process is not a needs analysis method

d Target Population Analysis

The objective of target population analysis is to not teach people what they already know

With careful examination of the characteristics (abilities education interests biases

experience) of those for whom the instruction is intended it is possible to select instruction

for any individual by subtracting what the individual knows from what the individual needs to

know The bottom line match the training to the needs of the individual This process is

more important in training design and development than in needs analysis

e Learning Outcome Analysis

The function of Learning Outcome Analysis is to define the indefinable to make tangible the

intangible - to help you say what you mean by your important but abstract goals to identify

the main performances that go to make up the meaning of the goal and Learning

Outcomes For example how do you measure someone who should develop proper

customer attitude This analysis is a tool for design and development of training material

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 6: Finance in Human Resource

2 Training Objectives

Training objectives are one of the most important parts of training program While some

people think of training objective as a waste of valuable time The counter argument here is

that resources are always limited and the training objectives actually lead the design of

training It provides the clear guidelines and develops the training program in less time

because objectives focus specifically on needs It helps in adhering to a plan

Training objective tell the trainee that is expected out of him at the end of the training

program Training objectives are of great significance from a number of stakeholder

perspectives

1 Trainer

2 Trainee

3 Designer

4 Evaluator

Trainer ndash The training objective is also beneficial to trainer because it helps the trainer to measure the progress of trainees and make the required adjustments Also trainer comes in a position to establish a relationship between objectives and particular segments of training

Trainee ndash The training objective is beneficial to the trainee because it helps in reducing the anxiety of the trainee up to some extent Not knowing anything or going to a place which is unknown creates anxiety that can negatively affect learning Therefore it is important to keep the participants aware of the happenings rather than keeping it surprise

Secondly it helps in increase in concentration which is the crucial factor to make the training

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 7: Finance in Human Resource

successful The objectives create an image of the training program in traineersquos mind that actually helps in gaining attention

Thirdly if the goal is set to be challenging and motivating then the likelihood of achieving those goals is much higher than the situation in which no goal is set Therefore training objectives helps in increasing the probability that the participants will be successful in training

Designer ndash The training objective is beneficial to the training designer because if the designer is aware what is to be achieved in the end then hersquoll buy the training package according to that only The training designer would then look for the training methods training equipments and training content accordingly to achieve those objectives Furthermore planning always helps in dealing effectively in an unexpected situation Consider an example the objective of one training program is to deal effectively with customers to increase the sales Since the objective is known the designer will design a training program that will include ways to improve the interpersonal skills such as verbal and non verbal language dealing in unexpected situation ie when there is a defect in a product or when a customer is angry Therefore without any guidance the training may not be designed appropriately

Evaluator ndash It becomes easy for the training evaluator to measure the progress of the trainees because the objectives define the expected performance of trainees Training objective is an important to tool to judge the performance of participants

3 Training Delivery

Delivery methodology considerations occur after we have defined the objectives assessments content types and learning sequence for the training Depending on objectives and the content types some or even all of training can be done outside of the classroom Or you may find that a mix of training methods can be used to minimize time out of the job and still ensure that the training is effective The assumption would be that the production operators are able to understand English being the language used for delivery Alternatively they can request for a trainer who can speak in Malay language

Blended training combines various types of training and information communication to achieve the most effective performance gains in the most efficient manner Blended training solution components typically include a combination of the following

Web-based training (WBT) Multimedia-based training (MBT) Other self-paced instruction Online help Distance-learning Reference documentation Instructor-led classroom-based training

Blended training solutions

Provide the skills knowledge and support when and where theyre needed

Take advantage of many learning methods

Reinforce knowledge and sustain performance over a longer period of time than

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 8: Finance in Human Resource

traditional training solutions

Below are the individual elements that may comprise a blended training solution

Possible Elements for a Blended Training Solution

Web-Based Training (WBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training available on the Internet

Employees need to access just in time and just enough training at the time of the need

Employees may be working on varying platforms (Windows Macintosh and Unix)

Allows easy access anytime and virtually anywhere

Allows simple update to content

May use a variety of multimedia effects to draw the user in

May be linked to resources outside of the course

Requires computer and Internet access

Requires self-motivation to complete the training

Multimedia-based Training (MBT)

Definition ExamplesUses Pros Cons

Self-paced interactive training presented on a CD-ROM using a variety of multimedia (eg audio and video)

Employees are geographically dispersed or otherwise unable to attend scheduled training

Employees may be unable to access the Internet

Allows easy access to training on a desktop or laptop

Does not need access to the Web

Is not able to take advantage of the power of the Web

Online Help

Definition ExamplesUses Pros Cons

Quick and immediate access to information about a specific task delivered to a user at the users request

Users need quick access to information or a quick refresher to get the job done

Users need a quick cue tip or prompt when they roll the mouse over a screen area

Users need an online tutorial that can be attached to the application

Allows user to get help and keep working

Allows limited detail

Distance Learning

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 9: Finance in Human Resource

Definition ExamplesUses Pros Cons

An instructor-led approach where the instructor and participant are separated by place or time

An instructor posts lessons and exercises and participants work independently yet have regular online chats with the instructor

Employees are scattered geographically

Schedules prevent employees from attending face-to-face training

Avoids costly travel for geographically scattered employees

Avoids the need to be physically in a classroom

Provides some interaction between the instructor and participants

Allows limited interaction with other participants and the instructor

Reference Documentation

Definition ExamplesUses Pros Cons

Factual or procedural information that supports a person doing a particular job after initial learning has occurred

Information includes job aids charts posters user manuals and reference guides

Helps sustain learning

Serves as ongoing reference

Allows users easy access to structured information

No interaction

Face-to-Face Training

Definition ExamplesUses Pros Cons

An interactive instructor-led approach where the instructor and employee meet in a classroom for a specific duration of time in a common location

Participants benefit from practice and feedback

Subject matter requires a classroom or laboratory situation

Allows participant and instructor to carry on detailed conversations about unclear points

Requires participants to travel to a certain location during a particular time period

As for this manufacturing company we have to ensure that the operators are able to attend face to face training which would solve the problems encountered in the production line

The training should be planned phase by phase to ensure that production is not interrupted

4 Training Evaluation

A vital aspect of any sort of evaluation is its effect on the person being evaluated Feedback

is essential for people to know how they are progressing and also evaluation is crucial to

the learners confidence too And since peoples commitment to learning relies so heavily on

confidence and a belief that the learning is achievable the way that tests and assessments

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 10: Finance in Human Resource

are designed and managed and results presented back to the learners is a very important

part of the learning and development process

People can be switched off the whole idea of learning and development very quickly if

they receive only negative critical test results and feedback Always look for positives in

negative results Encourage and support - dont criticize without adding some positives and

certainly never focus on failure or thats just what you will produce

This is a much overlooked factor in all sorts of evaluation and testing and since this element

is not typically included within evaluation and assessment tools the point is emphasised

point loud and clear here

Evaluation is not just for the trainer or teacher or organisation or policy-makers -

evaluation is absolutely vital for the learner too which is perhaps the most important

reason of all for evaluating people properly fairly and with as much encouragement as the

situation allows

Most of the specific content and tools below for workplace training evaluation is based on the

work of Leslie Rae an expert and author on the evaluation of learning and training

programmes and this contribution is greatly appreciated W Leslie Rae has written over 30

books on training and the evaluation of learning - he is an expert in his field His guide to the

effective evaluation of training and learning training courses and learning programmes is a

useful set of rules and techniques for all trainers and HR professionals

This training evaluation guide is augmented by an excellent set of follow up tools created by

Leslie Rae which includes

bull Evaluation of Learning Questionnaire (LQ) - ready to use

bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide

bull Action Plan template and instruction - ready to use

bull Four separate and different

The four levels of Kirkpatricks evaluation model essentially measure

reaction of student - what they thought and felt about the training

learning - the resulting increase in knowledge or capability

behaviour - extent of behaviour and capability improvement and implementationapplication

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 11: Finance in Human Resource

results - the effects on the business or environment resulting from the trainees performance

Kirkpatricks four levels of training evaluation in detail

evaluation level and type

evaluation description and characteristics

examples of evaluation tools and methods

relevance and practicability

1 Reaction

Reaction evaluation is how the delegates felt and their personal reactions to the training or learning experience for example

Did the trainees like and enjoy the training

Did they consider the training relevant

Was it a good use of their time

Did they like the venue the style timing domestics etc

Level of participation

Ease and comfort of experience

Level of effort required to make the most of the learning

Perceived practicability and potential for applying the learning

Typically happy sheets

Feedback forms based on subjective personal reaction to the training experience

Verbal reaction which can be noted and analysed

Post-training surveys or questionnaires

Online evaluation or grading by delegates

Subsequent verbal or written reports given by delegates to managers back at their jobs

Can be done immediately the training ends

Very easy to obtain reaction feedback

Feedback is not expensive to gather or to analyse for groups

Important to know that people were not upset or disappointed

Important that people give a positive impression when relating their experience to others who might be deciding whether to experience same

2 Learning

Learning evaluation is the measurement of the increase in knowledge or intellectual capability from before

Typically assessments or tests before and after the training

Interview or observation

Relatively simple to set up but more investment and thought required than

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 12: Finance in Human Resource

to after the learning experience

Did the trainees learn what what intended to be taught

Did the trainee experience what was intended for them to experience

What is the extent of advancement or change in the trainees after the training in the direction or area that was intended

can be used before and after although this is time-consuming and can be inconsistent

Methods of assessment need to be closely related to the aims of the learning

Measurement and analysis is possible and easy on a group scale

Reliable clear scoring and measurements need to be established so as to limit the risk of inconsistent assessment

Hard-copy electronic online or interview style assessments are all possible

reaction evaluation

Highly relevant and clear-cut for certain training such as quantifiable or technical skills

Less easy for more complex learning such as attitudinal development which is famously difficult to assess

Cost escalates if systems are poorly designed which increases work required to measure and analyse

3 Behaviour

Behaviour evaluation is the extent to which the trainees applied the learning and changed their behaviour and this can be immediately and several months after the training depending on the situation

Did the trainees put their learning into effect when back on the job

Were the relevant skills and knowledge used

Was there noticeable and measurable change in the activity and performance of the trainees when back in their roles

Was the change in behaviour and new level of knowledge sustained

Would the trainee be

Observation and interview over time are required to assess change relevance of change and sustainability of change

Arbitrary snapshot assessments are not reliable because people change in different ways at different times

Assessments need to be subtle and ongoing and then transferred to a suitable analysis tool

Assessments need to be designed to reduce subjective judgement of the observer or interviewer which is a variable factor that can affect reliability and consistency of measurements

The opinion of the trainee which is a relevant indicator is also subjective

Measurement of behaviour change is less easy to quantify and interpret than reaction and learning evaluation

Simple quick response systems unlikely to be adequate

Cooperation and skill of observers typically line-managers are important factors and difficult to control

Management and analysis of ongoing subtle assessments are difficult and virtually impossible without a well-designed system from the beginning

Evaluation of implementation and application is an extremely important

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 13: Finance in Human Resource

able to transfer their learning to another person

Is the trainee aware of their change in behaviour knowledge skill level

and unreliable and so needs to be measured in a consistent defined way

360-degree feedback is useful method and need not be used before training because respondents can make a judgement as to change after training and this can be analysed for groups of respondents and trainees

Assessments can be designed around relevant performance scenarios and specific key performance indicators or criteria

Online and electronic assessments are more difficult to incorporate - assessments tend to be more successful when integrated within existing management and coaching protocols

Self-assessment can be useful using carefully designed criteria and measurements

assessment - there is little point in a good reaction and good increase in capability if nothing changes back in the job therefore evaluation in this area is vital albeit challenging

Behaviour change evaluation is possible given good support and involvement from line managers or trainees so it is helpful to involve them from the start and to identify benefits for them which links to the level 4 evaluation below

4 Results Results evaluation is the effect on the business or environment resulting from the improved performance of the trainee - it is the acid test

Measures would typically be business or organisational key performance indicators such as

Volumes values percentages timescales return on investment and other

It is possible that many of these measures are already in place via normal management systems and reporting

The challenge is to identify which and how relate to to the trainees input and influence

Therefore it is important to identify and agree accountability and relevance with the trainee at the start of the training so they understand what is to be measured

Individually results evaluation is not particularly difficult across an entire organisation it becomes very much more challenging not least because of the reliance on line-management and the frequency and scale of changing structures responsibilities and roles which complicates the process of attributing clear accountability

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 14: Finance in Human Resource

quantifiable aspects of organisational performance for instance numbers of complaints staff turnover attrition failures wastage non-compliance quality ratings achievement of standards and accreditations growth retention etc

This process overlays normal good management practice - it simply needs linking to the training input

Failure to link to training input type and timing will greatly reduce the ease by which results can be attributed to the training

For senior people particularly annual appraisals and ongoing agreement of key business objectives are integral to measuring business results derived from training

Also external factors greatly affect organisational and business performance which cloud the true cause of good or poor results

For us to obtain a correct result we will assume that the productions operators are able to

read and write and comment truthfully on how the training was conducted They must be

convinced that their output is important for management to improve the working conditions

(b) Return on Investment for the training conducted

HR and training managers have to continuously justify the amounts of money that were spent on training Training is often seen as a waste of time money and effort The value added through training is questioned Assumptions are made regarding the value of training added in achieving overall organisational effectiveness How do we know the value of training if we do not measure the value of training ROI in training is about measurement Measuring the return on the money invested in training ROI is about accountability and responsibility for the particular investment in training

The ultimate aim of any training program is to improve organisational performance that will add to organisational effectiveness and profitability In order to measure performance a person needs to determine the monetary value of the performance in its current status After the training intervention has taken place the learners performance needs to be measured again thus determining if there was an improvement Training is not a once-off event but a continuous process in achieving organisational effectiveness Employees need to receive training on an ongoing basis to be able to apply the learning acquired to their daily activities The information acquired to reinforce learning will be converted to knowledge application once the specific task is performed more efficiently and effectively Only then can the ROI be calculated through a cost-benefit analysis by determining the cost (investment in training) versus the benefit of the learning that has taken place ie the benefit of learning as a result of training

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 15: Finance in Human Resource

There is resistance to measure the ROI in training and the following are some of the questions asked in the industry

How can I measure ROI to determine the value added to the organisation What are the steps to measure ROI How often should ROI be measured In which areas of learning and change in behaviour can ROI be measured How do I implement the ROI process in the organisation How do I attach a rand value to learning What are the benefits of measuring ROI How do managers overcome resistance to measuring ROI

The measuring of return on investment originated in the USA where the American Society for Training and Development (ASTD) has been championing the process for decades The ground work was done by Donald Kirkpatrick who designed an evaluation system that addresses the evaluation at four levels reaction learning behaviour and results Reaction evaluation refers to the typical evaluation forms we complete after a training course the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course

The second level refers to learning evaluation in other words the extent to which the learners have learned certain concepts principles knowledge and skills This type of evaluation is normally done by means of tests because you can assess the knowledge level of a learner

The third level of evaluation is called behaviour evaluation and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills Here we ask ourselves the questions Is there a change in behaviour Can we measure the application of skills in the workplace

The last level according to Kirkpatrick is results evaluation This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation For example has the quality of the production increased as a result of the training programme Is there an increase in productivity as a result of a training programme What is evident from all these questions is that there must be some form of pre- and post course measurement in order to be able to answer these questions

Jack Phillips a training management consultant is now considered to be the main expert in the field of ROI in America He visited South Africa in 2002 to share his ideas about ROI with local training and human resource managers

Training managers know that ROI must be measured yet very few are actually doing it The following are some of the reasons for not calculating ROI that have been provided by training managers during recent conferences organised by ASTD Global Network South Africa There is a lack of a measurement culture in training departments Managers do not know where to start or they dont have the resources to calculate the ROI It is perceived as too difficult complicated too much effort or time consuming They are so busy with all their training programmes and HRDF requirements that there is no time to calculate ROI They fear that if ROI is calculated it will show that their training adds no value and that they will be at risk to lose their jobs

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 16: Finance in Human Resource

The King Report on Corporate Governance states clearly that accountability must also be reported in the area of human capital management Training is an essential component of human capital management

The good news is that ROI can be determined through a scientific method ROI is the measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme In other words it is the extent to which the benefits (outputs) of training exceed the costs (inputs)The cost of training is the money invested in the training programme The benefits of the training programme would be the measurable output ie the ability to answer more telephone calls or manufacture more nuts and bolts

ROI is a key financial metric of the value of training investments and costs It is a ratio of net benefits to costs expressed as a percentage The formula can be expressed as

[(monetary benefits ndash cost of the training) cost of the training] x 100

If the ROI from a training programme is calculated at 143 it means that for every Ringgit spent there has been a return of RM143 in net benefit after all costs are factored in The exact form of that benefit depends on the objectives of the learning programme ie better quality and increased production skills after training The total cost of the employee to the organisation is calculated including floor space basic wages overtime rental etc

Below are the costs that were incurred to establish a training program for the company to improve production quantity and quality

Duration of training 33 hrs

Estimated employee numbers 100Period over which benefits are calculated 12 months

Costs

Design and development RM5600

Printing RM4744

Administration RM8000

Floor Space RM25000

Materials RM5000

Trainer Fees RM10500

Salary RM20000

Evaluation RM872

Total cost RM79716

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide
Page 17: Finance in Human Resource

Benefits

Labour savings RM45000

Productivity increases RM85000

Other cost savings RM65250

Other income generation RM0

Total benefits RM195250

Return on investment 145

Payback period 14 months

Hence the ROI is calculated as per below

(RM1077321 ndash RM79716 RM79716) X 100

ROI on Training = 145

The above is calculated on the assumption that

1 All the 100 new employees attend the training

2 All the 100 new employees are able to understand and apply the training methods

3 The employees are motivated to attend the training

4 That the management is willing to spend RM79716 on training to achieve the desired benefits

It is important to follow proper guidelines for the effective application of the ROI measurement process by adhering to the following steps

Create awareness for ROI in the organisation Build capacity for ROI by training staff to understand ROI Quantify information before the training in order to obtain a baseline (eg number of

units produced number of defect units etc) Covert this data to monetary value (eg the cost of a unit to produce) Allocate resources for ROI Develop a culture of measurement and accountability among training staff Start with only one course as a pilot programme to practise ROI skills Communicate results to training staff and the whole organisation Design improvement plans for training programmes in order to increase ROI Once ROI results are available use the data to market future learning programmes

Measuring ROI is not a one-person show It is a powerful tool that enables training managers to prove the value of training gaining credibility for the value added and a contribution to achieving organisational effectiveness It enables them to report to management in quantifiable terms ie Ringgit and sen Measuring ROI is about accountability and taking responsibility for the measuring the impact as a result of the training Measurement is about becoming a strategic business partner that add value and provide integrated business solutions

  • 3 Training Delivery
  • bull Evaluation of Learning Questionnaire (LQ) - ready to use
  • bull Evaluation of Key Objectives Learning Questionnaire (KOLQ) - guide