Evaluating HRD Programs - हे Buddy Web viewEvaluating HRD Programs. Evaluating HRD Programs. Page 5. Page 24. HUMAN RESOURCE DEVELOPMENT. EVALUATING HRD PROGRAMS. ... Philips

Evaluating HRD Programs

HUMAN RESOURCE DEVELOPMENT

EVALUATING HRD PROGRAMS

XAVIER INSTITUTE OF SOCIAL SERVICERANCHI

Submitted by: Submitted to:

Group: VI Prof. Sajeet LakraPersonnel Management-II2010-2012

Page 1


GROUP CONSTITUTION

Page 2

Members Roll No.

Ashutosh Sharma 06Aditi Gupta 15Kumar Pratik 28Divyendu Shekhar 38Utkarsh 52Shashank Shekhar Rai 66


ACKNOWLEDGEMENTSuccess of a project depends on the endeavor put behind it. Encouragement and patronization perfect an endeavor.We take this opportunity to express our heartfelt gratitude to our faculty Prof. Sajeet Lakra, for providing us with the opportunity of preparing the study material for the of the HRD curriculum and give presentation in the class. The group’s involvement and team work helped us understand the chapter in a lucid manner. The group’s cohesiveness and team work helped us to make this task a success. We thank all the team members and also all our colleagues for their unconditional support.Thank YouGroup VI

Page 3


CONTENTSS.No. Topics Pg. No.

1. Introduction 052. Purpose of HRD Evaluation 063. Principles of Evaluation 074. Importance of Training Evaluation 085. Benefits of Evaluation 106. Process of Training Evaluation 117. Models and Framework of Evaluation 158. Data Collection for HRD Evaluation 219. Linking Training and development

with Organization’s Business Strategy

22

10. Case Study 2611. Conclusion 2812. Bibliography 29

Page 4


1.INTRODUCTION TO HUMAN RESOURCE DEVELOPMENT EVALUATION PROCESS

A comprehensive and effective evaluation plan is a critical component of any successful training program. It should be structured to generate information of the impact of training on the reaction; on the amount of learning that has taken place; on the trainees behaviour; and its contribution to the job/ organization. Therefore evaluation of training is a measure of how well it has meet the need of its human resources.

Any attempt to obtain information (feedback) on the effect of training program and to assess the value of training in the light of that information for improving further training.”- Hamblin“The systematic collection of descriptive and judgmental information necessary to make effective training decisions related to the selection, adoption, value, and modification of various instructional activities.”“Any attempt to obtain information (feedback) on the effect of training program and to assess the value of training in the light of that information for improving further training.”

Page 5


Assessing the effectiveness of the training program in terms of the benefits to the trainees and the company must look into the following aspects:– process of collecting outcomes to determine if the training program was effective– from whom, what, when, and how information should be collected

2. PURPOSE OF EVALUATION

• Determining accomplishment of specific training objectives.

• Determining cost effectiveness of training. • Explain program failure, if it occurs.• Enhancement of the credibility of training and

development if tangible benefits to an organization are proved.

• Determine whether the program is meeting the intended objectives

• Identify strengths and weaknesses• Determine cost-benefit ratio• Identify who benefited most or least

Page 6


• Determine future participants• Provide information for improving HRD programs• Reinforce major points to be made• Gather marketing information• Determine if training program is appropriate• Establish management database

3. PRINCIPLES OF EVALUATION

• Clarity about the goals and purposes of evaluation.• Evaluation must be continuous.• Evaluation must be specific.• Evaluation must be based on objective methods

and standards.• Realistic target dates must be set for each phase

of the evaluation process.• Evaluation has to be accepted as a process than an

end product of training.

Page 7


4. IMPORTANCE OF TRAINING EVALUATION

An organization that invests in training programs for employees should also conduct some form of training evaluation. This can be done in a number of different ways. The purpose of evaluation is to determine whether or not the training achieved the desired objective. The objective might have been very precise, or extremely loose, but there would have been one. This is usually set as a result of assessing trainining needs. Evaluating training will help show how effective it was at meeting the objective. All organizations, regardless of size or type, should use some process of assessing the effectiveness of training undertaken by employees. Education in any form is an investment of time and money, which makes it important that the

Page 8


organization establishes, as far as it can, the return on that investment.

Evaluating training effectiveness is important because it sheds light on four aspects. They include:

How well the training program met the learner’s needs and objectives

What knowledge and skills it has imparted to learners

What desirable change it has brought in the learners’ performance

What organizational benefits it has yieldedHowever, unfortunately, in today’s organizations, this mechanism of training evaluation is in short supply. Organizations are unwilling to spend their resources for a comprehensive after-training evaluation. They end it by serving a ceremonious feedback form to learners and getting their opinions. But you must realize that it will not work. It will not make your training programs effective and yield the results you want.In this absence of a proactive follow-up, the following undesirable consequences will have to be faced:Lack of application of skills by learners:

You had analyzed the performance gaps in your employees and devised a suitable training program. At the end of the training program, your participants said that the training program was quite useful and they had learnt many useful skills. You are happy about that. But at the end of your annual auditing, you are horrified to find no progress in the performance in spite of the training. And your employees have not used the skills you intend them to after the training. This is quite

Page 9


possible. You can avert it by putting in place a proper evaluation mechanism.Inability to measure ROI:

Financial assessment of a training program is crucial in that it provides useful insights into the Returns on Investment (ROI). This assessment depends on evaluating how well the employees implemented the skills they have learnt in the training program.Inability to make future training programs contextual:

In today’s dynamic marketplace, the need for consistent skill up gradation never loses its importance. What was said to be an effective training program in the past may not be so now. You need to continuously adapt it to suit your current needs. This is possible when you know how effective your previous training program was.There are methods like Kirkpatrick’s Training Evaluation Model, Bloom’s Taxonomy of Learning Domains and Keller’s ARCS Model. Use the models suggested here depending on the available resources, size of the training program and your corporate culture. When you have complex and critical training programs, you need to choose elaborate tools and have a wider approach to evaluate each program at various stages. Only then can the evaluation become effective and accurate.

Page 10


5. BENEFITS OF TRAINING EVALUATION

Improved quality of training activities. Helps employees to monitor their own

improvement. Builds morale, by demonstrating an interest in staff

development. To maximize the training ROI (return on

investment). Helps to determine the form of future training

programs. Assists with identifying the effectiveness of

different forms of teaching (such as classroom based or web based).

Improved ability of the trainers to relate inputs to outputs

Better discrimination of training activities between those that are worthy of support and those that should be dropped

Better integration of training offered and on-the job development

Better co-operation between trainers and line-managers in the development of staff

Evidence of the contribution that training and development are making to the organization

Page 11


Reduction in preventable accidents measured Reduction in scrap/rework measured in cost of

labor and materials

6. PROCESS OF TRAINING EVALUATION

There are two considerations involved in evaluation:• Who is responsible for the validation and

evaluation processes?• What resources of time, people and money are

available for validation/evaluation purposes? Thus the process of evaluation involves 3 stages:• Before training• During training• After training

BEFORE TRAINING: The learner's skills and knowledge are assessed before the training program. During the start of training, candidates generally perceive it as a waste of resources because at most of the times candidates are unaware of the objectives and learning outcomes of the program. Once aware, they are asked to give their opinions on the methods used and whether those

Page 12


methods conform to the candidate’s preferences and learning style. DURING TRAINING: It is the phase at which instruction is started. This phase usually consists of short tests at regular intervals.AFTER TRAINING:

It is the phase when learner’s skills and knowledge are assessed again to measure the effectiveness of the training. This phase is designed to determine whether training has had the desired effect at individual department and organizational levels. There are various evaluation techniques for this phase.

The Training Evaluation should involve: senior management the trainer line management the training manager the trainee

SENIOR MANAGEMENT:

• Awareness of need & value of training • Involving Training Manager in senior management

meetings• Knowledge & support of training plans

Page 13


• Active participation in events• Evaluation to be done & regular summary reports• Policy & strategic decisions based on results &

return on investment data.THE TRAINER:• Provision of any necessary pre-program work • Identifying knowledge & skills level• Training & learning resources to enable the

learners to learn within objectives of program• Monitoring the learning as the program progresses• Assessment of & receipt of reports.

THE LINE MANAGER:• Work-needs & people identification• Involvement in training programme & evaluation

development• Pre-event preparation & holding briefing meetings• Giving ongoing & practical support to the training

programme• Reviewing the progress of learning implementation• Final review of implementation success and

assessment.

THE TRAINING MANAGER:Page 14


• Management of the training department & agreeing to the training needs

• Maintenance of interest & support in the planning & implementation of the programs including practical involvement where required

• The introduction & maintenance of evaluation systems

• Production of regular reports for senior management

• Contact with senior management• Liaison with the learners' line managers and

arrangement of learning implementation.

THE TRAINEE:• Plan & design the training programme• Contribute towards the evaluation process• Take interest & performing an active part in the

training programme or activity• Complete a personal action plan during & at the

end of the training• Support the evaluation processes.

Page 15


7. MODELS AND FRAMEWORK OF EVALUATION

• There are six frameworks for evaluation• Kirkpatrick• CIPP• Brinkerhoff• Kraiger, Ford & Salas

Page 16


• Holton• Philips

Two approaches are mainly used: Goal-based Approach-

The Kirkpatrick Model System-based Approach:

(i)CIPP Model (ii) TVS Approach (iii) IPO Approach

Kirkpatrick (1959) CIPP Model (1987) IPO Model (1990) TVS Model (1994)

1. Reaction: to gather data on participants reactions at the end of a training program

1. Context: obtaining information about the situation to decide on educational needs and to establish program objectives

1. Input: evaluation of system performance indicators such as trainee qualifications, availability of materials, appropriateness of training, etc.

1. Situation: collecting pre-training data to ascertain current levels of performance within the organization and defining a desirable

Page 17


level of future performance

2. Learning: to assess whether the learning objectives for the program are met

2. Input: identifying educational strategies most likely to achieve the desired result

2. Process: embraces planning, design, development, and delivery of training programs

2. Intervention: identifying the reason for the existence of the gap between the present and desirable performance to find out if training is the solution to the problem

3. Behavior: to assess whether job performance changes as a result of training

3. Process: assessing the implementation of the educational program

3. Output: Gathering data resulting from the training interventions

3. Impact: evaluating the difference between the pre- and post-training data

4. Results: to assess costs vs. benefits of training programs, i.e., organizational impact in terms of reduced costs, improved quality of work, increased quantity of work, etc.

4. Product: gathering information regarding the results of the educational intervention to interpret its worth and merit

4. Outcomes: longer-term results associated with improvement in the corporation’s bottom line- its profitability, competitiveness, etc.

4. Value: measuring differences in quality, productivity, service, or sales, all of which can be expressed in terms of dollars

GOAL-BASED SYSTEM-BASED

These models do not define the steps CIPP, IPO, and TVS) seem to be more

Page 18


necessary to achieve purposes and do not address the ways of utilizing results to improve training.

useful in terms of thinking about the overall context and situation but they may not provide sufficient granularity.

The difficulty for practitioners following such models is in selecting and implementing appropriate evaluation methods (quantitative, qualitative, or mixed).

Systems-based models may not represent the dynamic interactions between the design and the evaluation

of training.

Because of their apparent simplicity, trainers jump feet first into using such models without taking the time to assess their needs and resources or to determine how they’ll

apply the model.

Few of these models provide detailed descriptions of the processes involved in each steps. None provide tools for evaluation.

Many organizations do not use the entire model, and training ends up being evaluated only at the reaction, or at best, at the

learning level.

These models do not address the collaborative process of evaluation, that is, the different roles and responsibilities that people may play during an evaluation process

THE KIRKPATRICK MODEL:

Page 19


Level 1 : ReactionReaction evaluation is how the delegates felt about the training or learning experience. Evaluation tools and methods • 'Happy sheets', feedback forms.

Page 20


• Verbal reaction, post-training surveys or questionnaires

Relevance and practicability • Quick and very easy to obtain. • Not expensive to gather or to analyse

Level 2 : Learning Learning evaluation is the measurement of the increase in knowledge before and after.Evaluation tools and methods • Typically assessments or tests before and after the

training• Interview or observation can also be used

Relevance and practicability • Relatively simple to set up; clear-cut for

quantifiable skills• Less easy for complex learning

Level 3 : Behavior Behaviour evaluation is the extent of applied learning back on the job - implementation.Evaluation tools and methods

Page 21


• Observation and interview over time are required to assess change, relevance of change, and sustainability of change

Relevance and practicability • Measurement of behaviour change typically

requires cooperation and skill of line-managersLevel 4 : Results Results evaluation is the effect on the business or environment by the trainee.Evaluation tools and methods • Measures are already in place via normal

management systems and reporting - the challenge is to relate to the trainee

Relevance and practicability • Individually not difficult; unlike whole organisation

Process must attribute clear accountabilities

Page 22


8. DATA COLLECTION FOR HRD EVALUATIONThere are several techniques of data collection. They are as follows:

Observation Questionnaire Interview Written Test Simulation/ Performance test

Advantages and Disadvantages of Techniques :

METHODS ADVANTAGES DISADVANTAGES

1.Interview • Flexible• Opportunity

for clarification• Depth Possible• Personal

Contact

• High Reactive effects

• High cost• Face-to-Face

Threat Potential

• Trained Observers Needed

•

2.Questionnaire

• Low Cost• Anonymity

Possible• Respondent

sets pace

• Possibly Inaccurate Data

• Return Rate Beyond

Page 23


• Variety of Options

Control

Observation • Non- Threatening

• Excellent way to Measure Behaviour Change

• Reactive Effect

• Probable• Trained

Observers Needed

Written Test • Low Purchase Cost

• Readily Scored and Quickly Processed

• Easily Administered

• Wide Sampling Possible

• Possible low Relation to Job Performance

• Reliance on Norms may Distort Individual Performance

• Cultural BiasSimulation/ Performance Test

• Reliable• Objective• Close Relation

to Job Performance

• Time consuming

• Simulation often Difficult and Costly

Page 24


9. LINKING TRAINING AND DEVELOPMENT WITH BUSINESS STRATEGY

• Training and Development strategy is a comprehensive master plan which ensures that investment in learning is assessed and evaluated.

• Business Strategy is the determination of the long term goals and objectives of an organization, and allocation of resources necessary for carrying out these goals.

• Training functions lie at the heart of how businesses access and develop human resources required to maintain a sustainable competitive advantage over rivals.

• Training should result in creating skills that are appropriate for achieving the strategic objectives of a business.

The Factors governing the starting point for formulating an effective training and development strategy are as follows:

Page 25


• Understand the industry and the business’ unique offer and what gives it competitive advantage.

• Consider the changes predicted in the environment- the rate of growth/decline, the competition and degree of technological change.

• Assess the need to change and adapt to economic circumstances.

• Assess the level of knowledge and professional capability of staff now and that needed in the future.

• Understand the importance of customer service and reflect the community served- what sort of customers are served and the nature of their expectations.

Training and development strategy should also align with the cultural beliefs about what learning may achieve and the way it is regarded. Assumptions include: • All work is a process of continual learning and

improvement. This can be the starting point for creating a ‘learning organization’.

• Learning should be encouraged only where there is measurable benefit to the organization.

• Learning should be targeted on key performance skills.

• Learning may cover other aims than just a narrow definition of business benefit- it can support the

Page 26


organization’s duty to the community or wider social responsibility.

• Learning is a cost to the business and should be delivered effectively and efficiently.

• Learning and talent development is an investment for the future.

• Personal career development is critical for retention and engagement and should ne encouraged.

According to Rothwell & Kazanas the ten Approaches to link T&D strategy with Business Strategy are:

1. The Top Down Approach: The Organizational strategy determines and dictates the HRD offerings.

2. The Market Driven Approach: The HRD function or department identifies future learning on the basis of future market conditions confronting the organization.

3. The Career Planning Approach: The HRD effort helps individuals prepare for the future against the backdrop of organization’s strategic plan.

4. The Futuring Approach: The HRD function formally and directly assists top managers as they formulate the organizational strategy.

5. The Artificial Experience Approach: The HRD function simulates the conditions the organization may face in the future.

6. The Pulse Taking Approach: Since HRD function is at the forefront of communicating, assessing needs, design and develop instructions and

Page 27


evaluate results-they maintain a high profile. This unique position in organization’s communication chain is utilized to the best advantage in this approach. HRD people take pulse of organization regarding implementation of business strategy.

7. The Performance Diagnosis Approach: While assessing the training needs, HRD professionals often uncover performance problems or opportunities of strategic importance. They relay that information to the top management for their use in strategic planning.

8. The Education Approach: HRD practitioners try to link HRD with the organizational strategy by teaching people how to think strategically as they do their work. By applying strategic planning model people can anticipate the future problems and opportunities.

9. The Inter Personal Approach: It is informal and may involve socializing with strategists to know what they are seeking.

10. The Rifle Approach: It takes its name from its laser like focus on achieving pinpointed results. The HRD practitioners aim their efforts to the areas of greatest need or the business’ most pressing problem.

Page 28


10. TRAINING EVALUATION IN BRITANNIA

For each training programme conducted in Britannia, a training brochure is developed.The brochure developed consists of the following information: • Programme objectives: Need of the training and

what are the objectives that this training aims to achieve, what likely outcomes are expected to come out of impact of this training.

• Programme content: Topics being covered during the training.

• Methodology adopted. • Programme faculty. • Personnel to be covered. • Training methodology. • Training effectiveness criteria and scale. • The training effectiveness is measured by

measurement of the achievement of the objectives.

• A person gets nominated for a training programme which flows from the training needs of that individual

Page 29


• The programs are divided into three broad categories:

Functional Behavioural General/ Omnibus programmes

Functional programs:The outcome of the training is measured by comparing the data pre-training and post-training. A scale is developed for measuring the effectiveness of training based on the % achievement of the objectives.

Behavioural programs:The effectiveness of the training of this nature is measured annually. This is seen through the training need identification for the coming year for the employee. If the training need is repeated there, then the training provided is taken as ineffective. If the training need is repeated but with focus on a part of the need, then the training is partially effective. If not repeated, then the training is effective.

General programs:These are the training needs flowing directly from the organizational needs. Examples of these can be ISO 9000 training, ISO 14000 training and any awareness

Page 30


training. These are omnibus training programmes, which are run for a large number of employees.

The effectiveness of the training is measured by: Achievement of those organizational objectives

within the time lines. Number of audit issues raised on the areas covered

in the training. Any other such thing as defined in the training

brochure

11. CONCLUSIONEvaluation of HRD Program helps in gauging the degree to which a training (or other HRD program) achieves its intended purpose.In other words it measures the effectiveness of the HRD programThe importance of training programs must be understood by HRD professionals. Not every program needs to be evaluated to the same extent. The key is to have a well planned evaluation strategy that sets the stage for how and to what extent each program will be evaluated.

The value of the training evaluation forms is that they give immediate feedback and they are inexpensive to set up and process. However, it is possible for someone

Page 31


to leave a course having enjoyed it, and believing they learned a great deal. It is only when they come to apply this knowledge that they discover whether the training was as effective as they thought.A more thorough evaluation can involve a subsequent round of training evaluation forms, or an assessment interview, which are conducted a few days or weeks after the training. This allows the learner time to reflect on what they have learned and to have put at least some of it into practice. Their opinion of the training may have changed based on their experience in the workplace.Methods for the effective evaluation of training have been studied for some time. The Kirkpatrick model is now considered to be the industry standard, and is frequently used by HR departments when determining their own systems.Organizations that train but do not evaluate that training cannot be certain of its value, either to themselves or their employees.

12. BIBLIOGRAPHY

• Human Resource Development, Jon M. Werner & Randy L. Desimone, Pg. 151-201,Eighth Edition, Cengage Learning India Pvt. Ltd.

• Human Resource Development- Wikipedia, The Free encyclopedia http://en.wikipedia.org/wiki/Human_resource_development

Page 32


[Accessed on 27th November, 2011]

• http://managementhelp.org/trng_dev/evaluate/evaluate.htm


• http://www.ifets.info/journals/5_2/eseryel.html


Page 33

Documents

Evaluating HRD Programs - हे Buddy Web viewEvaluating HRD Programs. Evaluating HRD Programs. Page 5. Page 24. HUMAN RESOURCE DEVELOPMENT. EVALUATING HRD PROGRAMS. ... Philips