1 Return on Investment (ROI) in Training Programme developed and delivered by Dr Belinda Ketel Faranani Facilitation Services

11

Return on Investment

(ROI) in TrainingProgramme developed and delivered by Dr Belinda Ketel Faranani Facilitation Services

22

33

Activity: “Hello” to group

Your link to ROI / training?

44

Activity:Expectations for this workshop?

Programme

08h00 – 08h30 Registration

08h30 – 09h00 Fasset Welcome, Briefing and Introduction

09h00 – 10h30 Understanding ROI

10h30 – 10h50 Tea Break

10h50 – 12h45 Working with ROI

12h45 – 13h30 Lunch break

13h30 – 15h00 Application of ROI in practice

15h00 – 15h20 Comfort break

15h20 – 16h00 Q&A on ROI

16h00 – 16h15 Fasset closure

Lets map this out?

About this seminar

The seminar aims to challenge Skills

Development Facilitators (SDFs), Human

Resource Professionals and Managers

with understanding and applying ROI to

their training interventions in order to

make business sense

"Genius is one percent inspiration and ninety-nine

percent perspiration““Thomas EdisonThomas Edison

Understanding ROI

Activity Based Results Based

Trends in training evaluation

• no effort to prepare the work environment to support transfer

• environment prepared to support transfer

• no efforts to build partnerships with key managers

• partnerships established with key managers and clients

• no measurement of results or cost benefit analysis

• measurement of results and cost benefit analysis

• planning and reporting on training is input focused

• planning and reporting on training is output focused

Trends in training evaluation

Activity Based

• no business need for the program

• no assessment of performance issues

• no specific measurable objectives

• no effort to prepare program participants to achieve results

Results Based

• program linked to specific business needs

• assessment of performance effectiveness

• specific objectives for behavior and business impact

• results expectations communicated to participants

The “catch” in the Measurement Cycle::

What countsgets measured

What getsmeasuredgets done

What getsdone getsrewarded

What gets

rewardedcounts

So WHERE is ROI?So WHERE is ROI?How do you see it?How do you see it?

• Keep the process simple

• Use sampling for ROI calculations

• Always account for the influence of other factors

• Involve management in the process

• Educate the management team

• Communicate results carefully

• Give credit to participants and managers

• Plan for ROI calculations

A Rational Approach to ROI

The Journey to Increased Accountability

Time

HRD / Performance Improvement

Accountability Level 1 (Reaction)

Level 2 (Learning)

Level 3 (Application)

Level 4 (Business Impact)

Level 5 (ROI)

Profit Center

Establishing ROI• Showing added value of ROI must be a top priority

• Information management should align its goals with organizational strategy

• Communication of ROI is key

• ROI is a short list of added values expressed in:

ROI of what?

Calculating ROI: What are the costs?

ROI explained fromROI explained froman HR perspectivean HR perspective

Canadian Society for T&DCanadian Society for T&D

ROI of Training - keeping it simple

ISOLATE EFFECTS OF SOLUTION

CALCULATE THE RETURN ON

INVESTMENT

CAPTURECOSTS OF

SOLUTION

IDENTIFY INTANGIBLE

MEASURES

DEVELOP EVALUATION PLANS AND

BASELINE DATA

GENERATE IMPACT STUDY

COLLECT DATADURING

SOLUTION IMPLEMENTATION

DATA COLLLECTION EVALUATION PLANNING DATA ANALYSIS REPORTING

INTANGIBLE BENEFITS

5. ROI

DEVELOPOBJECTIVESOF SOLUTION

COLLECT DATA AFTER SOLUTION IMPLEMENTATION

3. APPLICATION/ IMPLEMENTATION4. BUSINESS IMPACT

1. SATISFACTION/ REACTION2. LEARNING

EVALUATION FRAMEWORK

Level Measurement Focus

1. Reaction, Satisfaction & Measures participant satisfaction with Planned Action the program and captures planned actions.

2. Learning Measures changes in knowledge, skills,and attitudes.

3. Application Measures changes in on-the-job behavior.

4. Business Impact Measures changes in business impactvariables.

5. Return on Investment Compares program benefits to the costs.

Training Solutions HR Solutions

Technology Solutions Organization Development Solutions

Change Initiatives

THE ROI PROCESS

CONVERT DATA TO

MONETARYVALUE

Calculating the Return on Investment of Performance Solutions

ROI = = 1.61 x 100 = 161%

ROI Calculation - Example

Net Program Benefits ROI = Program Costs

R230,000 - R88,000

R88,000

Costs per program (38 participants) R88,000

Benefits per program (1st year) R230,000 from 38 participants

Evaluating Training Programs: 4 Levels

Results

Behavior

Learning

Reaction

4: Final Results

3: Change in B.V.

2: KSA

1 : Exp

Relationship Between Levels

• Each subsequent level is predicated upon doing evaluation at lower level

• A Level 3 will be of marginal use, if a Level 2 evaluation is not conducted

Level 1 - Reaction

Was the environment suitable for learning?

Level 2 - KnowledgeDid they learn anything

Level 3 - BehaviorKSA being used on the job?

Level 4 - ResultsWas it worth it?

Types of Assessments Used at Each Level

Level 1 - Reaction

Was the environment suitable for learning?

Level 2 - KnowledgeDid they learn anything

Level 3 - BehaviorKSA being used on the job?

Level 4 - ResultsWas it worth it?

Type Form

Summative Correlation of business results with other assessment results

Summative Observation of Performance

360° Survey

Diagnostic

SummativeSelf-assessment

Test

Reaction

Formative

Survey

Real-time Polling

Quizzing

Levels of Measurement

Competencies

Level 3: Application in Work Setting

Were the participants able to apply the knowledge and skills (solution) on the job?

Key Business

Results

Level 4: Business Impact

Did the applied knowledge and skills (solution) positively influence the organization’s business measures?

Learning and performance

solutionsLevel 1: Reaction/ Planned ActionsWere the participants satisfied with the program and what do they plan to do differently?

Level 2: LearningWhat knowledge,skills or attitudesimproved as a resultof the program?

Level 5: Return onInvestmentDid the benefitsexceed the financialinvestment?

Activity: Identify training eg’s that your organisation has invested in

Were these evaluated?

Applying ROI Methodology

Collect Data After Solution

Implementation

Collect Data During Solution Implementation

Develop Evaluation Plans and

Baseline Data

Develop Objectives of

Solution

DATA COLLECTIONEVALUATION PLANNING

1. Satisfaction Reaction

3. Application / Implementation

2. Learning 4. Business Impact

The ROI Process

The ROI Process

Isolate the Effects of the

Solution

Convert Data to Monetary

Values

Calculate the Return on Investment

Tabulate Costs

Identify Intangible Measures

GenerateImpactStudy

REPORTINGDATA ANALYSIS

5. ROI

Intangible Benefits

Provides 6 types of data

Results Chain

• Link between “problem” impact and training decision

• What should happen after the training?

• And then what…

• And then what….

• Etc…

Results Chain

Your training example

What would you expect / like to see happening because of this

training?

And then what?

And then what?

And then what?

And then what?And then what? And then what?

And then what?

And then what?And then what?

Example:Example:

Attend Fasset ROI Seminar

Increase in understanding &

knowledge on ROI

Application at work = skills

Sharing of new info & skills with

colleagues

Business more aware of potential impact of training

decisions

Business makes more strategic performance

related training decisions

Training interventions

better planned, resourced and

evaluated

Business more successful

Performance and productivity

improves

Added value to business bottom

line increases

Activity: Develop a Results Chain for

a training example from your organisation

Indicators for measurement

• Baseline• Input• Process• Output• Outcome• Impact• Composite

Example:Example:



knowledge on ROI



colleagues


decisions





evaluated



improves


line increases

Activity: Identify the levels of RESULTS for

your training example

Rules for designing indicators::

1. Indicators are always written as a number

2. Indicators are neutral (no change in direction)

3. Indicators must be very specific and unambiguous

4. Indicator can only measure one result at a time

Indicators for ROIIndicators for ROI

Example:Example:



knowledge on ROI



colleagues


decisions





evaluated



improves


line increases

Activity: Design indicators for the ROI

training example

Example indicators:Example indicators:

• Number of staff attending ROI session

• Number of staff passing knowledge assessment

• % increase in self-assessment (before & after training)


knowledge on ROI


• Number of ROI tools used• Number of staff using ROI tools



• Number of staff sharing knowledge• Number of formal info sharing

sessions held• Number of staff able to work on ROI

that did not attend training session

Sharing of new info & skills with colleagues


• R-value of training planned for period• R-value of training implemented for

period• R-value of training programmes

evaluated

Training interventions better planned,

resourced and evaluated


• Number of strategic training related decisions made by business

• R-value of rewards provided to staff for performance improvement

• R-value of training linked directly to business objectives




• Number of strategic business decisions impacted by ROI results

• R-value of training decisions made at strategic level

• R-value of resources allocated to ROI processes

Business more aware of potential impact of training decisions


• R-value of ROI budget• No of strategic meetings where ROI results

discussed• No of strategic decisions influences by ROI

results• No of staff having ROI as part of their

responsibility

Added value to business bottom line increases


• No of staff scoring above average on performance ratings in period

• No of staff receiving performance rewards

• R-value of performance rewards

• No of units produced / completed

• R-value of units produced / invoiced

• Turn around time on client requests

• R-value of time spent on client requests

• No of complaints received

• No of errors picked up

• R-value of errors corrected

Performance and productivity improves


• % increase market share (R-value?)• % increase in client base (R-value?)• R-value of new accounts generated• R-value of turnover for period• R-value of new referrals (quotes)• No of current clients maintained (R-value)• No current client portfolios extended (R-value)


Data Collection and Analysis for ROI

Types of methods

Quantitative methods…

... Answer the question, “How many?”

... Are used when a number, rate, ratio or proportion is required

… Directly measure the status or change of a specific variable, for example changes in crop yield,

kilometres of road built over a period of time, or number of hours per week that women spend fetching water

Types of methods

Qualitative methods…...... Answer questions such as, “Why?” and “How?”

… Are used when the attitudes, beliefs, perceptions and experiences of the target population are

required to understand its reactions and responses to interventions

… Gather data by asking people to explain what they have observed, do, believe or feel

Types of data

Primary Data…

• Collection is necessary when a researcher cannot find the data needed in secondary sources

• Primary data can be qualitative or quantitative depending on what methodology was chosen

Common Primary Data Collection Methods for Monitoring

• Observations

• Interviews

• Groups- large and small

• Surveys

Types of dataSecondary Data…

• Refers to data that already exists and is available from national and local agencies, academic institutions or published in papers and books (e.g., reports, maps, photographs, diagrams)

Possible limitations of using secondary data

• Inadequacy

• Potential for poor quality

• Variation in concepts

• Data may be outdated

• Inaccessibility

The Four Major Categories of Hard Data

Primary Measurementsof Improvement

“Hard Data”

OutputIncreases

Quality

Improvement

Time

Savings

CostSavings

OUTPUT• Units Produced• Tons Manufactured• Items Assembled• Money Collected• Items Sold• Forms Processed• Loans Approved• Inventory Turnover• Patients Visited• Applications

Processed• Students Graduated• Tasks Completed• Output Per Period• Productivity• Work Backlog• Incentive Bonus• Shipments• New Accounts

Generated

TIME• Equipment Downtime• Overtime• On Time Shipments• Time to Project Completion• Processing Time• Supervisory Time• Break in Time for New Employees• Training Time• Meeting Schedules• Repair Time• Efficiency• Work Stoppages• Order Response• Late Reporting• Lost Time Days

Examples of Hard DataQUALITY• Scrap• Waste• Rejects• Error Rates• Rework• Shortages• Product Defects• Deviation From Standard• Product Failures• Inventory Adjustments• Time Card Corrections• Percent of Tasks Completed Properly• Number of Accidents

COSTS• Budget Variances• Unit Costs• Cost By Account• Variable Costs• Fixed Costs• Overhead Cost

• Operating Costs• Number of Cost Reductions• Project Cost Savings• Accident Costs• Program Costs• Sales Expense

Major Categories of Soft Data

Typical Measuresof Improvement

“Soft Data”

Initiative

Development/

Advancement

Work

Climate

Feelings/AttitudesNew

Skills

Work Habits

NEW SKILLS• Decisions Made• Problems Solved• Conflicts Avoided• Grievances Resolved• Counseling Problems Solved• Listening Skills• Interviewing Skills• Reading Speed• Discrimination Charges Resolved• Intention to Use New Skills• Frequency of Use of New Skills

DEVELOPMENT/ADVANCEMENT• Number of Promotions• Number of Pay Increases• Number of Training Programs Attended• Requests for Transfer• Performance Appraisal Ratings• Increases in Job Effectiveness

INITIATIVE• Implementation of New Ideas• Successful Completion of Projects• Number of Suggestions Submitted• Number of Suggestions Implemented• Work Accomplishment• Setting Goals and Objectives

Examples of Soft Data

WORK CLIMATE• Number of Grievances• Number of Discrimination Charges• Employee Complaints• Job Satisfaction• Employee Turnover• Litigation

FEELINGS/ATTITUDES• Favorable Reactions• Attitude Changes• Perceptions of Job Responsibilities• Perceived Changes in Performance• Employee Loyalty• Increased Confidence

WORK HABITS• Absenteeism• Tardiness• Visits to the Dispensary• First Aid Treatments• Violations of Safety Rules• Number of Communication Break-downs• Excessive Breaks• Follow-up

Activity: Identify the data sources for your

own training example

Example:Example:



knowledge on ROI



colleagues


decisions





evaluated



improves


line increases

ROI Calculations

ROI Calculation Steps:

1. Create an ROI measurement plan

2. Collect data

3. Isolate the effects of training

4. Covert data to monetary value

5. Calculate ROI

ROI Calculation: Example

Benefits per program (1st year) R230,000 from 25 participants

Costs per program (25 participants) R88,000


-ROI = = x 100 = %

R-values to work with: R-values to work with: BenefitsBenefits

• R-value of training planned for period (R 200 000)

• R-value of training implemented for period (R 120 000)

• R-value of training programmes evaluated (R 80 000)

• R-value of rewards provided to staff for performance (R 350 000)

• R-value of training linked directly to business objectives (R 80 000)

• R-value of training decisions made at strategic level (R 80 000)

• R-value of resources allocated to ROI processes (R 25 000)

• R-value of ROI budget (R 25 000)

• R-value of units invoiced (R 2 000 000)

• R-value of time spent on client requests (R 800 000)

• R-value of errors corrected (R 550 000)

• R-value of new accounts generated (R 800 000)

R-values to work with: R-values to work with: CostsCosts

• R-value of payments to training providers (R 120 000)

• R-value of accom/travel costs for training (R120 000)

• R-value of training venue/catering (R 60 000)

• R-value of staff man hours while on training (R 200 000)

• R-value of hours could be billed to client (R 600 000)

• R-value of errors occurring while staff on training (R 60 000)

ROI Calculation: Formula


-ROI = = x 100 = %

Activity: Calculate the ROI of the ROI

training example

Activity: What does your ROI result tell you?

(What does the business need to know & do)

ROI Challenges

ROI Dilemmas

• If the CEO/top executive requires ROI, it may be too late for the process

• The time to pursue ROI is when you do not have to pursue it

• The ROI process is not a quick fix

• If training staff do not see the need for ROI, it will usually fail

• Without cooperation of participants, the ROI process will usually fail

• Without the support of management, the ROI process will usually fail

• The ROI process requires much education with a variety of groups

Implementation Concerns

• Resources (staffing / budget)

• Leadership (individual, group, cross functional team)

• Timing (urgency, activities)

• Communication (various audiences)

• Commitment (staff, managers, top executives)

ROI Lessons from practice

Tips for HR Managers

• Understand that business needs will change over the duration of the project. The longer it takes to do a project, the more likely that change will occur. Be prepared to switch gears or re-tool. Stay alert for indications of learning gaps or obsolescence of course materials created by changes in technology, economic factors, or employee demographics

• Anticipate and plan for glitches. They will happen. Do not get discouraged. Deal realistically with unexpected events and surprises. Some of them will be problems that may reduce expected outcomes. Others may stimulate creativity and produce very positive results


• Keep an eye on costs and monitor any deviations from estimated or budgeted expenditures. Quick action or adjustments may enable the HR manager to curtail or reduce costs and keep the project within or near budget

• Balance short term and long-term goals. The pressure to deliver consistent short-term financial performance may limit the resources available for investment in growth opportunities, such as the "development" of employees for future roles

• Use the experience of senior managers and employees in estimating the results of training. Estimating requires a little math and lots of experience

• Secure managers' and supervisors' support for training. Effective training depends on three persons: the trainer, the trainee and the supervisor or manager. All three must agree on expected outcomes and how and when they will be measured

• Align training initiatives with strategic business plans. Training should improve your organization's competitive edge, which may be efficiency, first class service, or creativity and innovation

• Track training costs by employee for career pathing and succession planning


Final Advice

Using ROI as the “bottom line” may not be the best

criteria for determining the value of training.

It can be a good indicator,

but it may not be the indicator

What Next?Action Items To Implement A Comprehensive ROI Process

• Assess your progress and

readiness.

• Develop a policy and

philosophy statement on results-

based approach.

• Clarify roles of:

• Staff

• Employees

• Supervision and management

• Establish accountability

• Develop a transition plan

• Set targets for evaluation

levels.

• Develop guidelines

• Build staff skills

• Establish a management

support system

• Enhance management

commitment and support.

• Communicate results to

selective audiences

• Establish a quality review

process.

Useful website:Useful website:www.roiinstitute.netwww.roiinstitute.net

ROI: Is it complex or is it simple?

Thank you ….

Thank YouFasset Call Centre

086 101 0001www.fasset.org.za

Thank you ….

Head Office (Cape Town): 206 Rosmead Ave, Wynberg, 7800, PO Box. 457, Plumstead 7801

Gauteng: Postnet Suite 469, Private Bag X51, Bryanston.

Ph: (021) 762 - 5742 Fax: (021) 762 – 9745 Email: [email protected]

http://www.farananiservices.co.za

(PTY) LTD 2007/019026/07 Accreditation No: Services Seta: 0866 Directors: Benedict Pillay & Deborah Williams

Documents

1 Return on Investment (ROI) in Training Programme developed and delivered by Dr Belinda Ketel Faranani Facilitation Services