11
Return on Investment
(ROI) in TrainingProgramme developed and delivered by Dr Belinda Ketel Faranani Facilitation Services
22
33
Activity: “Hello” to group
Your link to ROI / training?
44
Activity:Expectations for this workshop?
Programme
08h00 – 08h30 Registration
08h30 – 09h00 Fasset Welcome, Briefing and Introduction
09h00 – 10h30 Understanding ROI
10h30 – 10h50 Tea Break
10h50 – 12h45 Working with ROI
12h45 – 13h30 Lunch break
13h30 – 15h00 Application of ROI in practice
15h00 – 15h20 Comfort break
15h20 – 16h00 Q&A on ROI
16h00 – 16h15 Fasset closure
Lets map this out?
About this seminar
The seminar aims to challenge Skills
Development Facilitators (SDFs), Human
Resource Professionals and Managers
with understanding and applying ROI to
their training interventions in order to
make business sense
"Genius is one percent inspiration and ninety-nine
percent perspiration““Thomas EdisonThomas Edison
Understanding ROI
Activity Based Results Based
Trends in training evaluation
• no effort to prepare the work environment to support transfer
• environment prepared to support transfer
• no efforts to build partnerships with key managers
• partnerships established with key managers and clients
• no measurement of results or cost benefit analysis
• measurement of results and cost benefit analysis
• planning and reporting on training is input focused
• planning and reporting on training is output focused
Trends in training evaluation
Activity Based
• no business need for the program
• no assessment of performance issues
• no specific measurable objectives
• no effort to prepare program participants to achieve results
Results Based
• program linked to specific business needs
• assessment of performance effectiveness
• specific objectives for behavior and business impact
• results expectations communicated to participants
The “catch” in the Measurement Cycle::
What countsgets measured
What getsmeasuredgets done
What getsdone getsrewarded
What gets
rewardedcounts
So WHERE is ROI?So WHERE is ROI?How do you see it?How do you see it?
• Keep the process simple
• Use sampling for ROI calculations
• Always account for the influence of other factors
• Involve management in the process
• Educate the management team
• Communicate results carefully
• Give credit to participants and managers
• Plan for ROI calculations
A Rational Approach to ROI
The Journey to Increased Accountability
Time
HRD / Performance Improvement
Accountability Level 1 (Reaction)
Level 2 (Learning)
Level 3 (Application)
Level 4 (Business Impact)
Level 5 (ROI)
Profit Center
Establishing ROI• Showing added value of ROI must be a top priority
• Information management should align its goals with organizational strategy
• Communication of ROI is key
• ROI is a short list of added values expressed in:
ROI of what?
Calculating ROI: What are the costs?
ROI explained fromROI explained froman HR perspectivean HR perspective
Canadian Society for T&DCanadian Society for T&D
ROI of Training - keeping it simple
ISOLATE EFFECTS OF SOLUTION
CALCULATE THE RETURN ON
INVESTMENT
CAPTURECOSTS OF
SOLUTION
IDENTIFY INTANGIBLE
MEASURES
DEVELOP EVALUATION PLANS AND
BASELINE DATA
GENERATE IMPACT STUDY
COLLECT DATADURING
SOLUTION IMPLEMENTATION
DATA COLLLECTION EVALUATION PLANNING DATA ANALYSIS REPORTING
INTANGIBLE BENEFITS
5. ROI
DEVELOPOBJECTIVESOF SOLUTION
COLLECT DATA AFTER SOLUTION IMPLEMENTATION
3. APPLICATION/ IMPLEMENTATION4. BUSINESS IMPACT
1. SATISFACTION/ REACTION2. LEARNING
EVALUATION FRAMEWORK
Level Measurement Focus
1. Reaction, Satisfaction & Measures participant satisfaction with Planned Action the program and captures planned actions.
2. Learning Measures changes in knowledge, skills,and attitudes.
3. Application Measures changes in on-the-job behavior.
4. Business Impact Measures changes in business impactvariables.
5. Return on Investment Compares program benefits to the costs.
Training Solutions HR Solutions
Technology Solutions Organization Development Solutions
Change Initiatives
THE ROI PROCESS
CONVERT DATA TO
MONETARYVALUE
Calculating the Return on Investment of Performance Solutions
ROI = = 1.61 x 100 = 161%
ROI Calculation - Example
Net Program Benefits ROI = Program Costs
R230,000 - R88,000
R88,000
Costs per program (38 participants) R88,000
Benefits per program (1st year) R230,000 from 38 participants
Evaluating Training Programs: 4 Levels
Results
Behavior
Learning
Reaction
4: Final Results
3: Change in B.V.
2: KSA
1 : Exp
Relationship Between Levels
• Each subsequent level is predicated upon doing evaluation at lower level
• A Level 3 will be of marginal use, if a Level 2 evaluation is not conducted
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - KnowledgeDid they learn anything
Level 3 - BehaviorKSA being used on the job?
Level 4 - ResultsWas it worth it?
Types of Assessments Used at Each Level
Level 1 - Reaction
Was the environment suitable for learning?
Level 2 - KnowledgeDid they learn anything
Level 3 - BehaviorKSA being used on the job?
Level 4 - ResultsWas it worth it?
Type Form
Summative Correlation of business results with other assessment results
Summative Observation of Performance
360° Survey
Diagnostic
SummativeSelf-assessment
Test
Reaction
Formative
Survey
Real-time Polling
Quizzing
Levels of Measurement
Competencies
Level 3: Application in Work Setting
Were the participants able to apply the knowledge and skills (solution) on the job?
Key Business
Results
Level 4: Business Impact
Did the applied knowledge and skills (solution) positively influence the organization’s business measures?
Learning and performance
solutionsLevel 1: Reaction/ Planned ActionsWere the participants satisfied with the program and what do they plan to do differently?
Level 2: LearningWhat knowledge,skills or attitudesimproved as a resultof the program?
Level 5: Return onInvestmentDid the benefitsexceed the financialinvestment?
Activity: Identify training eg’s that your organisation has invested in
Were these evaluated?
Applying ROI Methodology
Collect Data After Solution
Implementation
Collect Data During Solution Implementation
Develop Evaluation Plans and
Baseline Data
Develop Objectives of
Solution
DATA COLLECTIONEVALUATION PLANNING
1. Satisfaction Reaction
3. Application / Implementation
2. Learning 4. Business Impact
The ROI Process
The ROI Process
Isolate the Effects of the
Solution
Convert Data to Monetary
Values
Calculate the Return on Investment
Tabulate Costs
Identify Intangible Measures
GenerateImpactStudy
REPORTINGDATA ANALYSIS
5. ROI
Intangible Benefits
Provides 6 types of data
Results Chain
• Link between “problem” impact and training decision
• What should happen after the training?
• And then what…
• And then what….
• Etc…
Results Chain
Your training example
What would you expect / like to see happening because of this
training?
And then what?
And then what?
And then what?
And then what?And then what? And then what?
And then what?
And then what?And then what?
Example:Example:
Attend Fasset ROI Seminar
Increase in understanding &
knowledge on ROI
Application at work = skills
Sharing of new info & skills with
colleagues
Business more aware of potential impact of training
decisions
Business makes more strategic performance
related training decisions
Training interventions
better planned, resourced and
evaluated
Business more successful
Performance and productivity
improves
Added value to business bottom
line increases
Activity: Develop a Results Chain for
a training example from your organisation
Indicators for measurement
• Baseline• Input• Process• Output• Outcome• Impact• Composite
Example:Example:
Attend Fasset ROI Seminar
Increase in understanding &
knowledge on ROI
Application at work = skills
Sharing of new info & skills with
colleagues
Business more aware of potential impact of training
decisions
Business makes more strategic performance
related training decisions
Training interventions
better planned, resourced and
evaluated
Business more successful
Performance and productivity
improves
Added value to business bottom
line increases
Activity: Identify the levels of RESULTS for
your training example
Rules for designing indicators::
1. Indicators are always written as a number
2. Indicators are neutral (no change in direction)
3. Indicators must be very specific and unambiguous
4. Indicator can only measure one result at a time
Indicators for ROIIndicators for ROI
Example:Example:
Attend Fasset ROI Seminar
Increase in understanding &
knowledge on ROI
Application at work = skills
Sharing of new info & skills with
colleagues
Business more aware of potential impact of training
decisions
Business makes more strategic performance
related training decisions
Training interventions
better planned, resourced and
evaluated
Business more successful
Performance and productivity
improves
Added value to business bottom
line increases
Activity: Design indicators for the ROI
training example
Example indicators:Example indicators:
• Number of staff attending ROI session
• Number of staff passing knowledge assessment
• % increase in self-assessment (before & after training)
Increase in understanding &
knowledge on ROI
Example indicators:Example indicators:
• Number of ROI tools used• Number of staff using ROI tools
Application at work = skills
Example indicators:Example indicators:
• Number of staff sharing knowledge• Number of formal info sharing
sessions held• Number of staff able to work on ROI
that did not attend training session
Sharing of new info & skills with colleagues
Example indicators:Example indicators:
• R-value of training planned for period• R-value of training implemented for
period• R-value of training programmes
evaluated
Training interventions better planned,
resourced and evaluated
Example indicators:Example indicators:
• Number of strategic training related decisions made by business
• R-value of rewards provided to staff for performance improvement
• R-value of training linked directly to business objectives
Business makes more strategic performance
related training decisions
Example indicators:Example indicators:
• Number of strategic business decisions impacted by ROI results
• R-value of training decisions made at strategic level
• R-value of resources allocated to ROI processes
Business more aware of potential impact of training decisions
Example indicators:Example indicators:
• R-value of ROI budget• No of strategic meetings where ROI results
discussed• No of strategic decisions influences by ROI
results• No of staff having ROI as part of their
responsibility
Added value to business bottom line increases
Example indicators:Example indicators:
• No of staff scoring above average on performance ratings in period
• No of staff receiving performance rewards
• R-value of performance rewards
• No of units produced / completed
• R-value of units produced / invoiced
• Turn around time on client requests
• R-value of time spent on client requests
• No of complaints received
• No of errors picked up
• R-value of errors corrected
Performance and productivity improves
Example indicators:Example indicators:
• % increase market share (R-value?)• % increase in client base (R-value?)• R-value of new accounts generated• R-value of turnover for period• R-value of new referrals (quotes)• No of current clients maintained (R-value)• No current client portfolios extended (R-value)
Business more successful
Data Collection and Analysis for ROI
Types of methods
Quantitative methods…
... Answer the question, “How many?”
... Are used when a number, rate, ratio or proportion is required
… Directly measure the status or change of a specific variable, for example changes in crop yield,
kilometres of road built over a period of time, or number of hours per week that women spend fetching water
Types of methods
Qualitative methods…...... Answer questions such as, “Why?” and “How?”
… Are used when the attitudes, beliefs, perceptions and experiences of the target population are
required to understand its reactions and responses to interventions
… Gather data by asking people to explain what they have observed, do, believe or feel
Types of data
Primary Data…
• Collection is necessary when a researcher cannot find the data needed in secondary sources
• Primary data can be qualitative or quantitative depending on what methodology was chosen
Common Primary Data Collection Methods for Monitoring
• Observations
• Interviews
• Groups- large and small
• Surveys
Types of dataSecondary Data…
• Refers to data that already exists and is available from national and local agencies, academic institutions or published in papers and books (e.g., reports, maps, photographs, diagrams)
Possible limitations of using secondary data
• Inadequacy
• Potential for poor quality
• Variation in concepts
• Data may be outdated
• Inaccessibility
The Four Major Categories of Hard Data
Primary Measurementsof Improvement
“Hard Data”
OutputIncreases
Quality
Improvement
Time
Savings
CostSavings
OUTPUT• Units Produced• Tons Manufactured• Items Assembled• Money Collected• Items Sold• Forms Processed• Loans Approved• Inventory Turnover• Patients Visited• Applications
Processed• Students Graduated• Tasks Completed• Output Per Period• Productivity• Work Backlog• Incentive Bonus• Shipments• New Accounts
Generated
TIME• Equipment Downtime• Overtime• On Time Shipments• Time to Project Completion• Processing Time• Supervisory Time• Break in Time for New Employees• Training Time• Meeting Schedules• Repair Time• Efficiency• Work Stoppages• Order Response• Late Reporting• Lost Time Days
Examples of Hard DataQUALITY• Scrap• Waste• Rejects• Error Rates• Rework• Shortages• Product Defects• Deviation From Standard• Product Failures• Inventory Adjustments• Time Card Corrections• Percent of Tasks Completed Properly• Number of Accidents
COSTS• Budget Variances• Unit Costs• Cost By Account• Variable Costs• Fixed Costs• Overhead Cost
• Operating Costs• Number of Cost Reductions• Project Cost Savings• Accident Costs• Program Costs• Sales Expense
Major Categories of Soft Data
Typical Measuresof Improvement
“Soft Data”
Initiative
Development/
Advancement
Work
Climate
Feelings/AttitudesNew
Skills
Work Habits
NEW SKILLS• Decisions Made• Problems Solved• Conflicts Avoided• Grievances Resolved• Counseling Problems Solved• Listening Skills• Interviewing Skills• Reading Speed• Discrimination Charges Resolved• Intention to Use New Skills• Frequency of Use of New Skills
DEVELOPMENT/ADVANCEMENT• Number of Promotions• Number of Pay Increases• Number of Training Programs Attended• Requests for Transfer• Performance Appraisal Ratings• Increases in Job Effectiveness
INITIATIVE• Implementation of New Ideas• Successful Completion of Projects• Number of Suggestions Submitted• Number of Suggestions Implemented• Work Accomplishment• Setting Goals and Objectives
Examples of Soft Data
WORK CLIMATE• Number of Grievances• Number of Discrimination Charges• Employee Complaints• Job Satisfaction• Employee Turnover• Litigation
FEELINGS/ATTITUDES• Favorable Reactions• Attitude Changes• Perceptions of Job Responsibilities• Perceived Changes in Performance• Employee Loyalty• Increased Confidence
WORK HABITS• Absenteeism• Tardiness• Visits to the Dispensary• First Aid Treatments• Violations of Safety Rules• Number of Communication Break-downs• Excessive Breaks• Follow-up
Activity: Identify the data sources for your
own training example
Example:Example:
Attend Fasset ROI Seminar
Increase in understanding &
knowledge on ROI
Application at work = skills
Sharing of new info & skills with
colleagues
Business more aware of potential impact of training
decisions
Business makes more strategic performance
related training decisions
Training interventions
better planned, resourced and
evaluated
Business more successful
Performance and productivity
improves
Added value to business bottom
line increases
ROI Calculations
ROI Calculation Steps:
1. Create an ROI measurement plan
2. Collect data
3. Isolate the effects of training
4. Covert data to monetary value
5. Calculate ROI
ROI Calculation: Example
Benefits per program (1st year) R230,000 from 25 participants
Costs per program (25 participants) R88,000
Net Program Benefits ROI = Program Costs
-ROI = = x 100 = %
R-values to work with: R-values to work with: BenefitsBenefits
• R-value of training planned for period (R 200 000)
• R-value of training implemented for period (R 120 000)
• R-value of training programmes evaluated (R 80 000)
• R-value of rewards provided to staff for performance (R 350 000)
• R-value of training linked directly to business objectives (R 80 000)
• R-value of training decisions made at strategic level (R 80 000)
• R-value of resources allocated to ROI processes (R 25 000)
• R-value of ROI budget (R 25 000)
• R-value of units invoiced (R 2 000 000)
• R-value of time spent on client requests (R 800 000)
• R-value of errors corrected (R 550 000)
• R-value of new accounts generated (R 800 000)
R-values to work with: R-values to work with: CostsCosts
• R-value of payments to training providers (R 120 000)
• R-value of accom/travel costs for training (R120 000)
• R-value of training venue/catering (R 60 000)
• R-value of staff man hours while on training (R 200 000)
• R-value of hours could be billed to client (R 600 000)
• R-value of errors occurring while staff on training (R 60 000)
ROI Calculation: Formula
Net Program Benefits ROI = Program Costs
-ROI = = x 100 = %
Activity: Calculate the ROI of the ROI
training example
Activity: What does your ROI result tell you?
(What does the business need to know & do)
ROI Challenges
ROI Dilemmas
• If the CEO/top executive requires ROI, it may be too late for the process
• The time to pursue ROI is when you do not have to pursue it
• The ROI process is not a quick fix
• If training staff do not see the need for ROI, it will usually fail
• Without cooperation of participants, the ROI process will usually fail
• Without the support of management, the ROI process will usually fail
• The ROI process requires much education with a variety of groups
Implementation Concerns
• Resources (staffing / budget)
• Leadership (individual, group, cross functional team)
• Timing (urgency, activities)
• Communication (various audiences)
• Commitment (staff, managers, top executives)
ROI Lessons from practice
Tips for HR Managers
• Understand that business needs will change over the duration of the project. The longer it takes to do a project, the more likely that change will occur. Be prepared to switch gears or re-tool. Stay alert for indications of learning gaps or obsolescence of course materials created by changes in technology, economic factors, or employee demographics
• Anticipate and plan for glitches. They will happen. Do not get discouraged. Deal realistically with unexpected events and surprises. Some of them will be problems that may reduce expected outcomes. Others may stimulate creativity and produce very positive results
Tips for HR Managers
• Keep an eye on costs and monitor any deviations from estimated or budgeted expenditures. Quick action or adjustments may enable the HR manager to curtail or reduce costs and keep the project within or near budget
• Balance short term and long-term goals. The pressure to deliver consistent short-term financial performance may limit the resources available for investment in growth opportunities, such as the "development" of employees for future roles
• Use the experience of senior managers and employees in estimating the results of training. Estimating requires a little math and lots of experience
• Secure managers' and supervisors' support for training. Effective training depends on three persons: the trainer, the trainee and the supervisor or manager. All three must agree on expected outcomes and how and when they will be measured
• Align training initiatives with strategic business plans. Training should improve your organization's competitive edge, which may be efficiency, first class service, or creativity and innovation
• Track training costs by employee for career pathing and succession planning
Tips for HR Managers
Final Advice
Using ROI as the “bottom line” may not be the best
criteria for determining the value of training.
It can be a good indicator,
but it may not be the indicator
What Next?Action Items To Implement A Comprehensive ROI Process
• Assess your progress and
readiness.
• Develop a policy and
philosophy statement on results-
based approach.
• Clarify roles of:
• Staff
• Employees
• Supervision and management
• Establish accountability
• Develop a transition plan
• Set targets for evaluation
levels.
• Develop guidelines
• Build staff skills
• Establish a management
support system
• Enhance management
commitment and support.
• Communicate results to
selective audiences
• Establish a quality review
process.
Useful website:Useful website:www.roiinstitute.netwww.roiinstitute.net
ROI: Is it complex or is it simple?
Thank you ….
Thank YouFasset Call Centre
086 101 0001www.fasset.org.za
Thank you ….
Head Office (Cape Town): 206 Rosmead Ave, Wynberg, 7800, PO Box. 457, Plumstead 7801
Gauteng: Postnet Suite 469, Private Bag X51, Bryanston.
Ph: (021) 762 - 5742 Fax: (021) 762 – 9745 Email: [email protected]
http://www.farananiservices.co.za
(PTY) LTD 2007/019026/07 Accreditation No: Services Seta: 0866 Directors: Benedict Pillay & Deborah Williams