Case StudyHow to Create a Training Measurement Strategy

Join audio:• Choose “Computer audio” to use VoIP• Choose “Phone call” and dial using the

information provided

Questions/Comments:• Submit questions and comments via the

Questions panel.• Please download the materials from the

materials pane before leaving today’ssession.

• Please continue to submit your textquestions and comments using theQuestions Pane

Note: Today’s presentation is being recorded and will be posted at www.mimeo.com/webinars.

Your Participation

Jessica CoburnDirector of Training

Social Solutions

Ann McDonaldDirector of Talent Development

Mimeo

1. Overview of measurement strategies

2. Background on Social Solutions

3. Deep dive in Jessica’s training methods

4. Key takeaways and tips for you

Models for Measuring Training73.7

32.6

14.7 13.7

8.4

26.3

0

10

20

30

40

50

60

70

80

Kirkpatrick's 4Levels of Evaluation

Phillips' ROIMethodology

Six Sigma Brinkerhoff'sSuccess Case

Method

Bersin's TrainingEvaluation Method

Other

From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.24

Kirkpatrick Model

Level 1: Reaction

Level 2: Learning

Level 3: Behavior

Level 4: Results

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

How effective is yourtraining?

How effective is yourstrategy to measure

training?

Training at

User training:

• Customer training

• Employee training

• In-person workshops

• E-learning

• Microlearning

What to Measure

• Consider your audience• Determine your objectives and

ideal results• Then figure out how to measure

that

Reaction

What We Do: Clicktool Survey

Why It’s Important: High Correlation between learning and CSAT scores

Tip: If you’re not collecting this type of feedback, start now!

Learning

What We Do: Grading matrix for course assignments

Why This is Important: “Learning” must take place to ensure behavior change

Tip: Use pretest/post test if you don’t want to assume a zero baseline start

Behavior

What We Do: Pre-test and post-test before, immediately after, and 6 months after training

Why This is Important: It allows you to see a change in behavior

Tip: Consider your training objectives when you develop a strategy for measuring behavior

ResultsWhat We Do: Correlation metrics to show training impact on business

Why Is It Important: This is what it’s all about! Show your value to your organization

Tip: Much of this data exists elsewhere in your organization. You just need to find it.

Kirkpatrick Model

Clicktools survey

Grading matrix

Pre and post tests

Correlation metrics

Level 1: Reaction

Level 2: Learning

Level 3: Behavior

Level 4: Results

Lessons Learned

• Focus on metrics

• Get your trainers’ thoughts and opinions about your strategy and incorporate what is important to them

• Have a strategy to get the data you want

• Even if you don’t have the time or resources to analyze your data now, don’t let that stop you from getting started!

Additional Resources

• Learning Analytics: Measurement innovations to support employee development

by John R Mattox II and Mark Van Buren

published by CEB

• Spotlight Report on Training Measurement 2017

Published by Mimeo

Q&A

Next Webinar:

Take Your Training from In-Person to Online

Brian Washburn, CEO, Endurance Learning

October 182 pm ET

BRING YOUR CONTENT TO LIFE

AppendixSelected tables with more advice for working with learning

analytics

Efficiency Measures Effectiveness Measures Outcome Measures

# of people trained Satisfaction training Increase in customer satisfaction

# of people trained by learning methodology

Knowledge and skills gained due to training

Increase in employee performance

Reach (% of people trained in target population)

Intent to apply learning on the job Decrease in risk

Cost of training per program Expectation that training will improve individual performance on the job

Decrease in costs

Cost of training per learner Expectation that individual performance improvement will lead to organizational performance improvement

Increase in sales

Cost of training per hour Return on expectations Increase in revenue

From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.175

Relevant High quality Timely

Avoid analyzing data simply because of ease of collection and analysis

Provide oversight to data collection and analyses to ensure that insight is applicable to the problems it intends to address

Refresh data frequently, ideally whenever new training instances take place or when recurring processes occur

Gather data that has relevance to stakeholders beyond L&D function

Keep data collections methods consistent over business units, regions, and functions to ensure accurate comparisons at the functional level

Ensure that ad hoc data can be collected in a timely manner, so information supporting business decisions is updated

Make sure data collected can be cut for specific business units, functions, or regions so strengths/weaknesses can be pinpointed

Ensure that accurate thresholds are in place to notify management when data indicated declining performance

Keep KPIs/metrics and data collection methods consistent over time for accurate tracking of progress and improvement

From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.189

- Connect all data back to original desired business outcomes and organizational goals

- Outline how L&D and stakeholders share same priorities

- Do not assume stakeholders will make accurate assumptions about the data on their own

- Emphasize logical, clear connections between the data and desired business outcomes and organizational goals

- Focus on the central message and avoid including peripheral information

- Explicitly note areas where L&D is creating value for the organization

- Clearly and succinctly outline justification for key findings

- Suggest improvements and potential next steps based on data implications

- Establish productive environment for continuing conversation on L&D progress and improvement

From Learning Analytics: Measurement Innovations to Support Employee Development, Mattox and Van Buren, Kogan Page Limited, 2016, p.191

Set the Scene

Plot Development

Resolution