New e-Learning Measurements: The Challenges + Advantages Facing Your Business

New e-Learning Measurements:

The Challenges + Advantages Facing Your Business

&

About the presenter

2

Larry Israelite

SVP Assessment Solutions, Smarterer

Larry was born and raised on a small chicken

farm in Upper Black Eddy, PA. Since moving

to the big city, he has spent over 30 years

trying to answer a very simple question – how

can we improve business results through

learning? Larry has held senior learning and

talent management positions at several large

organizations, including Liberty Mutual

Insurance, Pitney Bowes and John Hancock.

He holds a PhD from Arizona State University

and has published two books, with a third,

More Lies About Learning, coming out in early

2015.

About Smarterer

• Smarterer helps you take control of your

professional destiny by quantifying your

skills, in as few 10 questions and 120

seconds.

• Pluralsight acquired Smarterer in 2014.

Together, we’re connecting the dots

between proving and improving your skills.

3

LEARNING

Individual Organization

About You

7

1 Maximizing learning

2 Measuring effectiveness

Ebbinghaus Forgetting Curve

9


10

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%

Elapsed Time Since Learning


11

58%

44%36%

33% 28% 25% 21%

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%



12

58%

44%36%

33% 28% 25% 21%

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%


How Long Does it Take?


13

58%

44%36%

33% 28% 25% 21%

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%



14

58%

44%36%

33% 28% 25% 21%

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%



15

58%

44%36%

33% 28% 25% 21%

Rete

ntion (

%)

0%

20%

40%

60%

80%

100%

100%


16

Spaced Repetition/Review

1 day 1 week 1 month

17



Learning

check

scores or

test results

18



Learning

check

scores or

test results

Work

tasks or

job

assignments

19



Learning

check

scores or

test results

Work

tasks or

job

assignments

Manager and/or

peer feedback

and personal

observations

20

Reflection

21

Training

Reflection

Learn

22

Training

Reflection

Learn

23

Reflect

Review

Training

Reflection

LearnDo

24

Reflect

Review

Learn

Training

Reflection

LearnDo

Be honest with yourself—ask the hard

questions

Do frequent, small after-action reviews

Find a friend

Be open to learning from feedback

25

26

27

1 Maximizing learning

2 Measuring effectiveness

About Your Organization

THEN

29

NOW

THEN

• Management control

30

NOW

THEN

• Management control • Learner control

31

NOW

THEN


• Few choices

• Learner control

32

NOW

THEN


• Few choices

• Learner control

• Many choices

33

NOW

THEN


• Few choices

• Simple oversight

• Learner control

• Many choices

34

NOW

THEN


• Few choices


• Learner control

• Many choices

• Complex oversight

35

NOW

THEN


• Few choices


• Small vendor community

• Learner control

• Many choices


36

NOW

THEN


• Few choices



• Learner control

• Many choices


• Large vendor community

37

NOW

THEN


• Few choices



• Employer owned data

• Learner control

• Many choices



38

NOW

THEN


• Few choices



• Employer owned data

• Learner control

• Many choices



• Employee owned data

39

NOW

The Kirkpatrick Model

41

Reaction

Behavior

Learning

44

Results

The Kirkpatrick Model

45

Did it work?Is it still

relevant?

How do we make other purchases?

Would Structure Help?

47

Learning Dashboard

Expectations Utility

Format Value Comments

• Classroom

• e-Learning

• Informal learning

• Other learning resources

48

Would it work for…

49

Complexity Too simple / too complex / about right

Would You Do It Yes / no / not sure

Would You Use Results Yes / no / not sure

Value High / medium / low

Measuring Learning Impact

• Is it an old world concept, centrally driven?

• In the new world, is it really about individual performance?

• Isn’t that what really matters?

50

Final Thoughts

51

Final Thoughts

52

New

Final Thoughts

53

NewCommunity-

based

Final Thoughts

54

New

Trust

Community-

based

Final Thoughts

55

New

Trust

Community-

based

Urgent

56

Smarterer helps people take control

of their professional destiny.

www.smarterer.com | [email protected] | @smarterer

&

Education

New e-Learning Measurements: The Challenges + Advantages Facing Your Business