What a CLO must know about Learning Analytics

What a CLO must know about Learning (Talent) AnalyticsPeter Howes

Vice President, Workforce Planning and Analytics

2 SuccessFactors Proprietary and Confidential © 2013 SuccessFactors, An SAP Company. All rights reserved.

Truth or Myth?

1. The best source of new hires is your current employees employee referrals?

2. High GPA Scores are a good predictor of successful graduate recruitment?

3. Employee equity awards of ANY size increase retention?

4. Industry experience is a KEY factor in successful salespeople?

Myth: Employees hired from referrals, turnover at twice the rate of the best new hire source

Myth: High GPA graduates had highest turnover between 2-4 years tenure.

Truth: Highly correlated at the company

Myth: In some positions studied, salespeople with little or no industry sales experience outperform seasoned people

Human Capital Analytics A Powerful Discovery Tool


Growth throughTalent Intelligence

Consumerizationof Technology

Changing Workforce

Demographics

48% of baby boomers expect to work past age of 651

80% of company valuations are intangibles related to talent such as IP, know-how, expertise, etc.7

90% of executives whose companies use social media technologies report measureable business benefits460% share of India,

other South Asian Nations and Africa in global labor force growth2

+1.3B workers will use mobile technologies in 20155

70% of US economic growth has been driven by knowledge workers over the past three decades6

1) Deloitte 2011, 2) McKinsey Global Institute 2012 3) US Department of Education 2013, 4) McKinsey Quarterly March 2013, 5) IDC’s Digital Universe Study, Sponsored by EMC December 2012, 6) McKinsey “Economic conditions Snapshot” 2010, 7) Deloitte “Extracting Maximum value from intangible assets”, 2010

59% of all 2013 master’s degrees will be earned by women3

Global Macro Trends Are Impacting the Workforce and Learning…


Compliance Risk

• Risk: In 2011, there were more than 1,000 regulatory changes affecting employee data worldwide– not counting industry regulations and legal updates.

• Risk: Do workers have the right certifications to do a job? And are they up to date with your industry, and company compliance standards?

Cost• Multiple Learning systems

• Move from on-premise

• Path to unified talent solution

…And Compliance and Cost Are Continuous Issues


Need to Overhaul the Economics of Learning

2.7%

$1,228

70%

$1,398

Learning spend per employee

Learning budget as a % of Payroll

Average cost per learning hour created

Learning delivery via ILT (live and online)

Source: ASTD, 2011 Study


Are Investments in Learning Paying Off?

“The relationship between learning and analytics is often misunderstood and under-utilized.

Typically, when people discuss analytics they are talking about reporting within a specific learning management system. They are talking about course completions, scores, pass rates and usage data.

Whilst useful, this information doesn’t really help you to make the connection with what is happening in your organization, or whether the training is having the desired effect of boosting productivity, improving service and raising levels of efficiency.”

Ed Cohen and Peter Howes, “Learning and Analytics”, SuccessFactors White Paper, 2012.

Research


82% 83% 85% 80% 95%

of business leaders don’t trust talent data

of business leaders don’t believe talent analytics are focused on the right issues

of HR leaders have not used information gleaned from data analytics to change a business decision

of senior HR leaders believe staff capabilities are a barrier to improving HR analytics

of senior HR leaders plan to increase investments in HR data and analytics in the next two years

Source: CEB | Talent Management Report in The Times, October 2013

The Opportunity Ahead


Business Results of aStrong Learning Culture

Organizations with a Strong Learning Culture Significantly Outperform Peers…

Innovation

46%more likely to

be first to market

Productivity

37%greater

employee productivity

Time to Market

34% better

response to customer

needs

Quality

26% greater ability

to deliver “quality

products”

Skills for the Future

58% more prepared to meet future

demand

Profitability

17% more likely to

be market share leader

SOURCE: Bersin & Associates


Yet…Many Organizations Are Flying Blind

“Only seven percent of respondents say they are very effective at using analytics to make workforce decisions.”

Without workforce insight, companies place their ability to grow effectively at risk.

They will have difficulty putting the right skills in the right places and will have trouble making intelligent decisions about locations suitable for expansion.

They will be challenged in knowing where their top performers are located and will struggle to justify the need for future workforce investments.”

Source: IBM, “Working Beyond Borders: Insights From the Global Chief Human Resource Officer Study,” 2010.

Missing an Opportunity to Drive Business Results?IBM Survey of 700 Chief Human Resource Officers


Learning World According to Bersin

Source: Josh Bersin, Bersin by Deloitte blog, “Today’s World-Class Chief Learning Officer,” May 2012

Chief Capability Officer

Chief Leadership Officer

Chief Talent Officer

Chief Change Officer

People Technology & Measurement Officer

World-Class

Chief Learning Officer


We’re not always strategic…

More than 65% of respondents view their HR function as a strategic partner to the organization while only 15% of its time is spent on strategic partnering activities.

27%

18%

15%

14%

14%

12%Delivering HR Services

Transacting/Recordkeeping

Strategic Partnering

Designing HR Programs or Systems

Internal Management

Compliance/Auditing

Mercer Consulting, 2010 HR Transformation Study

The Opportunity for Learning Analytics


A Measurement Maturity Model – From Learning Metrics and Analytics to Business Outcomes

Va

lue

/Im

pa

ct

Learning Metrics & Scorecards

Information – What is happening?

Learning AnalyticsInsight – Why is it

happening?

Learning OutcomesImpact –

How does it affect the business?

Time/Expertise Required

Measurement Maturity Model

Current practices in LMS


The Talent Analytics Journey…in more detail

Data Reporting Metrics Advanced Metrics

Bench-marking Analytics

Predictive Analytics

Stages

Foundations

Definitions:

inclusions

exclusions

Analysis Dimensions:

segmentation

trends

Aggregation as ratios & percentages

Aggregation & weighting to

generate index measures

External & internal

comparisons

Statistical analysis:

correlations

test of significance

analysis of variance

factor analysis

Hypothesis Testing:

structural equation modelling


What Questions Are We Looking to Answer?

Stage of Measurement Maturity

Common Learning Questions That Organizations Seek to Answer

Learning Metrics

What percentage of employees have an IDP assigned to them and what percentage of them complete the recommended steps on their IDP within each time period?

How many dollars are spent annually per FTE on learning & development? Do longer tenured employees continue to grow via training, or is training

disproportionately offered to, or attended by, newer employees? What percent of learning dollars are assigned to closing the gap between

needed and actual capabilities?

Learning Analytics

What is the percent increase in performance as a result of every $1,000 spent on training?

How does the performance of employees participating in optional training programs compare to those not participating?

What percentage of poor performers become good or better performers within a year as a result of specific learning programs?

Learning Outcomes

Is there a correlation between the percentage of all people costs spent on learning and firm profitability?

How have leadership development programs impacted productivity or employee satisfaction?

Source: Adapted from the SuccessFactors “100 Critical Human Capital Questions”.


More Questions…To How Many Questions Can Your CHRO or CLO Answer “Yes”?

1. Do you know the retention rate of your high performers & whether it is good or poor relative to industry benchmarks?

2. Can you run a headcount report now and segment the results by your primary organizational structure, workforce tenure, and gender?

3. Can you identify your best and worst performing managers, as measured by the performance and engagement of their teams?

4. Do you know the average performance score and the time to full productivity, for new hires?

5. How many people do you have in critical roles? What percentage are at risk and do you have good succession coverage?

6. Which hiring sources provide the most successful employees?

7. What is your average span of control and how does this vary across your company?

8. Do you know whether investment in Learning and Development in your organization correlates, positively or negatively, to increased performance and business results?

9. What is the average promotion speed for high and low performers?

10. What is the ratio of revenue generating to non-revenue generating employees?

Learning Analytic Overview…our view of the world


What you get with SuccessFactors…

Learning Analytics

Library of LMS

Metrics, Definitions

and Formulas

Learning Benchmarks

Platform

Online Report

Designer for ad hoc

and formatted reporting

LMSReports Dashboards

Workforce Analytics

Library of Metrics and Data Feeds for Core HR,

Talent Mgmt., Finance,

Surveys, etc., Benchmarks

Headlines, Question-

Driven Analysis,

Talent Flow, Analytics

Workspace, and More


Data Source

Workforce Analytics Learning Analytics


Training Data

Personal, Employment and Job Data

Dimensions and

Hierarchies

Volume• Learning Hours/Classes per FTE &

Event• Percentage Filled• Learning Penetration Rate• Total Training Classes Offered

Quality• Cancellation, Abandonment, Completion • Course Participation & Pass Rate• Number of Employees Successfully

Trained• Satisfaction with Learning

Process• Learning Channel Delivery Mix• Learning Program Content Breakdown• Learning Staff Ratio• Trainer Productivity

Examples of Foundational Learning Metrics


Metric Packs

Absence Management

Career & Development Planning

Compensation Planning

Employee Relations

Financial Management

Goals Management

HR Service Delivery

Learning ManagementLeave Accrual

Health and Safety (OH&S)

Performance Management

Recruitment / Staffing

Payroll and Benefits

Succession Management

Survey

SuccessFactors Workforce AnalyticsLeverage best practices to drive insight

Barriers to Business ImpactKeeping you from making optimal impact


We’re not always aligned

We identify key performance metrics

to improve with management

Learning initiatives support

skills the business needs

There’s board-level

accountability for learning

2012-13 Towards Maturity Benchmark

34% 37%

63%


Form Follows Function

Business Goals

Learning Strategy

Learning Org Structure

Funding Model

• 39% Employee Engagement

• 61% Business Impact

• 44% Centralized

• 30% Federated

• 22% Segmented

• 4% Decentralized

• 62% Centralized L&D Team

• 30% Business Unit L&D Team

• 8% Local Manager Spend

Source: Institute of Corporate Productivity

• KPIs, metrics, dashboard?

• Strategic Workforce Plan?

• People Strategy?

What are you here to achieve? Why does your department exist?


Better-aligned learning makes greater business impact

Source: Towards Maturity – Alignment Index 2012-2013

Bottom Q 3rd Q 2nd Q Top Q0

1

2

3

4

5

6

7

Alignment and Business Results

Alignment Index

Number of Bene-fits Reported

Case Studies and Learning/Talent Analytics


Customer Example: The Opportunity Cost of Investing in Poor Performers

2008 2009 20100

10

20

30

40

50

60

Exceptional Highly EffectiveAchieves Expectations Needs ImprovementUnacceptable

Trai

ning

Hou

rs p

er F

TE

From 2009 to 2010, training hours delivered decreased for all levels of performance EXCEPT for employees rated as “Unacceptable”.

• Is this outcome consistent with company strategy?

• What is the opportunity cost of failing to invest in training for high performers (those rated “Highly Effective” and “Exceptional”)?

Training Hours vs. Performance


Customer Example: Quantifying the Impact of Training on Salesforce Revenue


…transferred or promoted most individuals

OUT, by Performance Rating

…transferred or promoted most individuals IN, by Performance Rating

Performance Rating

Number of People

1-3 25

4 116

5 36

Performance Rating

Number of People

1-3 16

4 64

5 21

Top 10% of Managers who:

Average salary increase of top

importers of talent (9%)

Average salary increase of top

exporters of talent (6%)

Do we reward managers who are net exporters of talent?


For our turnover vary by performance rating and position tenure?

Total voluntary labor turnover, Total Organisation (8%)

Total voluntary labor turnover, Sales Reps (16%)

Total Turnover by Performance Rating

Performance Rating

Turnover

Not Rated 16%1 38%2 19%3 12%4 18%5 22%

Average 16%

How does labor turnover vary by performance rating & position tenure?Case study for Sales Reps


Case Study: Sales Reps (continued)

How does labor turnover vary by performance?

Rating and Position Tenure?

Total Turnover of Sales Reps:by Performance Rating, by Position Tenure

Position Tenure

Performance Rating

< 1 Yr 1-2 Yrs 3-5 Yrs 5+ Yrs Average

Not Rated 16% - - - 16%

1 - 32% 55% 17% 38%

2 - 26% 24% 16% 19%

3 - 17% 11% 18% 12%

4 - 8% 36% 20% 18%

5 - 10% 27% - 22%

Average 16% 14% 18% 9% 16%


Case Study 3:Sales Rep Revenue, by Tenure and Performance Rating

Tenure

Sales

Rating 4-5 Rating 2-3 Average

< 1 year $310,000

1 - < 2 yrs $520,000 $390,000 $455,000

2 - < 3 yrs $680,000 $410,000 $545,000

3 - < 5 yrs $685,000 $385,000 $535,000

5 + yrs $665,000 $340,000 $502,500

Average $637,500 $381,250 $469,500


Conclusions

• Do we address poor performers in their second year of employment inconclusive from information available?

• We are losing too many high performers who have been in their current position for more than three years. Why?o Lack of promotion optionso Lack of career optionso Poorly defined employment value proposition

• How do we get Average Revenue per Sales Rep to $600,000 per year?o Reduce recruitment of poor performerso Address poor performance quickero Retain more high performance Sales Reps (4 and 5) in the 3-5 years

of position tenureo 500 sales reps X $100,000 = $50,000,000 increased sales.

Resources


Yet we don’t yet have the data analysis skills…

4 McKinsey Global Institute. Big Data: The Next Frontier for Innovation, Competition, and Productivity. May 2011.

US shortage

200,000 workers with deep analytical skills

• 1.5 million managers able to analyze data and make decisions

based on findings


Summary….Six Steps to Success

1) Deliver fact/data based business cases for what training you are doing and why.

2) Demonstrate why investing in training program X, will help the company execute on its strategy and deliver ROI through authoritative insight.

3) Engage the business leaders, HR, finance, sales and operations with business metrics to really show what impact training is having and what needs to be measured.

4) Embrace workforce analytics as an integrated part of decision making and combine multiple data sources from finance, sales, HR and training, to allow you to measure the relationship and impact training has on the strategy, alignment, people performance and productivity of your organization, thereby linking your training strategy to the business.

5) Embark on a “transformation journey” - a phased development with each phase representing a more mature use of workforce analytics and a more specific strategic business impact. Move from reporting (understanding what’s happening today) to analytics (why it’s happening), to planning (how do we take action today to ensure we have the right workforce trained for tomorrow).

6) Engage with HR and the business to uncover potential problem areas and ensure that they can quantify the impact of interventions.

Source: Ed Cohen and Peter Howes, “Learning and Analytics”, SuccessFactors White Paper, 2012.

© 2014 SAP AG or an SAP affiliate company. All rights reserved.

Questions,Comments,Feedback?

PeterHowesVice President, Workforce Planning and AnalyticsSuccessFactors, an SAP [email protected]+44(0)79 7469 6218

mailto:[email protected]


Thank You