Informal Learning, Line Responsibility: Audit to Impact

E-Learning, Informal Learning,

Line Responsibility: Audit to Impact

Will Thalheimer, Work-Learning ResearchHillary Leisten & Sal Venegas, Walgreen Co.

103

March 11 - 13, 2009Orlando, FL

Page 1Session 103 – E-Learning, Informal Learning, Line Responsibility: Audit to Impact – Will Thalheimer, Work-Learning Research

March 2009Retail Best Practices

Sal VenegasCorporate Manager Walgreens Learning [email protected]‐914‐8188

E‐Learning, Informal Learning, Line Responsibility: Audit to Impact

Will Thalheimer, PhDPresidentWork‐Learning Researchinfo@work‐learning.com617‐666‐9637

1

Sal VenegasCorporate Manager, Program DevelopmentWalgreens Learning Services

2

Which of the following

characteristics make it difficult to excel in

the learning enterprise?

Which are opportunities?

Reflection Questions

3

• Long history of store‐based learning

• More than 6000 stores across the US

• Expanding into non‐traditional healthcare

• Tight margins, competitive industry, increased regulation

Walgreen Co.

4

• Geographically dispersed with over 200,000 employees

• 10 – 15 job roles per store

• Very top‐down management, very task focused

• No training budget – managers expected to fit training into budget

Walgreens Retail Stores

5 6

• Large support function, split into two departments: Learning Services & Systems Training

• Learning largely e‐learning, with some classroom training for management

• Main driver of what gets built – requests from Operations

• Learning group is centralized in Chicago

2

Walgreens Learning Landscape



What are the issues with a geographically‐

dispersed audience?

What is likely to happen if stores don’t have a

training budget?


7

• What was the situation like at Walgreens that prompted this effort?

• What was the driving issue?

• Why was someone from the outside called in?

How Did This Get Started?The Walgreens Perspective

8

Will Thalheimer, PhD

9

MissionË Help Clients Build More Effective Learning Ë Utilizing Wisdom from Research & PracticeË Compile Research, Share Findings

WorkË Consulting (Learning, Performance, Assessment)Ë Workshops & SpeakingË Learning Audits, Work‐Learning AuditsË Research and Writing

10

• What had you been doing with Walgreens?

• What seemed to be the driving issue?

• What were some of your biggest worries getting started?

• What were the biggest plusses getting started?

How Did This Get Started?The Work‐Learning Research Perspective

11

• How Well is Learning Supporting Store/ Pharmacy Success?

• What More Can be Done?

• What’s Wrong with the E‐Learning?

Note: E‐learning programs are called PPL’s at Walgreens

Framing the Audit Goals

12



LearnerFulfillment

LearningResults

LearnerApplies

The Learning Landscape

Learner Learns

LearnerRetrieves

LearningIntervention

PerformanceSituation

LearningOutcomes

© Copyright 2008-2009 Will Thalheimer, www.work-learning.com

13

Making Learning WorkThe 4 Factors

A. Preparation

Are Learners Able & Willing to Learn? Does the Culture Encourage and

Support Training?

A1. Do workers have time for learning?

A2. Do workers want to learn the topics taught?

A3. Do learners’managers encourage learning(vs. completion)?

A4. Do learners’coworkers encourage learning?

B. Training

Do Learning Programs Create Understanding

and Enable Remembering?

Do learning programs:

B1. Promote motivation to learn?

B2. Enable understanding?

B3. Focus on relevant tasks?

B4. Provide sufficient practice?

B5. Focus on key concepts, and avoid overload?

C. After-Training

Does the On-the-Job Situation

Support & Reinforce What was Learned?

C1. Do managers reinforce what was learned?

C2. Do workers have incentives and/or motivation to use what they learned?

C3. Do workers have sufficient authority, time, competence, and resources?

C4. Can learners remember what they learned?

D. OTJ-Learning

Does the Workplace

Support On-the-Job Learning?

D1. Do workers know good/poor performance when they see it?

D2. Do managers provide good coaching and feedback?

D3. Are there clear overarching job goals?

D4. Can workers learn from each other?

D5. Is there a learning attitude/culture?

14

• Maximize the benefits of Walgreens formal learning practices, within the business model’s typical cost and time constraints.

• Maximize the benefits of Walgreens store/pharmacy workplace‐learning environment to ensure store/pharmacy success.

Overarching Goals

15

• Phase 1 – Gather Data to Determine Needs

• Phase 2 – Build Consensus About the Issues

• Phase 3 – Bring Stakeholders Together to Make Plan

• Phase 4 – Begin to Implement Plan

• Phase 5 – Continue Momentum, Fix Problems

• Phase 6 – Ensure Integration, Look for Opportunities

Project Phases

16

l Interviews in Boston & New Hampshire with 20+ employees

l Focus Groups in Minneapolis, Tucson, and Southern California

l Store Observations and Shadowing

l Learning Audits of E-Learning Courses (PPL’s)

l In-class review of training for executive assistant managers

l Review of company artifacts (newsletters, magazines, etc.)

l Interviews with training management and staff

l Interviews/Discussions with key business leaders

l Attendance at Learning Task Force meetings

Methodology

17

Strengths of the

Methodology? Limitations of the

Methodology?


18



TheFindings

19

• Focus on competency, not completion

• Improve e‐learning quality

• Retool Walgreens learning function

• Develop better metrics

• Grow “performance development managers”

Recommendations

20

LearningCulture

21

A lesson well learned ‐ and never forgotten…

In a series of jobs with Chicago's leading pharmacists ‐ Samuel Rosenfeld, Max Grieben, William G. Valentine and, most importantly, Isaac W. Blood ‐Walgreen grew increasingly knowledgeable ‐ and increasingly dissatisfied ‐ with what he saw as old‐fashioned, complacent methods of running a drugstore. Where was the desire to provide superb customer service? Where were the innovations in merchandising and store displays? Where was the selection of goods that customers really wanted ... and could afford? Where was the sense of trying to understand, please and serve the many needs of drugstore customers?

Exact Quote from the Walgreens Website

Learning

Analysis

Vision

Action

LAVA

Charles R. Walgreen, Sr.

22

Formal Training

1. PPL’s2. PPL In‐store Worksheet Activities3. ADSM4. Tutoring by DTSS and DPTC, etc.5. Workshops (e.g., Beauty Trainings)6. Public Seminars7. Seminars for District Staff at

Corporate8. Emerging Leader

Meetings

1. All‐store meetings2. 5‐minute shift meetings3. Store management meetings (store

managers with their EXA’s and MGT’s)4. District meetings5. Task Force & committee meetings6. Management Seminar (Vegas)

Out‐of‐Store Work Experience

1. Rotations to other stores2. Time spent in training stores3. MGT’s get 1‐day orientation in one

store4. MGT’s spend 20 days in a training

store5. Task force work, (or more generally,

committee work)6. Special assignments for DM7. Special assignments for Region or

Corporate

Reports, Documentation, Databases

1. Reports2. KPI’s3. StoreNet4. Policy and Procedures5. Walgreens World

23 24



25 26

Formal Training1. PPL’s2. PPL In‐store Worksheet Activities3. ADSM4. Tutoring by DTSS and DPTC, etc.5. Workshops (e.g., Beauty Trainings)6. Public Seminars7. Seminars for District Staff at Corporate8. Emerging Leader

Meetings1. All‐store meetings2. 5‐minute shift meetings3. Store management meetings (store managers with their

EXA’s and MGT’s)4. District meetings5. Task Force & committee meetings6. Management Seminar (Vegas)

Out‐of‐Store Work Experience1. Rotations to other stores2. Time spent in training stores3. MGT’sget 1‐day orientation in one store4. MGT’s spend 20 days in a training store5. Task force work, (or more generally, committee work)6. Special assignments for DM7. Special assignments for Region or Corporate

Reports, Documentation, Databases1. Reports2. KPI’s3. StoreNet4. Policy and Procedures5. Walgreens World

Worker‐Initiated Learning1. People ask peers questions. Peers answer questions.2. People ask managers questions. Managers answer questions. 3. People ask for new challenges.4. People ask for feedback from manager(s), peers,

or direct reports.5. People ask to shadow someone with more experience.6. People take notes and refer to them later.7. People take notes to remember to ask questions later.8. People review the PPL’s (online or from printouts).9. People ask to do different tasks or work different shifts.10. People look for opportunities to work with (learn from) “experts.”11. People review StoreNet, KPI’s, and/or other reports12. People read Walgreen’s World, Applause, etc.13. People create a vibe that lets others know they are open to feedback.14. People step back, and observe and reflect on what is going on.15. People take advantage of open‐door policy.16. People gossip and listen to gossip.17. People evaluate their own strengths/weaknesses and look for opportunities to parlay their strengths and improve on

their weaknesses.18. People notice their tendencies and try to be more flexible.19. People look for opportunities for repetitions to solidify their learning and refine their skills.20. People make a special effort to remember who knows what.21. People contact the Help Center.22. Managers ask for feedback on their management behavior.

Manager‐Driven (Expert‐Driven) Learning1. DM’s do walkthrough’s pointing out good and bad things.2. Managers do walkthroughs pointing out good and bad things.3. Assistant Managers do walkthroughs pointing out good and bad things.4. Managers ask workers to complete tasks on task list, highlighting what is important.5. Managers assign tasks to be done, enabling workers to learn new tasks.6. Managers assign tasks to be done, reinforcing previous learning.7. Managers show how a task is done.8. Managers show how a task is done and then assign the task and give feedback. 9. Managers provide constructive criticism and positive feedback at critical moments.10. Managers assign workers to (a) Observe Task Being Done, (b) Do Task with Close Supervision, (c) Do Task with Far

Supervision, (d) Do Task On Their Own. 11. Managers leave problems, and see how long the problem will stay (w/o worker intervention).12. Managers intentionally pair “learners” with people with the right attitude, knowledge, and teaching skills.13. Managers let employees try experiments (i.e., merchandising) andthen look at data to see what effect it had.14. Managers give people new challenges. 15. Managers encourage an “experimental attitude,” encouraging workers to wonder what would happen if changes were

made, trying things out, learning together through trial and error.16. Managers assign people to take over certain responsibilities when they are off.17. Managers assign people to take over other people’s responsibility when they are off. 18. Managers answer questions, or find out who to ask.19. Managers give people a notebook to (a) write down questions, (b) take notes on things to remember. 20. Manager encourages others to approach him/her, encouraging questions, discussions, etc.21. Manager models appropriate behavior.22. Manager coaches/mentors store employees.23. Manager sets guidelines for each worker to know who to ask.24. Manager educates workers on who knows what.25. Manager educates workers on the location of critical information.26. Manager provides employees with detailed performance reviews.27. Managers give others responsibility for doing walkthroughs.28. Store manager assigns people particular departments or areas in the store to manage, and together they observe the

results (from a sales perspective).29. Store manager keeps track of each employee’s development, either through a mental or actual checklist, and provides

coaching, training, and learning opportunities as appropriate.

Hands‐onLearning

27

How many of you get to look at on‐the‐job learning/ performance?

A.ExtensivelyWe fully

understand it.


B.Significantly

We understand

some

C.Occasionally

We understand a

little

D.None/LittleWe don’t really get it

28

LearnMost

Learn a Lot

LearnSome

Learn a Little

Learn Least

“Off theboard”

Rough Example – Pharmacy Tech

PPL’s

Experience

Self Learning

StoreNet

WalgreensWorld

Customers

InsuranceCompanies

Corporate 3rd Party

WalgreensCorporate

Pharmacist

PharmacyManager

Other Pharmacy Techs

Senior Pharmacy

Tech

Pharmacy Supervisor

Store Manager

DPTC

EXA

MGT

29

Does your business side understand that people learn from those closest

to them?

Reflection Question

30



How Learning Works in Stores

31

OK

Expert

Weak

How Learning Works in Stores

32

Weak

Beauty Advisor

ExpertWeak

OK

HeadCashier

ServiceClerk

H. Beauty Advisor

Expert

StoreManager

OK

EXA

Weak

MGT

Expert

SIMSCoord’r

Weak

WeakWeakOK

PhotoTech

ServiceClerk

ServiceClerk

ServiceClerk

OK

H. PhotoTech

Moderate‐Skill Store

33

Weak

Beauty Advisor

ExpertWeak

HeadCashier

ServiceClerk

Expert

StoreManager

Weak

MGT

Expert

SIMSCoord’r

Weak

WeakWeakOK

PhotoTech

ServiceClerk

ServiceClerk

ServiceClerk

Expert

Expert

Expert

H. PhotoTech

H. Beauty Advisor

EXA

Strong‐Skill Store

34

Weak

Beauty Advisor

Weak

OK

ServiceClerk

H. Beauty Advisor

Expert

StoreManager

OK

EXA

Weak

MGT

Weak

WeakWeakOK

PhotoTech

ServiceClerk

ServiceClerk

ServiceClerk

OK

H. PhotoTech

OK

OK

SIMSCoord’r

HeadCashier

Weak‐Skill Store

35

Implications

1. On‐the‐job Learning is easier at some stores than others.

2. Causing a spiraling up/down effect:

• the more learning, the more experts, the more learning, etc.

• the less experts, the less learning, the less experts, etc.

36



Is this notion widely considered in your organization?

Reflection Question

37

E-Learning38

Store Manager:

“The problems before PPL were greater than they are now. I think PPL has created additional issues, so I don’t think it’s fair to say, get rid of PPL get rid of our problem. We just go back to the same problem we had before we didn’t know most stores probably weren’t taking the time to communicate with the new employees, which created a lot of problems.”

Exact Quote from the Focus Groups

Get Rid of PPL?

39

“We become more focused on getting people through, tab, tab, tab, tab, test. So that they’re done instead of them learning material. It’s an unrealistic timeline that a lot of times gets thrown out there…”

Manager: “I never asked somebody, you know, did you learn anything from the PPL, just points us to get it done that’s it.”

“The thing that’s important with the PPL is is like make sure that you like learn it right before it comes in. If you got that knowledge fresh in your mind than you got the hands‐on then boom you pick it up like that. If you learn the PPL and then six months later the new equipment comes you’re like well I better just go back and read that again and you’ve wasted time and we don’t have time to waste.”

PPL Feedback

40

“I’ll be real honest. Ever since the first day first time I got here and did PPL’s it was great and I learned a lot on the computer but I really, except for the legal ones, I haven’t really paid attention to a lot of them except one time a couple of years ago when they switched over to the new [pharmacy] system, the whole system got completely revamped, then I really paid attention to that. So, it’s that stuff that is really critical in knowing how to run the computer. The other stuff I really just look over it. I don’t look at it.”

“As I say that, part of me feels guilty actually. I feel guilty saying that because there is somebody out there who actually puts the time in to make sure that people should know this stuff and, all of a sudden I’m realizing, are putting it out there not just to put it out there, maybe somebody out there that feels this is important maybe I should look at it. I’m feeling a little guilty about that.”

PPL Feedback

41

What People LIKE About PPL’s

42



What People Do NOT Like About PPL’s

43

l Perceptions: Lots of strongly-felt complaints. Many see benefits too.

l PPL’s have been evolving to more hands-on application; better design.

l PPLs have a brand problem.

l PPL completion mandate is problematic

l PPL’s create wrong frame about what training/learning is

l Managers may spend less time on coaching, or thinking about development

l Due to corporate staffing model, training department was driven to hire less senior, lesser-skilled staff

l Technology base created limitations

Some Perspective on PPL

44

Re-Brand PPL’s, and revise them to better support hands-on learning.

Improve the PPL learning environment

Energize the courses

Rethink every PPL: Eliminate? Shorten? Redesign?

Add a “Steve Jobs” or “Oprah”hardnosed visioning and editing role

Gather better data on how PPLs are implemented

Improve E‐learning

45

TheCompetency

Way46

Jane Smythe MGT Store # 67384

Course Name Completion? Last Date

Tobacco Handling Yes 10‐1‐2007

Merchandising Introduction Yes 10‐4‐2007

Merchandising 1 Yes 10‐15‐2007

Merchandising 2 Partial 10‐17‐2007

Merchandising 3 No ‐‐‐

Customer‐Service Introduction Yes 10‐2‐2007

The LMS Way

47

The Competency‐Management Way

48




Expert

StoreManager

Store#98102

Store#98106

Store#98104

Store#98105

Store#98103

Store#98101

Expert

H. Beauty Advisor

I need to train my beauty advisor. I wonder who has an expert beauty advisor?

Expert

H. Beauty Advisor

StoreManager

OKHey Joe, can Sally train my beauty advisor?

Sure Sue, can you help me with my Performance‐Development stuff? How about sending your advisor over next week?

Store#98108

Store#98107

49


Store#98102

Store#98106

Store#98104

Store#98105

Store#98103

Store#98101 Store

#98108

Store#98107

DistrictManager

Oops. I’m getting an alarm. The system tells me I might have a weakness in photo skills in my district? Let me see.

Weak

PhotoTech

Expert

H. PhotoTech

OK

H. PhotoTech

Expert

H. PhotoTech

Weak

PhotoTech

Weak

PhotoTech

Weak

PhotoTech

Weak

PhotoTech

Ouch. It looks like we need to plan some extra hands‐on training? I’ll get those two expert Head Photo Techs to cycle around the district for the next month until we fix this.

50

Does your organization have anything like this?

Would this work for you?

Reflection Question

51

LearningFunction

52

• Training is a request‐driven operation for the most part.

• Training constantly educates Ops that issues may not be a training issue.

• Business strategy was to bring in entry‐level staff and grow from within, rather than hiring talent from outside.

• Learning materials are sometimes created without Training input (or knowledge).

Retool Walgreens Learning…

53

• More Focus on Workplace Learning, Management Development, and Performance Consulting – Less Focus on Developing Courses.

• Develop a better partnership with operations, more proactive.

• Seek more highly skilled talent.

• Spend more time on the ground, in stores, with audience

Retool Walgreens Learning

54



Do we need to retool our learning function from time to time? What’s your experience?

Reflection Question

55

Metrics &Feedback Loops

56

Have store personnel rate their developmental opportunities

Rate store development culture, rate store manager, assistant managers.

Have store managers, asst. managers rate district learning support and coaching

Create better measurement for PPL effectivenessLet each learner rate each course, develop competency-management metrics

Do formal control-group evaluations of 1% of courses every year

Develop Better Metrics

57

Are you able to do the learning

measurement you want to?

Reflection Question

58

LeveragingManagers

59

What Sets Them Above?1. Gets Both Store & People Results2. Tracks Employee Competencies3. Looks for Development Opportunities for his/her People4. Is Open for Questions, Discussion. Is Approachable. Is Inviting.5. Makes Developmental Assignments, even when it temporarily reduces store performance.6. Assigns Responsibilities, not just Tasks7. Gives People Practice in Noticing what is wrong and what is right8. Enables Experimentation and Follow-up Data Analysis an Decision-Making9. Uses PPL’s to Drive Hands-on Learning10. Is a Good Role Model11. Creates a Trusting, Caring, Respectful Environment

Grow Performance Development Managers

60



Has your organization been able to leverage its managers to push

on‐the‐job learning?

Reflection Question

61

TheFollow-up

62

Current State

• In first year of a 5 year plan to change the learning culture.

• As part of larger corporate initiative, we’re re‐evaluating the jobs and skills needed for the learning function and reorganizing the department.

• We’ve been bringing in higher level more experienced people, to manage and staff the function.

63

• As part of a larger talent management initiative, we are building a competency‐learning system.

• As part of a larger initiative, we are putting more training resources into the field.

Current State

64

Will Thalheimer, PhDPresidentWork‐Learning Research, Inc.2 Belmont TerraceSomerville, Massachusetts, 02143, USA

617‐666‐9637

Mail: info@work‐learning.comWebsite: www.work‐learning.comBlog: www.willatworklearning.com

Sal VenegasCorporate Manager, Program DevelopmentWalgreens Learning Services304 Wilmot Road, MS #3165Deerfield, IL 60015

(847) 914‐8188

Mail: [email protected]@venegas.info

65

Documents

Informal Learning, Line Responsibility: Audit to Impact