2/12/2015

1

Jessica Vander Zanden, CHC Dirc. Of Medicare Compliance and

Star Quality, Network Health

Kristen Grenzebach, Learning Program Manager, Anthem Inc.

1

Kristen Grenzebach, Learning Program Manager, Anthem Inc.(Kristen.Grenzebach@anthem.com)

2

Sunday, February 15, 2015 – 12:30-2:00pm

2/12/2015

2

� About Anthem – A Health Benefits Leader

� Course History

� “It Takes a Village”

� Defining the Audience

� The Evaluation

� Analyzing the Results

4

`̀

BC or BCBS licensed plans (6)

Medicaid presence (11)

BC or BCBS licensed plans + Medicaid presence (8)

BC or BCBS plans in 14 states and

Medicaid presence in 19 states

MedicareMedicareMedicareMedicare

4%4%4%4%

BlueCardBlueCardBlueCardBlueCard

14%14%14%14%

MedicaidMedicaidMedicaidMedicaid

13%13%13%13%

Individual Individual Individual Individual

5%5%5%5%

National National National National AccountsAccountsAccountsAccounts

19%19%19%19%

LocalLocalLocalLocalGroupGroupGroupGroup

41%41%41%41%

FEPFEPFEPFEP

4%4%4%4%

Diverse customer base of

37 million medical members

2/12/2015

3

• Annual web-based, self-paced training since 2002

• Content: Ethics, Compliance, Privacy, Fraud & Abuse, Information security and HR

• Partner with Training Systems Design (TSD)

• Hosted on LMS powered by Saba

• Customized by audience selection

Collaborate with business areas

Set expectations for all users

Acknowledge feedback

Different

viewpoints,

priorities

and

expertise

Face

challenges

Tackle

problems

Find

solutions

Deliver

results

2/12/2015

4

KPI Definition

Quality/Satisfaction:

Courseware Effectiveness

& Delivery

Effectiveness of course design including: content, scope

(breadth/depth), flow, degree of interactivity.

Effectiveness of online delivery including: ease of use and

navigation, interaction opportunities, graphics and

illustrations that are meaningful and within context.

Learning EffectivenessInsight into the level of learning (new knowledge and skill

gains)

Job Impact

(Ability to Apply)

Insight into the level and likelihood of application and

impact to the job

Value

(Return on Expectations)

Indicator of overall value and perceived benefit of the

course vs. the opportunity cost of being away from the job

• Measure 4 Key Performance Indicators (KPI)• Analyze all comments into category themes• Offer opportunities for improvement

2/12/2015

5

Start upStart upStart upStart up

Statement of work

Goals/Instructional

objectives

Requirements and

constraints

Analysis & DesignAnalysis & DesignAnalysis & DesignAnalysis & Design

Completed

information

gathering

Content points

Instructional

strategy

DevelopmentDevelopmentDevelopmentDevelopment

Instructional flow

Scripts and

screens

ProductionProductionProductionProduction

User Acceptance

Pilot version

Final product

Implement & Implement & Implement & Implement &

EvaluateEvaluateEvaluateEvaluate

Master course

Other services as

contracted

2/12/2015

6

� Pilot Testing – Pre Course

� Focus Group – Post Course

� Manager Follow Up

2/12/2015

7

Going from one-size-fits-all to the right ‘break down’

•Defined “functional areas” of population

•Determine appropriate scenarios for each

•Worked with designers to code

2011

2/12/2015

8

15

2012

Compliance Training

I Am Anthem Training (75-90 minutes, required for all)

Medicare Compliance

Medicaid Compliance

Fraud & Abuse

Manager course

2/12/2015

9

17

2014

2/12/2015

10

•Location

•Business AreaParticipant DemographicsParticipant DemographicsParticipant DemographicsParticipant Demographics

•Material was appropriate to my needs

•Scenarios presented helped me understand the contentCoursewareCoursewareCoursewareCourseware

•Online course was user friendly and easy to navigateOnline DeliveryOnline DeliveryOnline DeliveryOnline Delivery

•I would rate the overall training communication (pre-

course messages, screensavers, website, etc) to be:CommunicationsCommunicationsCommunicationsCommunications

•The training was effective in helping me understand the

process (where and how) to report suspected violationsLearning EffectivenessLearning EffectivenessLearning EffectivenessLearning Effectiveness

•I would be able to apply the concepts learned in this

course to the workplace ImpactImpactImpactImpact

•This course met my expectations

•What about this course was useful to you?ValueValueValueValue

Specific Course Content (60% of Most Useful

comments)

� By far, respondents found the scenarios and examples presented to be most useful. 58% of all Course Content comments were example or scenario related.

� Next most useful was the overall information presented on Fraud & Abuse, followed by information related to Refresher, such as updates and reminders regarding policies and processes.

Courseware Findings (20% of Most Useful comments)

� The response to the course online format and delivery was solidly positive. Learners who submitted comments reported that the courseware was easy to navigate, interactive, and presented the material in a fun and engaging manner.

Specific Course

Content

%

Comments

Scenarios 33%

Examples 25%

Fraud 6%

Refresher 4%

Courseware

%

Comments

Easy/User

Friendly 27%

Videos 6%

Interactive 5%

Resources 4%

2/12/2015

11

• All Good / NA / Nothing (55.3% of Improvement comments) – Over half of the comments relating to improving training were tied to “All Good / NA / Nothing to Improve” indicating that respondents felt the training could not be further improved to be more relevant to their position.

• Content (26.2% of Improvement comments) – Requests for improvements received from respondents were largely centered on customization of scenarios and examples to allow for a variety of roles across the organization.

• Courseware ( 10.2% of Improvement comments) – The vast majority of courseware improvement remarks (61.8%) involved requests for customization by business unit or job role.

• Problem (5.3% of Improvement comments) – Some respondents reported experiencing system functionality issues. Most issues reported were related to issues with screen response (froze or slow to respond).

Content % Comments

Scenario 25%

Examples 14%

Answers 6%

Definitions 5%

Relevant 5%

All Good / Apply Learning on the Job / Apply Content – Nearly half of all respondents indicated that they are currently applying or plan to apply the content, both on the job and during daily interactions with members, associates, and vendors. Those who plan on applying the content reported greater awareness when accepting gifts, sending secure emails, and general awareness of policies and procedures.

All Good / Apply on Job / Apply

Learning % Comments

Apply/Currently Apply 31%

WAH/Offsite 8%

Claims 7%

HIPAA 5%

SIU 4%

Members 4%

Gifts 4%

2/12/2015

12

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Online Delivery &

Courseware

Learning Effectiveness Ability to Apply Value

2010

2012

2014

2008 (Baseline)

BIC - Benchmark

Customized Customized Customized Customized

Curricula/Exceptional Curricula/Exceptional Curricula/Exceptional Curricula/Exceptional

ResultsResultsResultsResults

Asking for Asking for Asking for Asking for

feedbackfeedbackfeedbackfeedback

Analyzing Analyzing Analyzing Analyzing

feedbackfeedbackfeedbackfeedback

“Village”“Village”“Village”“Village”

Culture/ Culture/ Culture/ Culture/

LeadershipLeadershipLeadershipLeadership

2/12/2015

13

Contact Me: Contact Me: Contact Me: Contact Me: Kristen.Grenzebach@anthem.comKristen.Grenzebach@anthem.comKristen.Grenzebach@anthem.comKristen.Grenzebach@anthem.com