3/13/17

1

PREPARING STUDENTS TO PROVIDE BEHAVIORAL HEALTH SERVICES IN A PRIMARY CARE SETTING:

THE STUDENT INTEGRATED CARE COMPETENCY SCALE

2017 All Ohio Institute on Community Psychiatry March 25, 2017

Tamara S. Davis, PhD, MSSWAssociate Dean for Academic AffairsProgram Principal Investigator

College of Social Work

Rebecca Reno, PhD, MSW, MAPostdoctoral Researcher,OSU College of Public Health

Additional Co-authors:

Joe Guada, PhD, MSWAssociate Professor

Adriane Peck, MSW, LISW-SMEDTAPP Integrated Care Program Manager

Lauren Haas-Gehres, MA, MSWMEDTAPP Data Analyst

Staci Swenson, MA, MSW, LISW-SIntegrated Care ManagerPrimaryOne Health

Shannon Evans, LPCC-SMEDTAPP Clinical Supervisor

Stacey Saunders-Adams, PhD, MSSA, LISW-SMEDTAPP Senior Research Associate

This program is partially funded by the MEDTAPP Healthcare Access (HCA) Initiative and utilizes federal financial participation funds through the Ohio Department of Medicaid. Views stated in this presentation are those of the researchers only and are not attributed to the study sponsors, the Ohio Department of Medicaid or to the Federal Medicaid Program. MEDTAPP HCA Initiative funding supports teaching and training to improve the delivery of Medicaid services and does not support the delivery of Medicaid eligible services.

Thisworkforcedevelopmentinitiativeisacollaborationamong:

OSUCollegeofSocialWork

PrimaryOne Health

(andoriginatingpartnerMentalHealthAmericaofFranklinCounty)

Acknowledgements

3/13/17

2

Introduction to the Integrated and Culturally Relevant Care (ICRC)

Field Education Program

and the

Student Integrated Care Competency Scale (SICCS) &

Student Integrated Care Competency Scale-Supervisor

(SICCS-S)

APPROACH TO INTEGRATED CARE

Peek,C.J.(2013,p.13).Integratedbehavioralhealthandprimarycare:Acommonlanguage.InM.R.TalenandA.BurkeValeras(eds.)IntegratedBehavioralHealthinPrimaryCare.NewYork:Springer.

INTEGRATED PRIMARY CARE OR PRIMARY CARE BEHAVIORAL HEALTH

“Combines medical & BH services for problems patients bring to primary care, including stress-linked physical symptoms, health behaviors, MH or SA disorders. For any problem, they have come to the right place –“no wrong door” … BH professional used as a consultant to PC colleagues” (emphasis added)

3/13/17

3

ICRC CURRICULUM COMPONENTS

Specializededucationandareasofcompetency

IntegratedCare

Healthcarebasics

Technology

Screening,assessment&diagnosing

Carecoordination&communication

Culturallyresponsivecare

Evidence-informedapproaches

Documentation

(Davis,T.S.,Guada,J.,Reno,R.,Peck,A.,Evans,S.,MoskowSigal,L.,&Swenson,S.,2015)

• Demand outpacing workforce prepared for integrated behavioral health services àNeed for quality training programs (McCabe & Sullivan, 2015; Rishel, 2015)

• Social workers in integrated health settings do not feel prepared (Horevitz & Manoleas, 2013)

• Social work positioned to lead interdisciplinary training (Taylor, Coffey, & Kashner, 2015)

• CSWE 2015 EPAS (CSWE, 2015)• Curriculum expands generalist competencies• Curriculum includes advanced clinical competencies

• Aligns with SAMHSA and HRSA integrated care competencies for behavioral health clinicians (Hoge, Morris, Laraia, Pomerantz, & Farley, 2014; Stanhope, Videka, Thorning, & McKay, 2015)

ICRC CURRICULUM CURRENCY

3/13/17

4

SICCS DESIGN

IntegratedCare

(4Questions)

Healthcarebasics

(4questions)Technology(3questions)

Specializededucationandareasofcompetency

Screening,assessment&diagnosing(4questions)

Carecoordination&communication(4questions)

Culturallyresponsivecare(4questions)

Evidence-informedapproaches

(10questions)

Documentation(4questions)

Pretest Mid-placement Student Final

Self-Evaluation

Supervisor

Evaluation

B E G I NNI NG O F S T U D E NT ’ S E ND O F S T U D E NT ’ S

TIMING OF SICCS & SICCS-S ADMINISTRATION

F I E L D T R A I N I NG F I E L D T R A I N I NG

Orientation

Trainings

Individual & Group Supervision

3/13/17

5

SICCS & SICCS-S DEVELOPMENT• CBPR development process for curriculum &

scale development– Incorporated feedback from supervisors,

students, practitioners and community health partners

• Iterative Process– Scale reflects curricular content

– Curriculum & scale shifted to incorporate various populations and evidence-based practices

– Input gathered from focus groups with students and formal and informal engagement with others

Mental Health America of Franklin County(Originating Partner)

SICCS• Administered through Qualtrics

• 37 Item Scale

• Student self-report

• All items scaled 0 to 50 = No opportunity to demonstrate competence

5 = Demonstrates advanced skills

• 185 is the highest possible score

3/13/17

6

SICCS-S• Administered through Qualtrics

• 37 Item Scale

• Items scored by Supervisors– Independently (first wave) and Collaboratively (second wave)

• All items scaled 0 to 5 with guiding level of supervision required– Level 0: No opportunity to demonstrate competence

– Level 1: Needs constant supervision/modeling/feedback

– Level 5: Needs minimal supervision/feedback

• 185 is the highest possible score

Competency Scale

Student Evaluation

Program Evaluation

ICRC Evaluation Model: Student Integrated Care Competency Scale (SICCS)

Development,administration,refinementcycle

Administered3xduringthestudent’splacement

Preliminaryfindings

3/13/17

7

METHODS

DESIGN & ANALYSIS•Design:

– SICCS: Interrupted Time-Series Design (Pretest, Midterm, Posttest)

– SICCS-S: Posttest only

• Analyses:– Friedman’s Test

– Wilcoxon Sign Rank Test– Student SICCS

– Supervisor (SICCS-S)/Student Post-test (SICCS)

3/13/17

8

SAMPLE• Convenience Sample

– 23 MSW ASAP or MSW II students completing the ICRC Training & Field Placement (data include only two-semester placement students)

• Demographics– 3 Male, 20 Female

• Two waves: – 2014-2015 and 2015-2016

2013-2014 Student Team

RESULTS

3/13/17

9

STUDENT SICCS TREND

N=23

ü First Administration: First week of Fall Semester

ü Mid-Program: First week of Spring Semester

ü Program Completion: End of Spring Semester1.96

3.70

4.41

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

First Administration Mid-Program Program Completion

Mea

n of

Sco

res

p=.000

STUDENT PERFORMANCE ON SICCS

FRIEDMAN’S TEST

Test Statistics

N 23

Chi-Square 46.000

Df 2

Asymp Sig. .000

Significant difference between Median scores at each SICCS Administration

3/13/17

10

WILCOXON SIGN RANK TESTProgram Start

to Mid-Program

Mid-Program to Program

Completion

Program Start to Program

Completion

Z -4.198 -4.198 -4.198

Asymp. Sig. (2-tailed) .000 .000 .000

r -.62 -.62 -.62

A significant (positive) difference between each testing point with a large effect.

WILCOXON SIGN RANK TESTProgram

Completion vs.Supervisor

Score

Z -2.572

Asymp. Sig. (2-tailed) .010

r .38

A significant difference between student ratings and supervisor ratings. In general, students rated themselves higher than their supervisors on the SICCS; 78% of the time, the supervisor rating was lower than the students’ rating.

Mean Median

StudentRating 4.4065 4.378

Supervisor Rating 4.1152 4.054

3/13/17

11

SICCS & SICCS-S: RELIABILITY

Chronbach’s alpha:– SICCS (student scale) α=.910

– SICCS-S (supervisor scale) α=.938

CONCLUSIONS &NEXT STEPS

3/13/17

12

CONCLUSIONS:• SICCS demonstrates strong reliability across two student

cohorts

• SICCS demonstrates sensitivity to change in three-month administrations

• SICCS demonstrates student gains in competencies over time in the expected direction

• SICCS-S demonstrates strong reliability and utility in triangulating student self-reports of competencies gained

NEXT STEPS:Continue testing SICCS psychometric properties with each cohort of students (currently in year three of instrument implementation)

Additional validity testing with students not receiving ICRC curriculum and field program

Develop online CEU program for licensed behavioral health professionals based on ICRC curriculum

Develop parallel instrument for licensed behavioral health professionals re-tooling to work in integrated healthcare settings

3/13/17

13

REFERENCES:CouncilonSocialWorkEducation(2015).EducationalPolicyandAccreditationStandards,forBaccalaureateandMaster’sSocialWork

Programs.Retrievedfrom:http://www.cswe.org/File.aspx?id=81660

Davis,T.S.,Guada,J.,Reno,R.,Peck,A.,Evans,S.,Moskow Sigal,L.,&Swenson,S.(2015online).Integratedandculturallyrelevantcare:Amodeltopreparesocialworkersforprimarycarebehavioralhealthpractice,SocialWorkinHealthCare,54(10),909-938.

Hoge,M.A.,Morris,J.A.,Laraia,M.,Pomerantz,A.,&Farley,T.(2014).CoreCompetenciesforIntegratedBehavioralHealthandPrimaryCare.Washington,DC:SAMHSA-HRSACenterforIntegratedHealthSolutions.Available:http://www.integration.samhsa.gov/workforce/Integration_Competencies_Final.pdf

Horevitz,E.,&Manoleas,P.(2013).Professionalcompetenciesandtrainingneedsofprofessionalsocialworkersinintegratedbehavioralhealthinprimarycare.SocialWorkinHealthCare,52(8),752–787.

McCabe,H.A.,&Sullivan,W.P.(2015).Socialworkexpertise:Anoverlookedopportunityforcutting-edgesystemdesignunderthePatientProtectionandAffordableCareAct.Health&SocialWork,40(3),155-157.

Rishel,C.(2015).Establishingaprevention-focusedintegrativeapproachtosocialworkpractice.FamiliesinSociety:TheJournalofContemporarySocialServces:96(2),125-132.

Stanhope,V.,Videka,L.,Thorning,H.,&McKay,M.(2015).Movingtowardintegratedhealth:Anopportunityforsocialwork.SocialWorkinHealthCare,(54),383-407.

Taylor,L.D.,Coffey,D.S.,&Kashner,T.M.(2015).Interprofessional educationofhealthprofessionals:Socialworkersshouldleadtheway.HealthSocialWork,41(1),5-8.

Forfurtherinformation…

Tamara S. Davis, Ph.D., MSSWAssociate Dean for Academic AffairsPrincipal Investigator, PCBH Workforce Development InitiativeCollege of Social WorkThe Ohio State Universitydavis.2304@osu.edu