In P ursuit of E vidence- B ased P ractice in S chool M ental H ealth

In Pursuit of Evidence-Based Practice in School Mental Health

December 8, 2011

Aaron Lyon, PhD.

University of WashingtonPsychiatry & Behavioral Sciences

Overview / Summary1. Schools provide more mental health services than all other service

sectors combined, reducing access disparities

2. Schools represent a unique context in which the applicability of many evidence-based practices (EBP) is untested

3. School-based mental health (SBMH) clinicians are nevertheless faced with the task of integrating research evidence into their practice

4. In Seattle’s school-based health centers (SBHCs), we have recently pursued two methods of supporting service quality through flexible EBP implementation• Modular psychotherapy• Standardized assessment & progress monitoring

SBMH and Service Access

• Most youth who require mental health services do not receive them (Kataoka et al., 2002)

• Gaps disproportionately affect low-SES and racial/ethnic minority youth (Ringel & Sturm, 2001; US DHHS, 2001; Zimmerman, 2005)

• Schools provide service access to underserved youth (Kataoka et al., 2007)

Service Access: Youth with Depression Sx

Lyon, Ludwig, Vander Stoep, Gudmundsen, & McCauley (under review)

SBMH and Service Access

• “de facto” service setting for all youth• SBMH offers reduced stigma for service seeking

(Nabors & Reynolds, 2000)

• SBMH accounts for >70% of all MH services (Burns et al., 1995; Farmer et al., 2003)• About 20% of all students receive SBMH services annually

(Foster et al. 2005)

• Multiple models of school-based MH service delivery.

Major Models of School-Based MH Service Delivery (Foster et al., 2005)

1. School-financed student support services• School districts hire staff to provide traditional MH services

2. Formal connections with community mental health services• Agreements made between schools/districts and community agencies to provide MH

services• Services may be located in the school or at the community agency• e.g., Externally-contracted Wellness/School-Based Health Centers (SBHCs),

Practitioners from outside agencies provide school-based services

3. School-district mental health units/clinics• Districts operate and finance their own mental health clinics or multidisciplinary teams

for services/training/consultation

4. Classroom-based curricula• Usually prevention-oriented, aimed at optimizing learning by enhancing social/emotional

growth• e.g., PBIS, Second Step

5. Comprehensive, multifaceted, & integrated approaches• Multiple partners brought together to provide a full spectrum of MH services• e.g., Systems of Care models, Wraparound

Models of School-Based Service Delivery

School-based health centers (SBHCs)• Integrated care clinics situated in schools• Operate in nearly 2,000 schools in the US (NASBH,

2008)

• Typically provide primary care and mental health services (Brown & Bolen, 2003)

• Well-substantiated as a mechanism to increase service accessibility to underserved and under/uninsured (Gance-Cleveland & Yousey, 2005; Kaplan et al., 1999; Wade et al., 2008)

Seattle’s SBHC System

• 15+ SBHCs:• 10 Comprehensive HS• 5 Comprehensive MS

• Public Health-Seattle & King County serves as program manager

• Four additional health care organizations serve as SBHC “sponsors”

• School District and City are key partners

SBHC Sponsors & Sites

Sponsor: Group Health Cooperative

• Aki Kurose MS• Franklin HS• Nathan Hale HS• Washington MS

Sponsor: Neighborcare Health

• Denny MS• Madison MS• Hamilton MS• Roosevelt HS• Sealth HS• West Seattle HS• Secondary Bilingual Orientation Center

(SBOC)

Sponsor: Public Health Seattle & King County

• Cleveland HS• Ingraham HS• Rainier Beach HS

Sponsor: Swedish Medical Center

• Ballard HS

Sponsor: SCH Odessa Brown Children’s Clinic

• Garfield HS

Seattle’s SBHC Model

Staffing: 1.0 FTE Mid-Level Practitioner (NP/PA)• 1.0 FTE MH Counselor

• 1.0 FTE Administrative Support

Middle schools have a .5 FTE medical provider

Some sites offer “enhanced” services (health education, nutrition, naturopathic medicine)

SBHC Scope of Services

Primary care services:• Preventive healthcare including

immunizations and well-child care• Primary and acute health care

assessment, diagnosis, treatment and referral

• Age appropriate reproductive health care/family planning

• Screening and treatment for sexually transmitted diseases

• Mental health screening, counseling, case management, and referral

• Health education and health promotion• Care coordination and referral for

drug/alcohol services and dental care

Interventions and strategies that support school success:

• Standardized behavioral and health risk assessment

• Mental and behavioral health interventions

• Medical evaluation and intervention• Facilitating on-going peer support groups• Facilitating communication on students’

behalf within school communities• Linkages and coordination with

community organizations• Referral to substance abuse services• Linkage to tutoring and academic

support.

SBMH and Evidence-Based Practice (EBP)

“It is important to emphasize that promotion of effective mental health practices in schools…involves more than simply ‘trumpeting’ the selection of ‘evidence-based’ approaches, most of which have not been examined for their effectiveness, palatability, durability, affordability, transportability, and sustainability in real-world school or clinic settings” (Paternite, 2005; p.660)

The Context of SBMH

• EBP developers have paid insufficient attention to the school context and how it might influence effective service delivery (Ringeisen et al., 2003)

• SBMH providers must deal with a wide range of…1. Presenting problems

2. Symptom severity / Level of functioning

3. Treatment engagement / duration

4. Organizational supports / barriers

The Context of SBMH

Presenting problems• Most common: social, interpersonal, or family

problems (Foster et al., 2005)

• Externalizing / disruptive behavior disorders common in many SBMH settings, especially among males

• Most common diagnoses dependent on factors such as: age/demographics, referral processes, confidentiality approaches• e.g., Most common in Seattle SBHCs: Interpersonal

problems, depression, anxiety

The Context of SBMH

Symptom severity• Varies widely• Many youth receiving services are experiencing sub-clinical levels of

symptoms. SDQ results from SBHCs…Non-

Clinical“Borderline”

ClinicalClinical

Emotional Symptoms 63% 13% 24%

Behavioral Difficulties 66% 16% 18%

Hyperactivity/Attention 66% 10% 24%

Peer Problems 55% 37% 8%

TOTAL DIFFICULTIES 61% 21% 7%

Prosocial Behavior 94% 3% 3%

The Context of SBMH

Symptom severity• PHQ-9 (depression) and MASC-10 (anxiety) scores across

>180 youth in September 2011:

The Context of SBMH

Treatment duration / engagement•Significant service flexibility (timing, content, etc.)•Small number of providers responsible for an entire school Time, practice, and caseload size parameters•Intervention is often short-term for many youth

• Avg. number of MH therapy sessions in SBHCs = 4.6 / semester

•Attendance can be unpredictable

The Context of SBMH

Organizational supports•Organization may be the school, district, or a community sponsor agency

• Considerable variability in the level and type of supports. e.g.,…• outside referral opportunities• time for training• caseload size

•EBP implementation in schools is facilitated by the presence of (Langley et al., 2010)…

• Social networks of practitioners delivering similar interventions• Administrative support for implementation• Funding devoted to implementation

Evidence-Based Practice in SBMH

• Commonly-cited concerns about the transportability of EBP to new contexts are also relevant to SBMH.• Substantial need for flexibility• Ability of EBP to address the full range of client problems• SBMH service accessibility may make concerns about the cultural

relevance of EBP even more important

• Majority of specific SBMH interventions with empirical support have been for behavioral problems (Hoagwood et al., 2007)• Many prevention-oriented, classroom-level interventions

• Relatively little research surrounding the impact of SBMH interventions on academic outcomes (Atkins et al., 2010; Franklin et al., 2009)

EBP Implementation in Seattle’s School-Based Health Centers

Evidence-Based Practice in SBHCs

• Since 2006, our team of UW researchers has had an ongoing training relationship with SBHC mental health providers• Open to providers at all district sites from all

participating sponsor agencies• Provide CE credits, case consultation, ongoing

support • Focused on addressing barriers to

implementation and enhancing the EBP-context “fit”

Evidence-Based Practice in SBHCs

Recent efforts:•2009-10: Feasibility of implementing a modular intervention model within existing consultation structure

• Ultimate goal: Increase flexible use of EBPs

•2011: Increase provider use of standardized assessment (SA) and routine progress monitoring

Common Elements & Common Factors

• Common elements• Generic components/procedures of treatment (e.g.,

exposure, psychoeducation) cut across distinct treatment protocols

• Common factors• Personal and interpersonal components (e.g., alliance,

therapist effects) common to all interventions are responsible for treatment outcomes

Barth, R. P., Lee, B. R., Lindsey, M. A., Collins, K.S., Strieder, F., & Chorpita, B. F. et al. (in press). Evidence-based practice at a crossroads: The emergence of common elements and factors. Research on Social Work Practice.

Modular Psychotherapy Pilot

Rationale: •Modular psychotherapies more flexible than traditional manuals with regard to treatment timing (McHugh et al., 2009; Weisz & Chorpita, in press)

•Flexibility facilitates the dissemination of the model to school settings (Weist et al., 2009)

•Have the potential to be tailored in order to be maximally effective in the limited number of sessions available to SBHC providers•Utilized common elements

Common Elements / Modular Psychotherapy Resources

•Three elements of the MAP system:

PracticeWise Website•Up-to-date research information database•Matches youth problems and characteristics to practice elements

Practice Elements Clinical Dashboard•Track implementation and client progress

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child

Mother

Father

Other

Progress Measure:

Fear rating

Rewards

Commands

Time Out

Praise

Problem Solving

Parent Monitoring

Response Cost

Ignoring/DRO

Stimulus Control/Antecedents

Attending/Directed Play

Modeling

Cognitive

Parent Psychoeducation

Self-Monitoring

Relaxation

Exposure

Maintenance

Psychoeducation

Activity Scheduling

Skill Building

Social Skills

Self Monitoring

Other

Other

Other

Other

Other

Other

Other

Other

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

• Informed by modular therapy tools available from PracticeWise.com (Chorpita et al., 2009)

• 12 practice elements introduced• Corresponded to the most common presenting

MH problems in our SBHCs (depression and anxiety)

• Counselors tracked practice element implementation and outcome monitoring using Excel “dashboard”


• 7 SBHC counselors selected 66 students for tracking• Primary presenting problem:

• Depression – 75%; Anxiety – 14%; Mixed Dep. & Anx. – 11%

• 487 Total sessions across 66 students• Mean # sessions per student = 7.4 (range: 1-24, median: 6, mode: 3)

• In 94% of sessions, students received at least one standardized assessment measure• Most common measure: Short Mood & Feelings Questionnaire

(SMFQ; Angold et al., 1995)


• Standardized assessment (SA): Use of measurement tools with empirical support for their reliability, validity, etc (Jensen-Doss & Hawley, 2005)

• Focus on incorporating initial assessment and progress monitoring into school-based practice. • Rating scales can increase the ease and accuracy of

clinical diagnosis (e.g., Youngstrom et al., 2004; 2005)

• Clinicians are often not able to detect client deterioration (Hannan et al., 2005)

• Administering SA measures and providing the results to clinicians can result in improved in outcome (Lambert et al., 2003)

Standardized Assessment and Progress Monitoring

Steps of Standardized Assessment Use

1. Selection

2. Administration

3. Scoring

4. Interpretation

5. Feedback to clients

6. Treatment planning

7. Monitoring (over time w/ feedback)

8. Treatment plan adjustment

Feedback:•Feedback Intervention Theory (Kluget & DeNisi, 1996): behavior is regulated by comparisons of feedback to goals•Feedback helps shape therapist behavior in response to a reliable information source•Across theoretical orientations and intervention approaches, progress monitoring is associated with improved outcomes (Bickman et al., in press; Lambert et al., 2005)

•Two levels: (1) feedback to the counselor & (2) feedback to the client

• Feedback to clients may also improve therapeutic alliance, further contributing to positive treatment effects

• Feedback is associated with higher rapport with provider, client self-understanding (Allen et al., 2003), and increased engagement in therapy (Ackerman et al., 2000)


Progress monitoring and normative data:•Progress monitoring involves measures that are less dependent on normative data / strong psychometrics (e.g., attendance, homework completion, positive peer interactions)•Monitoring is determined by treatment goals

• Instead of comparing kids to the general population, we are comparing kids to themselves

•Although standardized assessment tools often make good progress monitoring targets, they sometimes do not



Principles of progress monitoring:•Monitoring is a good idea for ALL youth receiving services•Only track those targets that are meaningful to the specific case/match treatment goals•Graphical feedback increases

understanding and makes the feedback more memorable (Kluger & DiNisi, 1996; Miller & Watkins, 2010)

•Can be used as a tool to validate changes in a student’s experience


Selecting WHAT to monitor:•Method of tracking/measuring ANY treatment goal

•Only those areas that have elevations or are targeted for change

•Track the presence or absence of Bx

•Must be something doable • Know where you / they want to go

Consider “baby steps” - Targets might change over time

(start w/ something easy to change)


Selecting WHAT to monitor:1.Identify presenting problem / treatment goal

2.Select preliminary monitoring targets. State whether target is…

• Negative behavior to be reduced• Positive opposite to be increase

3.Identify data source (student, teacher, parent?)• Standardized tool?• Ideographic behavior (e.g., attendance, self-injury,

interpersonal conflict, positive social experiences)?

4.Determine frequency of monitoring & feedback


Moving from the WHAT to WHY… •Progress monitoring is the first step and gives you the WHAT•Next steps include identifying the WHY (i.e., why is a target changing or not changing?)

• Moving beyond just “success” or “failure”

•The WHY is important to the kid feeling successful• Kid’s feelings of agency and control (therapy isn’t being done to

them or for them but something they are willfully and actively participating in)

Summary

1. Schools provide more mental health services than all other service sectors combined, reducing access disparities

2. Schools represent a unique context in which the applicability of many evidence-based practices (EBP) is untested

3. School-based mental health (SBMH) clinicians are nevertheless faced with the task of integrating research evidence into their practice

4. In Seattle’s school-based health centers (SBHCs), we have recently pursued two methods of supporting service quality through flexible EBP implementation• Modular psychotherapy• Standardized assessment & progress monitoring

Acknowledgments

• Ann Vander Stoep• Elizabeth McCauley• Eric Bruns• Evalynn Romano• Jane Koltracht• Jessica Knaster Wasse• Kelly Thompson• Kristy Ludwig• Michael Pullmann• Sarah Walker• Suzanne Kerns• TJ Cosgrove• School-based practitioners!

Collaborators: Funding:• National Institute of

Mental Health• Public Health of Seattle

& King County• Bill and Melinda Gates

Foundation

For additional information on SBHCs

http://schoolhealthcare.org/SBHCWA.aspx

http://www.kingcounty.gov/healthservices/health/child/yhs/locations.aspx

Documents

In P ursuit of E vidence- B ased P ractice in S chool M ental H ealth