Upload
neil-lyons
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Data Driven Decision Making within a Multi-tiered System of
SupportLucille Eber, Ed.D.
Midwest PBIS Network and PBIS National TA CenterNovember 4, 2015
School Mental Health Capacity Building Meeting20th Annual Conference on Advancing School Mental Health
New Orleans, LA
The CSMH’s Meeting Goals for Today
1. Learn how to use the national census and SMH performance measures;
2. Explore engagement strategies to promote cross agency and stakeholder partnerships;
3. Learn about best practices, policies, financing strategies that will help improve your comprehensive SMH system;
4. Obtain innovative resources and tools related to advancing quality and sustainability in SMH to take back to your state/community;
5. Begin an action plan to improve your state/district/school comprehensive SMH system.
This DDDM Session
– Define DDDM in context of school mental health– Provide system and practice examples of DDDM in
schools that can inform – Consider opportunities for advancing SMH
innovations through DDDM
Acknowledgements
Rob Horner, UOKent McIntosh, UOKelsey Morris, UO Tamika LaSalle, UConnSusan Barrett , Mid-Atlantic PBIS NetworkMark Weist, USCOthers from all of our Centers…..
DDDM: A Key Feature of Effective Systems of Support in Schools
– Tiered prevention logic– Cross-system problem solving teams – Use of data to decide which evidence based
practices to implement.– Progress monitoring for both fidelity and
impact. – Active involvement by youth, families, and
other school and community stakeholders.
DDDM: A Key Feature of Continuous Quality Improvement
Plan
Implement
Evaluate
DDDM: A Key Feature of Sustainability
Plan
Implement
Evaluate
What is most related to high sustainability? The frequency that data are presented to all school staff.McIntosh, K., Predy, L., Upreti, G., Hume, A. E. & Mathews, S. (2014).
• Data are the many sources of information we use to make decisions about how to allocate resources (e.g. time and attention) for implementing EBPs to achieve desired outcomes.
• In Schools, data come in many forms such as office referrals, attendance records, grades, visits to the nurse/counselor, surveys (student, teacher, families), verbal feedback, and observations.
• Data is only useful if it is documented, shared, and applied to specific action planning (plan, implement, evaluate, refine) .
What is Data?
What is Data-Driven Decision Making?
• DDDM is the process of using relevant information to make objective decisions.
• DDDM can help inform decisions related to systems development/improvement and student outcomes.– Student Performance Data (academic,
social/emotional)– Fidelity Data (measuring core feature
implementation)
Data Collection vs Data Analysis
Data Collection• Systematic gathering of
required information.
Data Analysis• Examination of data such
that relationships, patterns, trends, etc. emerge and foster hypotheses and decision making.
Value and Utility of Data
Repeatedly giving people the right information, at the right time, in the right format is the single most effective way to improve decision making and achieve valued outcomes (Gilbert, 1978).
Data
PracticesSystems
Valued Outcomes
Improving Decision Making
Problem
Problem
Solving
Solution
Action Plannin
g
Problem
Solution
Problem-solving Steps : Systems or Student Level
1. Determine problem
2. Conduct Assessment
3. Obtain Data
4. Analyze Data
5. Determine Conclusion
6. Implement Strategy
Rationale
• Data allows stakeholders/teams to look at a problem more objectively.
• Without data, teams (or individuals) are more likely to make ambiguous, or emotionally driven decisions.
• Data is used for identifying and planning to address problems, celebrating successes, and accountability.
• Data helps teams ask the right questions…it does not provide the answers.
• Use data to:– Identify problems– Refine problems– Define the questions that lead to solutions
• Data helps place the “problem” in the context rather than in the students.
Why Use Data For Decision Making?
Problem Solving with Precision
• The statement of a problem is important for team-based problem solving.– Everyone must be working on the same problem with
the same assumptions.
• Problems are often framed in the “primary” form. – Raises awareness– Not useful for problem solving
• Precise problem statements result from a detailed data review and are solvable.
Problem Solving with Precision
Primary Statements• There are too many
referrals
• Gang behavior is increasing
• The cafeteria is out of control
• Student disrespect is a big problem
Precision Statement
• There are more ODRs for aggression on the playground than last month. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.
Problem Solving with Precision
There are more ODRs for aggression on the playground than last month. These are most likely to occur during first recess, with a large number of students, and the aggression is related to getting access to the new playground equipment.What? Where? When? Who? Why?
Aggression Playground 1st Recess Large number of students
To get new playground equipment
Student Outcomes: Data Examples
Attendance Academic performance Social/emotional Discipline referrals
Suspensions Placements outside of natural environments
Expulsion Arrests Hospitalizations
Student Perception Safety Surveys School Climate Surveys
Other…
Recognize and RespondMultiple Data Points as Natural Screeners - LOW Threshold
– Attendance– Office Referral– Suspension– Instructional Time– Tardy– HW Completion– GPA– Credit Accrual
Example: DDDM Guides Necessary Conversations within a MTSS
CICO
SAIGGroup w. individual
feature
Complex
FBA/BIP
Problem Solving Team
Tertiary Systems Team
Brief
FBA/BIP
Brief FBA/BIP
WRAP
RENEW
Secondary Systems Team
Plans SW & Class-wide supports
Uses Process data; determines overall
intervention effectiveness
Standing team with family; uses FBA/BIP process for one youth
at a time
Uses Process data; determines overall
intervention effectiveness
UniversalTeam
Universal Support
Family and community Family and
community
Family and community
Community
School Data Community Data
• Academic (Benchmark, GPA, Credit accrual etc)
• Discipline• Attendance• Climate/Perception• Visits to Nurse,
Social Worker, Counselor, etc
• Screening from one view
• Community Demographics
• Food Pantry Visits• Protective and Risk
Factors• Calls to crisis centers,
hospital visits• Screening at multiple
views
School Climate Data as a Meaningful Resource
• Efficient & relatively easy to assess• Provides information about the school
environment from multiple perspectives• Allows for the comparison of data across
participants, time, settings, etc. • Adaptable
Teacher Perceptions
Parent PerceptionsCT Georgia Parent School Climate Survey Overall Average and Average by Subscale
Secondary Student Perceptions
Example: Tier 2/3 Data Decision Rules
• Staff need to know how students are eligible for the intervention (IN), how progress is monitored (ON), and how students exit the intervention (OUT)
– IN = Identification– ON = Progress Monitoring– OUT = Exiting/Transitioning
• Examples of data used to progress monitor– Points earned on Daily Progress Report (DPR)– Reduction in ODRs– Attendance improvement– Reduction in In School Suspensions– Reduction in Out of School Suspensions– Improvement in grades– Reduction of tardies– Student survey
• Reverse Request for Assistance Form– Secondary systems team gives RRA Form to student’s teacher(s) to determine any
intervention changes
Student perspective Survey Example‘I feel connected to my school’
44%
11%11%
11%
22%
Pre-TestStrongly Disagree Disagree
Neither Agree nor Disagree Agree
Strongly Agree
33%
22%
45%
Post-TestStrongly Disagree Disagree
Neither Agree nor Disagree Agree
Strongly Agree
U-46 SD,IL
Daily Progress Report (DPR) SampleNAME:______________________ DATE:__________________
EXPECTATIONS1st block 2nd
block3rd block 4th block 5th block 6th block 7th
block
Be Safe 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0
Be Respectful 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0
Be Responsible 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0 2 1 0
Total Points
Teacher Initials
Mark will keep hands to self
Mark will hold up a yellow card to
indicate needing a break
Mark will fill out assignment notebook
“Individualized Student
Card for Mark” (FBA/BIP)
Replacement behaviorPossible behaviors taught in previous SAIG groups
“Social & Academic
Instructional Groups”
(sample academic skills group)
Walk to classKeep hands to
self
Use appropriate language
Raise hand to speak
Bring materials Fill out
assignment notebook
Examples of Data to Guide System Development
Readiness Surveys Resource Maps/System Audits Fidelity of Implementation
Systems Fidelity of Specific Practices
CICO Fidelity Tools CBITs Fidelity measures Assessing Core features of FBA/BIP Wraparound Fidelity Tools
DDDM Applications
• System Readiness Examples• Fidelity-Systems
– Measures extent to which core features are in place
– Self-assessment or outside evaluator
Interconnected System Framework Readiness Survey (SMH & PBIS)
• 63-item survey • All school faculty and
staff• Used to determine
readiness to integrate PBIS and SMH
• Pilot study conducted by Vittoria Annello and Mark Weist (publication forthcoming)
Westchester NY Example Readiness Survey (District Application)
Westchester NY Example Readiness (Ditrict Application)
Westchester Readiness Survey Results
Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q80
10
20
30
40
50
60
70
80
1 2 3
Fidelity Tools
40
Fidelity tools are the GPS, MapQuest, Google maps of our journey to support all teachers, students, and instructional leaders on their pathway to success.
Erin A. Chaparro, University of OregonKimberly St. Martin, Michigan’s Integrated Behavior & Learning Support Initiative, MiBLSi
FIDELITY“Fidelity of implementation of academic and
behavior interventions is documented before measuring effects and assigning worth to them or blaming children for continuing failure. Continuing emphasis and dialogue on the importance of treatment fidelity speaks to the need for documenting the extent to which essential features of effective core instruction are in place and implemented with integrity
(cf. Miller, 2010)” Algozzine, Putnam and Horner 2012
• Primary purpose of the instrument is to help school teams improve
• Primary audience for instrument results is the team, faculty, families and administrators of the school.
• Effective use of the instrument requires multiple administrations (progress monitoring)
Example: SWPBIS Tiered Fidelity Inventory ( a self-assessment example)
The purpose of the School-wide PBIS Tiered Fidelity Inventory is to provide an efficient and valid index of the extent to which PBIS core features are in place within a school.
Fidelity Assessment: Assessing Core Elements
Intensive SWPBIS Features• Teams• Resources• Support Plans• Evaluation
Targeted SWPBIS Features• Teams• Interventions• Evaluation
Universal SWPBIS Features• Teams• Implementation• Evaluation
An Example of an Innovation (Today’s Meeting Goal #4)
• ISF Enhancement Action Planning (Companion Guide to SWPBIS TFI)– The purpose is to guide action planning for integration of
Mental Health into PBIS
Implementation Inventory for an Interconnected Systems Framework
• Purpose:– To assist school and community partners in their
installation and implementation of interconnected PBIS and SMH at the building level
– To assess baseline and/or ongoing progress on critical features of SMH/PBIS interconnected systems framework
– To guide action planning for a PBIS/SMH interconnected systems framework
Action Planning Companion Guide
Action Planning Companion Guide
Example of Action Planning from I3 Tool Data:
Data-Driven Decision Making• Effective teams use data to document progress and outcomes, guide
decisions, and inform stakeholders (Boudett, City, & Murnane, 2006; Burke, 2010; Deno, 2005; Hill 2010; Newton, Algozzine, Algozzine, Horner, & Todd, 2011; Newton, Horner, Algozzine, Todd, & Algozzine, 2009; Pidgeon & Gregory, 2004; Renfro & Grieshaber, 2009)
• Fidelity and student outcome data are essential (Fixsen, Blase, Metz, & Van Dyke, 2013)
• Continues to be a struggle for schools (Dunn, Airola, Lo, & Garrison, 2013; Schildkamp, Ehren, & Lai, 2012; Telzrow, McNamara, & Hollinger, 2000)
• Advances in computer technology could provide efficient means for data management (Wayman, 2005)
• A critical predictor of sustained implementation of SWPBIS (Coffey & Horner, 2012; McIntosh et al., 2013)