Tier 3 Systems Development: Using Data for Problem Solving
Don Kincaid and Therese Sandomierski, University of South Florida Maximizing Your Session Participation
When Working In Your Team Consider 4 questions: Where are we in our implementation? What do I hope to learn? What did I learn? What will I do with what I learned? Where are you in the implementation process
Where are you in the implementation process?Adapted from Fixsen & Blase, 2005 Exploration & Adoption We think we know what we need so we are planning to move forward (evidence-based) Installation Lets make sure were ready to implement (capacity infrastructure) Initial Implementation Lets give it a try & evaluate (demonstration) Full Implementation That worked, lets do it for real and implement all tiers across all schools (investment) Lets make it our way of doing business & sustain implementation (institutionalized use) Rationale Individual student progress monitoring data Fidelity data Aggregating data to have a wider impact on support Multi-Tiered System of Supports (MTSS)
ACADEMIC and BEHAVIOR SYSTEMS Tier 3: Intensive, Individualized Interventions & SupportsThe most intense (increased time, narrowed focus, reduced group size) instruction and intervention based upon individual student need provided in addition to and aligned with Tier 1 & 2 academic and behavior instruction and supports. Tier 2: Supplemental Interventions & Supports More targeted instruction/intervention and supplemental support in addition to and aligned with the core academic and behavior curriculum. Tier 1: Core, Universal Instruction & Supports General academic and behavior instruction and support provided to all students in all settings. By now, youre probably familiar with the MTSS triangle. Each tier of support is defined here, with Tier 1 applying to ALL students in ALL settings; Tier 2 providing more TARGETED instruction and support for GROUPS of students who have similar needs; and Tier 3 providing INDIVIDUALIZED and INTENSIVE interventions for students who have the MOST SEVERE needs. Its important to note that each tier of support builds on the one before it, and the 4-step problem solving process is what is used to decide how to adjust a students supports. Remember, too, that the tiers reflect SUPPORTS, NOT STUDENTS. You can have a student who receives Tier 3 supports for one domain, such as reading, Tier 2 supports for another domain, such as math, and Tier 1 support for another domain, such as behavior. Theres no such thing as a Tier 2 student, or a Tier 3 student only OUR students who receive different levels of support depending on their needs. (FL RtI State Transformation Team, Dec. 2009) Students receiving Tier 3 supports are at increased/high-risk for:
Absenteeism Restraint/seclusion or other punitive strategies Disproportionate poor outcomes Unaddressed learning problems Poor performance in class Social isolation Repeat offenders (e.g., multiple ODRs) Safety issues (e.g., suicide, drugs, gangs, abuse/trauma) Neglect (basic necessities missing or insufficient) Graduates with Disabilities
Exiters in each year. Percentages, to give an idea of numbers, in there were 4647 special diplomas. Due to rounding, totals may not equal 100. Post-school Outcomes for SWD (Performance)
Source: Florida Education and Training Placement Information Program (FETPIP) Tier 3 Support Systems FBA BIP Distal Outcomes Proximal Outcomes
Increase in graduation, Decrease in dropouts, Reduced restraint and seclusion, Reduced suspension and expulsion disproportionality Effective data collection and behavioral assessment Evidence-based practices and implementation fidelity 1. Decreases in problem behavior 2. Increases in social skills 3. Academic engaged time Proximal Outcomes FBA BIP Effective Ineffective Poor post secondary outcomes: unemployment, substance abuse, incarceration, mental health issues, poor social relationships Professional development, data based problem solving, systems restructuring, etc. Tier 3 Systems in Schools
Effective evidence-based processes exist to support students with serious problem behaviors Overall, not being implemented with fidelity Improved student outcomes not being realized Ineffective tier 3 systems may impact district performance on indicators: Restraint/Seclusion Suspensions/Expulsions Disproportionality Drop out/Graduation Rates Systemic issues contributing to problem Tier 3 results (outcomes) driven model requires systems
Supporting Decision Making Supporting Staff Behavior OUTCOMES SYSTEMS Information PRACTICES Supporting Student Behavior 3 Types of assessment: Tier 3
Formative Assessment/ Progress Monitoring* How effective is the intervention? Is the intervention being implemented as intended? *Essential for a results-driven system Summative Assessment: How did we do? Diagnostic Assessment: What can and cannot a student/students do and why? What supports are indicated? Progress Monitoring The National Center on Intensive Intervention definesbehavioral progress monitoringas repeated measurement of targeted behavior for the purpose of helping educators determine the effectiveness of intervention programs for students in grades K-12 who have intensive social, emotional, or behavioral needs. (www.intensiveintervention.org) Progress monitoring measures:
are valid are reliable are quick to administer are relatively easy to do are easy to summarize can be repeated frequently are relatively inexpensive provide instant information provide relevant information can indicate potential problem(s) are sensitive tosmall changes Progress Monitoring Across the Continuum
District administrators decisions about time, curricula, and personnel focus on compliance and documentation requirements (Hawley-Miles, Baroody & Regenstien, Education Resource Strategies, no date; Pan, Rudo, Schneider & Smith-Hansen, 2003) Data-Driven approachmay lead to better studentoutcomes (Carlson, Borman & Robinson, 2011) Allocate resources (time, expertise, personnel) according to student need Data should help you prevent problems from getting worse, remediate deficits Formative and diagnostic assessments guide the process Data availability is critical: having better data earlier in the process will facilitate effective intervention Nuts and bolts, how to do at T3 with emphasis on prevention Are your data useful? For what?
Data source(s) should: Help you to identify students' needs in relation to established goals;(DEFINE) Help you to understand the barriers to students' reaching their goals;(ANALYZE) Provide sufficient info to select, match, and deliver services/supports to students; (IMPLEMENT) andcontribute to evaluating the effectiveness of services and supports provided to students. (EVALUATE) Educators have access to LOTS of data, but many times its not the kind of data thats helpful for problem solving. In order for problem solving to be effective, districts & schools need to have data that can inform instruction. Historically, databases have been built to meet reporting requirements (compliance).The data used for compliance arent always the same data needed for problem solving. Data used in the problem solving process needs to be able to do certain things (slide). Many existing data sources for behavior dont always meet these guidelinesthats where the FLPBS Project comes in. 16 Permanent Products, Artifacts
Assess/ Collect Evaluate ODRs, Minors Permanent Products, Artifacts Behavior Report Cards Individualized Tools Weekly (+) Throughout Day 2x/ Month Daily One of the keys to making Tier 2 for behavior work for your school is to ensure that youre collecting and evaluating students progress monitoring data on a regular basis. As schools move up the continuum of supports, the amount of time and detail required for data collect and analysis will increase, just as the intensity of supports increase. Since Tier 1 represents the lowest level of intensity of supports, collecting data on problem behaviors as they occur will be sufficient to make sure that all students maintain a basic level of success. Under most circumstances, these data only need to be examined about once each month to see if there have been changes in school-wide patterns. The level of support at Tier 2 is more intensive than whats provided at Tier 1, and the frequency of data collection and analysis should also increase. Because the goal of Tier 2 is to PREVENT problem behaviors from getting worse, progress monitoring data must be collected FREQUENTYLY in order to identify a poor or questionable response to intervention before it has a change to continue and eventually get worse. Progress monitoring data at Tier2 are examined every couple of weeks to see if there have been any changes to students performance. This helps teams respond to students who need more support in a timely manner, and also helps identify students who no longer need supplemental supports so they can begin to be faded from this level of intervention thereby making room for NEW students who are in need of behavioral support. Keep in mind that while the FREQUENCY of data collection analysis at Tier 2 has increased, the TYPE of data collected at Tier 2 is still easy and efficient for teachers to collect Its a quick, simple assessment of the students performance on the behavioral expectations for the school. In contrast, data collection at Tier 3 is much more specific. This is the level of support where you would collect frequency counts, duration, or latency measures for a specific behavior. Data collection at Tier 3 becomes more technical and more sensitive as students level of need increases, and data should be examined more frequently to ensure that these resource-intensive interventions are effective for your students. As incidents occur Monthly Permanent Products, Artifacts
Who Collects Who Evaluates ODRs, Minors Permanent Products, Artifacts Behavior Report Cards Individualized Tools Teachers, educational assistants, outside support personnel Student-Centered teams with additional expertise Teachers, educational assistants Teachers, school-based teams This identifies the people who are typically responsible for those data sources. Teachers, anyone School-Wide team Ethical Legal Considerations
Consent Duty to Respond Confidentiality State and Federal Regulations Feasible and Efficient Progress Monitoring within a Tier 3 System
Behavior Report Card Behavior Rating Scale Direct Observation Treatment Fidelity Individual Interventions Systems Aggregate Data: Using student outcomes to improve Tier 3 systems Selecting a Progress Monitoring Tool
Define the behavior(s) Decide whenyou will monitorthe behavior(s) Select the mostappropriate tool Explain how to choose a progress monitoring tool at Tier 3 Be specific when defining behavior. What is behavior? When should you be monitoring the behavior? When is the behavior occurring most frequently? Selecting the most appropriate tool what SHOULD the student be doing instead? Ask them for an example think about one of your kids An efficient way to measure progress
Able to be administered repeatedly & efficiently, Measure specific goals, Sensitive to small changes, Summarized in graph format, Able to compare progress acrossstudents The behavior report card is generally used at Tier 2, but theres nothing saying it cant also be used at Tier 3. If schools arent doing ANYTHING to progress monitor behavior goals at Tier 3, this may be something they can use to get their feet wet. This is also a good tool for emphasizing the connection to school-wide expectations. 4/27/2017 8:13 AM Walk through the displayed forms. The form on the left is the SWIS Referral Form Compatibility Checklist and the form on the right is an example Office Discipline Referral Form. Demonstrate how the example office discipline referral form meets the requirements on the compatibility checklist. More information and examples are available atin the Resources section More information and examples are available atin the Resources section 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. FLPBS Tier 2 Progress Monitoring Tool
FLBPS homepage; Resources; Tier 2; Progress Monitoring; Spreadsheet Is the student making progress?
And get the information quickly. The Behavior Rating Scale (BRS)
If one of the monitoring tools you chose is the BRS, this is what it looks like. Youve already defined your behavior Use behavior anchors handout to further describe. Remember that anchor is basically just a rating. Just about anything that you can fit into a category will fit here. Its been proven to work- you can read the article on our website As you fill out, the teacher gets immediate feedback Then enter and get the overall bigger picture Think about your kids behavior yesterday, does this make sense? Behavior Rating Scale Questionable response: more sensitive data will help identify progress & patterns in behavior. Iovannone, Greenbaum, Wang, Dunlap & Kincaid (2013). Inter-rater Agreement of the Individualized Behavior Rating Scale Tool. Assessment for Effective Intervention , published online 16 May DOI: / FBA/BIP within a problem-solving framework has been established as an effective process for supporting individualized student needs. (Cook et al., 2012; Filter & Horner, 2009; Ingram, Lewis-Palmer, & Sugai, 2005; Newcomer & Lewis, 2004) Unfortunately, the FBA/BIP process is often of poor quality and compliance driven (e.g., IDEIA, 2004). (Blood & Neel, 2007; Conroy et al., 2002; Cook, et al., 2007; Van Acker et al., 2005) Implementation Fidelity
If students arent responding positively Are Tier 3 procedures, assessments and interventions being implemented with fidelity? Fidelity of Implementation
Necessary to evaluate student response to intervention Considerations: All essential intervention components are identified Each component is implemented correctly/accurately Intervention is implemented for the appropriate length of time Dosage: How often (frequency), how long (duration) Example Fidelity Checklist: PTR
Interventions Implemented Impact (1 = no impact; 5 = great impact) PREVENT Transition Supportsvisual checklist Visual checklist provided to Isaiah Choice of reinforcement presented and described on checklist Y / N / NA TEACH Replacement behavioracademic engagement Checklist reviewed during study skills class Goal set Gave 1 minute at end of class for Isaiah to self-assess Reviewed Isaiahs self-assessment and gave feedback Replacement behaviorescape by asking to be excused Prior to non-preferred activity, provided a verbal prompt/cue to remind Isaiah that he can ask to be excused. REINFORCE Reinforce academic engagement Presented choice reinforcement menu to Isaiah when goal met Provided verbal praise Provided reinforcement for surpassing goal Reinforce asking to be excused Provide 1 minute break each time Isaiah asks to be excused Discontinue reinforcement of problem behavior Got Isaiahs attention and used agreed upon signal when Isaiah stops Waited for Isaiahs attending response Tapped activity on teacher copy of checklist to remind Isaiah to be engaged Sidebar in hallway if Isaiah stops again Behavior Plan Assessment:Y/Y + N total/Mean impact (Dunlap et aI., 2010) Example Fidelity Checklist
Intervention Components M T W Th F Component Fidelity 1. 2. 3. 4. 5. 6. 7. 8. Daily Integrity Michigan's Integrated Behavior and Learning Support Initiative (MiBLSi) is a Mandated Activities Project (MAP), funded under theIndividuals with Disabilities Education Act(IDEA) through the Michigan Department of Education, Office of Special Education. Cook, C.R., Wright, D.B., Gresham, F.M., & Burns, M. K. (2010). Transforming School Psychology in the RtI Era: A Guide for Administrators and School Psychologists. Horsham, PA: LRP Publications Example Self-Assessment Form: Classroom Management
Classroom Management Practice Rating 1. I have arranged my classroom to minimize crowding and distraction YesNo 2. I have maximized structure and predictability in my classroom (e.g., explicit classroom routines, specific directions, etc.). 3. I have posted, taught, reviewed, and reinforced 3-5 positively stated expectations (or rules). 4. I provided more frequent acknowledgement for appropriate behaviors than inappropriate behaviors (See top of page). 5. I provided each student with multiple opportunities to respond and participate during instruction. Yes No 6. My instruction actively engaged students in observable ways (e.g., writing, verbalizing) 7. I actively supervised my classroom (e.g., moving, scanning) during instruction. 8. I ignored or provided quick, direct, explicit reprimands/redirections in response to inappropriate behavior. 9. I have multiple strategies/systems in place to acknowledge appropriate behavior (e.g., class point systems, praise, etc.). 10. In general, I have provided specific feedback in response to social and academic behavior errors and correct responses. Overall classroom management score: 10-8yes=Super 7-5yes=So-So