Upload
lora-caldwell
View
219
Download
6
Tags:
Embed Size (px)
Citation preview
Elementary Social Behavior Assessment: Integrating Universal Screening and Progress Monitoring to Measure Behavioral Response to Intervention
Jeffrey Sprague, Brion Marquez, Pamela Yeaton, Jessie Marquez, Jordan Pennefather, & Claudia Vincent
IRIS Educational Media
Providing school communities with research-based training and tools that support positive educational outcomes.
Agenda
• Need for universal screening and progress monitoring
• Utility of existing tools• Challenges in monitoring student social
behavior• Efficacy of teacher judgment of student
social behavior• Description of the ESBA delivered via
the irisPMTTM
Universal Screening and Progress Monitoring
• The Response-to-Intervention (RtI) framework requires teachers to:– Identify students
• who make typical progress towards an instructional goal
• who are at risk of falling behind their typical peers• who have significant difficulties in meeting the
instructional goal– Provide students with the necessary support before they fail
• RtI applies to academic as well as social skills instruction
RtI FrameworkAcademic Supports Behavioral Supports
1-5% 1-5%
5-10% 5-10%
80-90% 80-90%
Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity
Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures
Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response
Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response
Universal Interventions•All students•Preventive, proactive
Universal Interventions•All settings, all students•Preventive, proactive
0-1 ODR
2-5 ODR
6+ ODR
RtI FrameworkCommonly used academic
measuresCommonly used behavioral
measures
• DIBELS• AIMSweb• Acuity• easyCBM
See: http://www.rti4success.org/screeningTools
• Office discipline referrals• E.g., School-wide Information
System www.swis.org • Behavioral rating scales
• E.g., Systematic Screening for Behavior Disorders www.sopriswest.com
• Daily behavior report cards • (see Riley-Tilman, Kalberer, &
Chafouleas, 2005)• Direct behavioral observations
RtI FrameworkCommonly used academic
measuresCommonly used behavioral
measures
See: http://www.rti4success.org/screeningTools
• What do you currently use to identify students’ risk level?
• What do you like about the measures you are using?
• What features do you wish your measures had?
Challenges in Universal Screening and Progress Monitoring for Behavioral
Adjustment
• Need for assessment tool that is– Cost-efficient– Accurate– Socially valid– Easy to use– (see Walker, Marquez, Yeaton & Pennefather, 2012)
Goals of Universal Screening
• Fast, efficient, and respectful
• Include all children and youth of interest• Identify students for further assessment that
ARE at-risk– If we make a screening error, the error should
identify students that are not at-risk– Errors should not overlook students that are at-
risk
Important Guidelines• Ensure each student is assessed• Respectful and non-stigmatizing language• Identify students with internalizing as well as
externalizing behavior• Adaptable to variations in school schedules
and teacher preferences• Required teacher time and effort is
reasonable
Universal behavioral screening using office discipline referrals
• Advantage– Most schools track these already
• Disadvantages– “Wait to fail”– Misses “internalizers”– Teacher/system bias – inconsistent use of the
protocol
Universal behavioral screening using office discipline referrals
• Teacher-motivated referrals (Gerber & Semmel, 1984)
– Argument One - Teacher desires to be rid of troublesome, difficult-to-teach students
– Argument Two - Teacher desires to secure assistance for students whose problems and needs exceed teacher’s skill level and accommodation capacity
Universal behavioral screening using office discipline referrals
Sample Decision Rules
• < 1 – remain in Tier I, universal supports
• 2-3 ODRs – on the radar
• > 3 ODRs – in need of Tier II, secondary supports
Jeffrey Sprague, Ph.D. [email protected]
For example: The School-wide Information System (SWIS)
13
1. Who should be on an intervention based on these data from the previous school year?
2. Whom should we monitor carefully?
3. Who requires a new intervention/evaluation?
Decision Rule
Universal behavioral screening using office discipline referrals
• ODR tracking systems (e.g. SWIS), often recommended in conjunction with school-wide positive behavior support implementation, focus on negative behaviors
• Students with ODR have already failed behaviorally
Use of behavior rating scales
• Allows longitudinal tracking of “at-risk” students if a consistent rating schedule is followed (recommended 3X/year)– Corroborating evidence can be found in archival
records such as office referrals, suspensions, attendance (individual), grades
• Identified students should also be “progress monitored” using measures linked to the rating scales
Use of behavior rating scales
Although behavior rating scales are one of the most commonly used measures of social-emotional behavior, such measures were traditionally developed for diagnostic purposes (i.e., measuring existing symptoms of a diagnosable disorder) rather than for identifying present or future risk, an important aim of universal mental health screening (Albers, Glover, & Kratochwill, 2007).
Jeffrey Sprague, Ph.D. [email protected]
Universal behavioral screening using behavioral rating scales
Use of behavioral rating scales
• Completing behavioral rating scales can be time intensive – e.g.: Systematic Screener for Behavior Disorders:
about 45 minutes for a class of 25 students• Few scales are available on-line– Data might be difficult to aggregate
• Some on-line scales might not maintain required confidentiality– (e.g. Strengths and Difficulties Questionnaire:
www.sdqinfo.com)
Universal screening based on teacher judgment
• Teachers are the best judges of student behavior• Teachers commonly compare students to each
other to identify those who – meet behavioral goals
– have difficulty meeting behavioral goals
– need significant support in meeting behavioral goals
• Comparative ratings of students are more meaningful for teachers than frequency or duration of specific behaviors
Universal screening needs identified by one teacher
• “I need…– A CBM-like method to screen ALL students
behaviorally– A quick and efficient approach
– Immediate results to inform instruction
The irisPMTTM
• A web-based application to – screen entire classrooms– progress monitor individual students
• Quick and efficient– 30 min per class (15 min once familiar with tool)
• Easily interpretable data summaries for effective decision-making
• Seamless continuity between universal screening and progress-monitoring
• Developed through 2 IES-funded projects focused on (a) social skills intervention and (b) classroom management training
The irisPMTTM
• Scoring scale:– Universal screening:• 3-point scale:
– 3 = mastery– 2 = needs improvement– 1 = cause for concern
– Progress monitoring• 6-point scale:
– 6 = responsive to intervention– 1 = non-responsive to intervention
The irisPMTTM
• Ability to house various scales, e.g.:– Elementary Social Behavior Assessment (ESBA)
– 12-items measuring behavioral competencies teachers associate with school success » (derived from over 30 years of research by Walker and
colleagues)– Positively phrased: teachers rate how well a student is
performing, rather than how poorly– Good psychometric properties
» (internal consistency, rest-retest reliability, criterion validity with Walker-McConnell Scale of Social Competence and School Adjustment and Brief Behavior Rating Scale, see Pennefather & Smolkowski, 2012)
The irisPMTTM
• Social validity– Teachers found the irisPMT • easy to use• relevant for decision-making
• Meets confidentiality and data security standards– Different users have different access privileges• E.g. teacher (own classroom), administrator (all
classrooms in a school)
Using the irisPMTTM to for universal screening
• Enter class roster – import from existing spreadsheets
• Rate each student on each of the 12-items of the ESBA (screen by student)– Or
• Rate how well each skill is performed by students (screen by skill)
• Review report
12 skills assessed through the ESBA
• Listens to and respects the teacher
• Follows the teacher’s directions
• Works with effort• Does seatwork
assignments as directed• Makes assistance known
in an appropriate manner• Follows rules
• Avoids breaking rules even when encouraged by a peer
• Behaves appropriately outside the classroom
• Works out strong feelings• Can have “normal”
conversations without becoming hostile
• Gets along with peers• Resolves peer conflicts
without teacher assistance
irisPMT dashboard: Teacher
irisPMT dashboard: Administrator
Universal Screening: By student
Universal Screening: By skill
Universal Screening Report
Overall Student Scores
Overall Skill Scores
Focus on individual students
Focus on individual skills
Focus on skills and students
Universal Screening Report Interpretation
• Lots of red: many students do not perform many of the target skills– Teacher needs to re-teach behavioral
expectations– Teacher needs to focus on teaching specific
skills– Re-teach and re-screen
Universal Screening Report
Universal Screening Report Interpretation
• Lots of green: most students perform most of the target skills– A few students might benefit from more
support in the form of a targeted intervention– Progress-monitor individual students
Decision-making based on universal screening outcomes: Teacher
• Based on a 15-minute screening, the teacher knows– How the class performs behaviorally– What changes he/she needs to make in the class– Which students need support
Universal Screening Outcomes: Administrator
Decision-making based on universal screening outcomes: Administrator
• Administrator– Which classrooms need support– Where to allocate resources
Decisions result in actions: Additional support for some students
Assign specific students to Tier 2 support– Overall behavior rankings
– Comparative data from US 1 to US 2
– Approximately lowest 20% of students
– Progress monitor students receiving Tier 2 support
Using the irisPMT for Progress Monitoring
Progress monitoring Report
Progress Monitoring Report: Interpretation: Flat lines
Progress Monitoring Report: Interpretation: Trend is improving
Progress Monitoring Report: Interpretation: Maintained mastery
Summary
• Universal screening and progress monitoring are essential elements of the RtI framework
• Existing tools tend to – measure irrelevant constructs (e.g., ODR) – have difficult to interpret scales– be time and/or cost-intensive
• Teachers need – quick, easy-to-use measure of relevant constructs
that generates intuitively interpretable reports • The irisPMTTM delivers what teachers need
Future Developments
• irisPMT has the capacity to house multiple scales– Merge behavioral and academic assessments
• irisPMT has the capacity to accommodate multiple users with different access privileges– Create opportunities for students to self-monitor– Create opportunities for parents to rate their children on
behaviors of interest (e.g. homework completion)– Compare ratings across raters, promote home-school
collaboration• irisPMT has the capacity to provide teachers with a
comprehensive assessment portfolio
For more information
Providing school communities with research-based training and tools that support positive educational outcomes.
www.irismedia.com
www.irispmt.com