Elementary Social Behavior Assessment: Integrating Universal Screening and Progress Monitoring to...

Elementary Social Behavior Assessment: Integrating Universal Screening and Progress Monitoring to Measure Behavioral Response to Intervention

Jeffrey Sprague, Brion Marquez, Pamela Yeaton, Jessie Marquez, Jordan Pennefather, & Claudia Vincent

IRIS Educational Media

Providing school communities with research-based training and tools that support positive educational outcomes.

Agenda

• Need for universal screening and progress monitoring

• Utility of existing tools• Challenges in monitoring student social

behavior• Efficacy of teacher judgment of student

social behavior• Description of the ESBA delivered via

the irisPMTTM

Universal Screening and Progress Monitoring

• The Response-to-Intervention (RtI) framework requires teachers to:– Identify students

• who make typical progress towards an instructional goal

• who are at risk of falling behind their typical peers• who have significant difficulties in meeting the

instructional goal– Provide students with the necessary support before they fail

• RtI applies to academic as well as social skills instruction

RtI FrameworkAcademic Supports Behavioral Supports

1-5% 1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual Interventions•Individual Students•Assessment-based•High Intensity

Intensive, Individual Interventions•Individual Students•Assessment-based•Intense, durable procedures

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Targeted Group Interventions•Some students (at-risk)•High efficiency•Rapid response

Universal Interventions•All students•Preventive, proactive

Universal Interventions•All settings, all students•Preventive, proactive

0-1 ODR

2-5 ODR

6+ ODR

RtI FrameworkCommonly used academic

measuresCommonly used behavioral

measures

• DIBELS• AIMSweb• Acuity• easyCBM

See: http://www.rti4success.org/screeningTools

• Office discipline referrals• E.g., School-wide Information

System www.swis.org • Behavioral rating scales

• E.g., Systematic Screening for Behavior Disorders www.sopriswest.com

• Daily behavior report cards • (see Riley-Tilman, Kalberer, &

Chafouleas, 2005)• Direct behavioral observations

RtI FrameworkCommonly used academic

measuresCommonly used behavioral

measures

See: http://www.rti4success.org/screeningTools

• What do you currently use to identify students’ risk level?

• What do you like about the measures you are using?

• What features do you wish your measures had?

Challenges in Universal Screening and Progress Monitoring for Behavioral

Adjustment

• Need for assessment tool that is– Cost-efficient– Accurate– Socially valid– Easy to use– (see Walker, Marquez, Yeaton & Pennefather, 2012)

Goals of Universal Screening

• Fast, efficient, and respectful

• Include all children and youth of interest• Identify students for further assessment that

ARE at-risk– If we make a screening error, the error should

identify students that are not at-risk– Errors should not overlook students that are at-

risk

Important Guidelines• Ensure each student is assessed• Respectful and non-stigmatizing language• Identify students with internalizing as well as

externalizing behavior• Adaptable to variations in school schedules

and teacher preferences• Required teacher time and effort is

reasonable

Universal behavioral screening using office discipline referrals

• Advantage– Most schools track these already

• Disadvantages– “Wait to fail”– Misses “internalizers”– Teacher/system bias – inconsistent use of the

protocol

Universal behavioral screening using office discipline referrals

• Teacher-motivated referrals (Gerber & Semmel, 1984)

– Argument One - Teacher desires to be rid of troublesome, difficult-to-teach students

– Argument Two - Teacher desires to secure assistance for students whose problems and needs exceed teacher’s skill level and accommodation capacity

Universal behavioral screening using office discipline referrals

Sample Decision Rules

• < 1 – remain in Tier I, universal supports

• 2-3 ODRs – on the radar

• > 3 ODRs – in need of Tier II, secondary supports

Jeffrey Sprague, Ph.D. jeffs@uoregon.edu

For example: The School-wide Information System (SWIS)

13

1. Who should be on an intervention based on these data from the previous school year?

2. Whom should we monitor carefully?

3. Who requires a new intervention/evaluation?

Decision Rule

Universal behavioral screening using office discipline referrals

• ODR tracking systems (e.g. SWIS), often recommended in conjunction with school-wide positive behavior support implementation, focus on negative behaviors

• Students with ODR have already failed behaviorally

Use of behavior rating scales

• Allows longitudinal tracking of “at-risk” students if a consistent rating schedule is followed (recommended 3X/year)– Corroborating evidence can be found in archival

records such as office referrals, suspensions, attendance (individual), grades

• Identified students should also be “progress monitored” using measures linked to the rating scales

Use of behavior rating scales

Although behavior rating scales are one of the most commonly used measures of social-emotional behavior, such measures were traditionally developed for diagnostic purposes (i.e., measuring existing symptoms of a diagnosable disorder) rather than for identifying present or future risk, an important aim of universal mental health screening (Albers, Glover, & Kratochwill, 2007).

Jeffrey Sprague, Ph.D. jeffs@uoregon.edu

Universal behavioral screening using behavioral rating scales

Use of behavioral rating scales

• Completing behavioral rating scales can be time intensive – e.g.: Systematic Screener for Behavior Disorders:

about 45 minutes for a class of 25 students• Few scales are available on-line– Data might be difficult to aggregate

• Some on-line scales might not maintain required confidentiality– (e.g. Strengths and Difficulties Questionnaire:

www.sdqinfo.com)

Universal screening based on teacher judgment

• Teachers are the best judges of student behavior• Teachers commonly compare students to each

other to identify those who – meet behavioral goals

– have difficulty meeting behavioral goals

– need significant support in meeting behavioral goals

• Comparative ratings of students are more meaningful for teachers than frequency or duration of specific behaviors

Universal screening needs identified by one teacher

• “I need…– A CBM-like method to screen ALL students

behaviorally– A quick and efficient approach

– Immediate results to inform instruction

The irisPMTTM

• A web-based application to – screen entire classrooms– progress monitor individual students

• Quick and efficient– 30 min per class (15 min once familiar with tool)

• Easily interpretable data summaries for effective decision-making

• Seamless continuity between universal screening and progress-monitoring

• Developed through 2 IES-funded projects focused on (a) social skills intervention and (b) classroom management training

The irisPMTTM

• Scoring scale:– Universal screening:• 3-point scale:

– 3 = mastery– 2 = needs improvement– 1 = cause for concern

– Progress monitoring• 6-point scale:

– 6 = responsive to intervention– 1 = non-responsive to intervention

The irisPMTTM

• Ability to house various scales, e.g.:– Elementary Social Behavior Assessment (ESBA)

– 12-items measuring behavioral competencies teachers associate with school success » (derived from over 30 years of research by Walker and

colleagues)– Positively phrased: teachers rate how well a student is

performing, rather than how poorly– Good psychometric properties

» (internal consistency, rest-retest reliability, criterion validity with Walker-McConnell Scale of Social Competence and School Adjustment and Brief Behavior Rating Scale, see Pennefather & Smolkowski, 2012)

The irisPMTTM

• Social validity– Teachers found the irisPMT • easy to use• relevant for decision-making

• Meets confidentiality and data security standards– Different users have different access privileges• E.g. teacher (own classroom), administrator (all

classrooms in a school)

Using the irisPMTTM to for universal screening

• Enter class roster – import from existing spreadsheets

• Rate each student on each of the 12-items of the ESBA (screen by student)– Or

• Rate how well each skill is performed by students (screen by skill)

• Review report

12 skills assessed through the ESBA

• Listens to and respects the teacher

• Follows the teacher’s directions

• Works with effort• Does seatwork

assignments as directed• Makes assistance known

in an appropriate manner• Follows rules

• Avoids breaking rules even when encouraged by a peer

• Behaves appropriately outside the classroom

• Works out strong feelings• Can have “normal”

conversations without becoming hostile

• Gets along with peers• Resolves peer conflicts

without teacher assistance

irisPMT dashboard: Teacher

irisPMT dashboard: Administrator

Universal Screening: By student

Universal Screening: By skill

Universal Screening Report

Overall Student Scores

Overall Skill Scores

Focus on individual students

Focus on individual skills

Focus on skills and students

Universal Screening Report Interpretation

• Lots of red: many students do not perform many of the target skills– Teacher needs to re-teach behavioral

expectations– Teacher needs to focus on teaching specific

skills– Re-teach and re-screen

Universal Screening Report

Universal Screening Report Interpretation

• Lots of green: most students perform most of the target skills– A few students might benefit from more

support in the form of a targeted intervention– Progress-monitor individual students

Decision-making based on universal screening outcomes: Teacher

• Based on a 15-minute screening, the teacher knows– How the class performs behaviorally– What changes he/she needs to make in the class– Which students need support

Universal Screening Outcomes: Administrator

Decision-making based on universal screening outcomes: Administrator

• Administrator– Which classrooms need support– Where to allocate resources

Decisions result in actions: Additional support for some students

Assign specific students to Tier 2 support– Overall behavior rankings

– Comparative data from US 1 to US 2

– Approximately lowest 20% of students

– Progress monitor students receiving Tier 2 support

Using the irisPMT for Progress Monitoring

Progress monitoring Report

Progress Monitoring Report: Interpretation: Flat lines

Progress Monitoring Report: Interpretation: Trend is improving

Progress Monitoring Report: Interpretation: Maintained mastery

Summary

• Universal screening and progress monitoring are essential elements of the RtI framework

• Existing tools tend to – measure irrelevant constructs (e.g., ODR) – have difficult to interpret scales– be time and/or cost-intensive

• Teachers need – quick, easy-to-use measure of relevant constructs

that generates intuitively interpretable reports • The irisPMTTM delivers what teachers need

Future Developments

• irisPMT has the capacity to house multiple scales– Merge behavioral and academic assessments

• irisPMT has the capacity to accommodate multiple users with different access privileges– Create opportunities for students to self-monitor– Create opportunities for parents to rate their children on

behaviors of interest (e.g. homework completion)– Compare ratings across raters, promote home-school

collaboration• irisPMT has the capacity to provide teachers with a

comprehensive assessment portfolio

For more information

Providing school communities with research-based training and tools that support positive educational outcomes.

www.irismedia.com

www.irispmt.com