Online Universal Screening/Progress Monitoring to Measure Response to

Social Skills Intervention

2012 National PBIS Leadership ForumOct 19 10:55-12:10

Jeffrey Sprague, Ph.D., Principal Investigator,

University of Oregon

∙

Calli Dean, Behavior Specialist, Thurston

School District, Springfield Oregon

∙

Jessie Marquez, Instructional Designer

Pamela Yeaton, Co-Investigator

Brion Marquez, Principal Investigator

IRIS Educational Media, Eugene, OR

Efficient and engaging delivery of evidence-based social skills lessons combined with universal screening and continuous progress monitoring will result in better student outcomes and improved teacher efficacy and effectiveness in early elementary school settings.– Walker , Marquez, Yeaton et al; in press– Marquez, Marquez, Vincent, Pennefather, Yeaton,

Sprague; in press

Purpose of presentation

Report on an K-3 social/behavioral intervention that used online screening and progress monitoring (irisPMT™) in conjunction with media-based social skills instruction (We Have Skills!). This intervention was developed by IRIS Educational Media with support from a grant

by the Institute of Education Sciences.

The following slides contain links to videos that describe:

• The social skills intervention: We Have Skills!

• The universal screening and progress monitoring tool—irisPMT™.

This introductory video provides an overview of the social skills curriculum and the tool that measures

student response to the curriculum.

The premise of We Have Skills! is that students can learn social skills through direct and explicit instruction. We recommend that teachers first use the program as universal classroom instruction as shown in this video.

This video shows how universal screening for behavioral skills gives teachers valuable information about what students have learned, who needs more

help, and what areas of instruction to target.

Watch how targeted instruction (Tier II support) ensures that each student gets the level of support

that they need.

Students who are being progress monitored receive targeted support provides teachers with data about

student learning.

Development

• We Have Skills! and the irisPMT™ were developed using an iterative process that involved practitioners, students, parents, school personnel, and research scientists.

• Technology applications were tested for usability.

• In 2010, we conducted an evaluation of We Have Skills! with the irisPMT™

Evaluation

• Randomized waitlist controlled trial with 70 K-3 classrooms, randomly assigned to condition, including 1616 students (Marquez et al., 2012).

• Intervention group received training and access to intervention components; control group conducted business as usual and received the intervention at the end of the study.

• Teachers in the intervention condition taught one social skills lesson a week for 8 weeks. Teachers in both conditions used the irisPMT™ to screen students’ social skills.

Measures

• Teacher Sense of Efficacy Scale-Short Form (TSES; Tschannen-Moran & Woolfolk Hoy, 2001).

• Teachers completed a pre and post-intervention universal screening of their students using the irisPMT™

• Additional student measures at posttest included the Brief Behavior Rating Scale (BBRS; Gresham et al., 2010).

Measures (continued)

• Teacher attitudes, Stages of Concern (SOC; Hall, George, & Rutherford, 1986)

• User satisfaction (Rusby, Taylor, & Marquez, 2004)

• Technology acceptance (TAM; Davis, Bagozzi, & Warshaw, 1989).

Results

• Improved student social skills.

– Students in the intervention condition (M = 32.8) scored significantly higher than students in the control condition (M = 32.2, F(1,58) = 6.97, p = .011).

– Students in the intervention condition improved

by an average of 3.5 points on the PMT universal screening assessment while students in the control condition improved by 1.7 points.

Results (continued)

• Improved teacher self efficacy.

– Teachers in the intervention condition reported greater gains (M = .54) in self-efficacy, measured by the TSES, than teachers in the control condition (M = .13; F(1, 67) = 4.82, p = .032 ).

Results (continued)

• Consumer satisfaction on We Have Skills!– Teachers were very satisfied with the student materials,

with a mean consumer satisfaction score of 5.39 out of 6.

– 100% of teachers said they would recommend the overall program to other teachers (56% said strongly recommend) and 100% said they were likely to use the program in the classroom (59% reported the highest likelihood).

• Consumer satisfaction on the irisPMT™– Overall satisfaction with 3.957 out of 6; significantly

higher than neutral (3.5), t(34) = 2.615, p < .05.

Results (continued)

• PMT Scale Analysis

– The PMT demonstrated strong internal reliability (Cronbach’s alpha = .95).– Student post-intervention PMT scores significantly correlated with the

BBRS (r = -.766; F(1, 1547) = 2194.53, p < .001)

– Another study with 187 elementary school students also demonstrated a significant correlation of the PMT (r = .83, F(1,185) = 423.5, p < .001) with the Walker McConnell Scale of Social Competence and School Adjustment (Walker & McConnell, 1995).

For more information

• Click on We Have Skills!

• Click on irisPMT™

Providing school communities with research-based training and tools that support positive educational outcomes.