PBS Year 1 Report Final - Loudoun County Public SchoolsFinal).pdf · Positive Behavioral Support ... The findings of this study provided strong supporting evidence that the PBS

Program Evaluation

Positive Behavioral Support Framework (First Year)

Research Office

Loudoun County Public Schools

Chad Green, PMP, Program Analyst Stephan Knobloch, Ed.D., Director of Research

March 10, 2009

Research Office 2 Department of Instruction 03/30/09

Program Evaluation: PBS Framework (First Year)

Table of Contents Executive Summary.......................................................................................................................................................3 Background Information................................................................................................................................................4 Methodology..................................................................................................................................................................7 Results ...........................................................................................................................................................................8 Conclusions ...................................................................................................................................................................9 Recommendations .........................................................................................................................................................9 References ...................................................................................................................................................................10 Appendix A: PBS Data Collection Instruments...........................................................................................................11 Appendix B: Tables .....................................................................................................................................................21 Appendix C: Figures....................................................................................................................................................24 Appendix D: Sample Artifacts from the First Year .....................................................................................................26



Executive Summary The Positive Behavioral Support (PBS) initiative trains school-based teams of educators in the areas of systems thinking, effective behavior management, and research-driven instructional practices that promote socially appropriate student behaviors in classroom and non-classroom settings. Implementation of the three-tiered PBS framework is built around the following critical implementation steps: team formation, clarification of roles, development of a data-based PBS action plan, implementation of the plan, monitoring and evaluation of interventions, and finally, revisions to the plan for continuous process improvement. PBS was piloted in Loudoun County Public Schools (LCPS) at five schools during the 2007-08 school year. In part, the initiative was intended to address the following School Board goal: “LCPS will strengthen programs that support safe learning environments free of bullying, teasing, and physical violence.” PBS is currently supported by a district-level PBS Coordination Team and funded by the Department of Pupil Services. During its first year of operation, the PBS Coordination Team provided workshop training and follow-up on-site coaching with a focus on school-wide, universal interventions in non-classroom settings. A number of strategies were used to recruit and train the five pilot school teams on the PBS implementation process. These included introductory presentations about PBS, an initial two-day training for the selected school teams, on-site coaching by trained facilitators, ongoing communication of updates and promising practices, and collaborations throughout LCPS and with external PBS technical assistance providers. The purpose of this study was to compare the actual implementation of PBS with the first-year goals developed through the program’s logic model. The short-term goals served as a conceptual framework that directed the collection and analysis of multiple sources of evidence. Categorized into process and outcome indicators, these expectations defined an effective first-year program in the 2007-08 school year. The findings of this study provided strong supporting evidence that the PBS Coordination Team and school-based teams achieved eight of the nine performance indicators set at the beginning of the school year. One school-level outcome indicator had a mixed level of supporting evidence. It is important to note, however, that these expectations are subject to change as PBS is expanded into the second and third tiers of the framework. Therefore the indicators of an effective PBS implementation may similarly evolve given the nature of continuous process improvement. Five recommendations are provided to facilitate this capacity-building process.

Category

Indicators of an Effective PBS Implementation (First Year)

Level of Supporting Evidence

1) Participation of school behavior support teams in two full-day training sessions will be 90%. Participants will be satisfied with the quality of training provided. Strong

2) PBS coaches will attend monthly school team meetings, providing support for data-based decision-making, planning, and implementation. Strong

3) The PBS Coordinator will facilitate communication and dissemination of PBS best practices to school teams through regular contact with the coaches and bi-monthly newsletters.

Strong Process

4) The PBS Coordination Team will complete a district PBS action plan to sustain the growth of the program. Strong

5) Using the results of the School-wide Evaluation Tool, school teams will complete an action plan and review progress quarterly. Strong

6) School teams will attain a 60% summary score at a minimum on their PBS Team Implementation Checklists. Strong

7) School teams will present school referral data to their staff monthly in order to share successes and elicit feedback. Mixed

8) School teams will attain a 60% summary score at a minimum on the year-end School-wide Evaluation Tool. Strong

Outcome

9) School teams will determine a baseline office referral rate (per 100 students/year) for benchmarking purposes. Strong



Background Information Positive Behavioral Support (PBS) is a general term that refers to “a set of processes that combine information from social, behavioral, and biomedical science and applies this information at the individual and/or systems level to reduce behavioral challenges and improve quality of life” (APBS, 2007). PBS interventions can be used by a wide range of service providers to support the development of positive behaviors in both school and out-of-school contexts (e.g., family, community). This section of the report describes school-wide PBS in general, followed by a summary of the first-year implementation of the training program in LCPS during the 2007-08 school year. Definition of School-wide PBS According to the Loudoun County Public Schools (LCPS) website, school-wide PBS is defined as a “process for systemic problem solving, planning, and evaluation” that trains school-based teams of educators in (a) systems thinking along a three-tiered continuum of interventions, (b) effective behavior management principles and procedures, and (c) research-driven instructional practices that promote socially appropriate student behaviors in classroom and non-classroom settings. The overarching purpose of school-wide PBS is to design and sustain safe learning environments for all students “by making problem behavior less effective, efficient, and relevant, and desired behavior more functional” (Sugai et al., 2000). The systems perspective of PBS facilitates this process by organizing behavioral support within a three-tiered continuum as follows (see Figure 1):

• Universal primary interventions aim to reduce new cases of problem behavior by promoting appropriate or pro-social behaviors among all students in classroom and non-classroom settings.

• Targeted secondary interventions are designed to reduce current cases of problem behavior among groups of students who do not respond to school-wide strategies.

• Intensive tertiary interventions seek to reduce the intensity, complications, and/or severity of problem behavior among individual students who do not respond to either school-wide or targeted strategies.

Over a period of at least three years, PBS schools are expected to review and enhance their policies, structures, routines, and environments (classroom and non-classroom), as well as influence the attitudes and behaviors of students, parents, teachers, and staff, so that appropriate behaviors are emphasized and problem behaviors are prevented along this continuum (Sugai et al., 2000). During the first year of implementation, PBS schools typically focus on school-wide, universal interventions in classroom and/or non-classroom settings. In subsequent years, secondary and tertiary systems are established to accommodate students who do not respond to the universal interventions. Currently, about 8,000 schools in 47 states have adopted the PBS framework (Spaulding, Horner, May, & Vincent, 2008). Support for the initiative is coordinated by the National Technical Assistance Center on Positive Behavioral Interventions and Supports, a consortium of service providers funded by the Office of Special Education Programs. Implementation Process Implementation of school-level PBS is built around five critical implementation steps (see Figure 2). Schools first form a school-based team of 6-8 stakeholders who are representative of the school community. This “behavior support team” typically comprises a school administrator, special and general education teachers, teaching assistants, and parents. In addition, a dedicated PBS coach (school- or district-based) serves as the primary point of contact with a PBS coordination team at the district level. During the process of team formation, participants next agree on their respective roles to ensure active participation and equal distribution of work. Common roles include Team Leader, Recorder, Timekeeper, Communications, Administrator, Behavior Specialist, PBS Coach, and Data Specialist. The primary function of the PBS Coach is to assist with implementing the five steps rather than serving as Team Leader. Once the roles are established, the team completes a school-wide PBS action plan with an initial focus on universal primary interventions (tier one). To assist teams with this process, the Center on Positive Behavioral Interventions and Supports provides a number of instruments for self assessment and action planning (see Appendix A). The most



critical tools are the School-wide Evaluation Tool (SET), a pre-/post-assessment completed by a trained external evaluator each year, and the Team Implementation Checklist (TIC), a quarterly self assessment and action plan created and updated by the teams themselves. Scores on both assessments can be compared to verify the validity of the implementation scores. To complete the PBS action plan, team members must define three to five universal behavioral expectations for students in classroom and/or non-classroom settings (e.g., respect, responsibility, safety). This “teaching matrix” of expectations is then introduced and modeled by school staff along with an explanation of the consequences for unacceptable behavior. To promote the expectations, the behavior support team also develops activities that publicly recognize students for demonstrating the positive behaviors. Implementation of the plan is monitored for effectiveness through monthly reviews of behavior data, typically in the form of classroom (minor) and office (major) discipline referrals. Using problem-solving templates to interpret the data, the team completes the PBS implementation cycle by acting on the evaluation results to revise their action plans. Implementation of the school-wide PBS also requires a leadership team at the district level (see Figure 3). This team of central office stakeholders assumes responsibility for sustaining program resources, promoting visibility throughout the district, and monitoring progress through an action plan and implementation checklist. A critical measure of success for the coordination team is the development of a core group of PBS demonstration sites. Program Delivery in the First Year The PBS framework in LCPS was piloted at four elementary schools and one middle school in the 2007-08 school year (see Table 1). In part, the selection of the initiative was intended to address the following School Board goal: “LCPS will strengthen programs that support safe learning environments free of bullying, teasing, and physical violence.” The program is currently managed by a district-level PBS Coordination Team and funded by the Department of Pupil Services. During its first year of operation, the PBS Coordination Team provided workshop training and follow-up on-site coaching with a focus on school-wide, universal interventions in non-classroom settings (see Table 2). The following strategies were utilized by program staff to recruit and train the five school teams on the PBS implementation process: Awareness and recruitment of pilot schools: In December 2006 and February 2007, all school principals and central office administrators in LCPS were invited to attend introductory presentations about PBS. Information about the framework’s essential features, implementation process, data-driven decision making focus, use of evidence-based practices, and success stories were shared and discussed. In addition, the Assistant Superintendent of Pupil Services, Director of Special Education, and Director of Student Services were involved in disseminating information and supporting schools that expressed interest in PBS. Based on student demographics and feedback from the principals, five pilot schools were selected to pilot the program (see Table 1). With the support of the school’s leadership, school faculty and staff from these schools attended a PBS presentation and obtained at least 80% agreement to implement PBS school-wide for at least three years. Recruitment and training of coaches: The PBS Coordination Team, consisting of central office administrators from the Department of Pupil Services, identified PBS coaches from within the Department based on their educational background, experience and interest in the program. Central office staff who received coaching assignments included social workers, consulting teachers, a school psychologist, and an educational diagnostician. These individuals participated in a separate training designed specifically for PBS coaches, which focused on research-based promising practices in PBS. During the two-day school team training, the coaches developed relationships with their assigned school teams and sustained the relationships by being available for guidance, consultation, and support throughout the first year of implementation on a monthly basis. Initial needs assessment and action planning for school teams: After two days of intensive training with Dr. Terrance Scott, a consultant affiliated with the National Technical Assistance Center on Positive Behavioral Interventions and Supports, the five school teams identified their needs and developed action plans using various



data sources provided to the school. One important source of information for goal setting purposes was the School-wide Evaluation Tool (SET) pre-assessment results compiled by the Behavioral Support Coordinator. Additional resources given to the schools during the training included comprehensive PBS training handbooks, the book Best Behavior: Building Positive Behavior Supports in Schools (Sprague & Golly, 2004), and continuous support provided by the PBS coaches. On-site coaching and promotion of appropriate behaviors: To support the school teams after the initial training session, the PBS coaches attended monthly PBS team meetings on-site, assisting with the analysis and interpretation of behavioral data across school settings (e.g., classroom and office discipline referrals) and maintaining focus on the team’s goals through quarterly updates to the online Team Implementation Checklist (TIC). Several coaches also helped to create informal staff surveys to monitor the quality of the school’s positive behavior interventions. The PBS coaches also played a role in promoting the program school-wide by communicating “PBS Updates” during staff meetings and presenting refresher “booster” sessions to students and staff. Ongoing communication and support: As the behavior support teams in each school implemented their PBS action plans, the district-level PBS Coordination Team met monthly to discuss the resource needs of the schools and review progress toward the district PBS action plan. Additional support for the school teams was provided through the following methods (see Appendix D for examples):

• Beginning in August 2007, the PBS coaches met with the Behavior Support Coordinator at bi-monthly coaches’ meetings and individual support sessions.

• Referral tracking forms were created in September 2007 for schools to monitor the effect of PBS interventions.

• End-user training on the PBS StarWeb site was offered between September and November 2007. • The Department of Pupil Services provided funding for coaches to attend national conferences in October

2007 and March 2008. • The PBS Newsletter, “Creating a Climate for Success with PBS,” was sent to the schools on a bi-monthly

basis beginning in November 2007. Topics of interest, upcoming events, and spotlights on PBS school practices were featured in each publication.

• Dr. George Sugai, co-director of the National Technical Assistance Center, presented the workshop “Creating Positive Results with Positive Behavior Support” in November 2007 to the school teams and LCPS administrators.

• A PBS website was developed in December 2007 to provide resources and contacts to both the pilot schools and interested non-participating schools. A “Coaches Corner” section was also developed to support the PBS coaches.

• Dr. Cynthia Anderson, a consultant affiliated with the National Technical Assistance Center, presented the workshop “PBS in the Classroom” to the five school teams in January 2008.

Intra- and inter-organizational collaboration: The PBS Coordination Team collaborated and met on a regular basis to support the pilot schools within the district. Represented initially by central office administrators in the Offices of Student Services and Special Education Services, the team was expanded to include the Research and Elementary Education Offices from the Department of Instruction. In addition, efforts to inform other stakeholder groups within the district included presentations to counselors (elementary and secondary), social workers, educational diagnosticians, nurses, and clinic aides. External collaborations included George Mason University’s Training and Technical Assistance Center (T-TAC), the National Technical Assistance Center on Positive Behavioral Interventions and Supports, and PBIS Maryland. Evaluation Focus The purpose of this study was to compare the actual implementation of the LCPS PBS framework with the first-year goals developed through the program’s logic model. The short-term goals served as a conceptual framework that directed the collection and analysis of all evidence. Categorized into process and outcome indicators, these expectations defined an effective first-year implementation of the PBS program in the 2007-08 school year.



Process Indicators 1) Participation of school behavior support teams in two full-day training sessions will be 90%. Participants

will be satisfied with the quality of training provided. 2) PBS coaches will attend monthly school team meetings, providing support for data-based decision-making,

planning, and implementation. 3) The PBS Coordinator will facilitate communication and dissemination of PBS best practices to school

teams through regular contact with the coaches and bi-monthly newsletters. 4) The PBS Coordination Team will complete a district PBS action plan to sustain the growth of the program.

Outcome Indicators

5) Using the results of the School-wide Evaluation Tool, school teams will complete an action plan and review progress quarterly.

6) School teams will attain a 60% summary score at a minimum on their PBS Team Implementation Checklists.

7) School teams will present school referral data to their staff monthly in order to share successes and elicit feedback.

8) School teams will attain a 60% summary score at a minimum on the year-end School-wide Evaluation Tool.

9) School teams will determine a baseline office referral rate (per 100 students/year) for benchmarking purposes.

Methodology The study was a formative evaluation that examined the LCPS PBS framework for evidence of an effective first-year implementation in the 2007-08 school year. The evaluation design followed an embedded, single-case study (Yin, 2003) with the five PBS teams from the pilot schools serving as the subunits of analysis. Multiple sources of evidence were used to enhance the validity of the research through the triangulation of data (see Table 3). Program Participants The PBS school teams from the five pilot schools were the primary focus of the study. Each team typically comprised 6-8 representatives from the school–an administrator, special and general education teachers, teaching assistants, and parents–as well as a dedicated PBS coach from the Department of Pupil Services. Instruments and Procedures School-wide Evaluation Tool (SET): Results from the June 2008 SET evaluations (see Appendix A) were collected from the Behavior Support Coordinator and reviewed for their overall implementation scores. The overall score was an average of the following seven PBS implementation indicators: expectations defined, expectations taught, reward system, violations system, decision-making, team management, and district support. In addition, two specific interview questions from the SET were selected as performance indicators given their significance to the program design. Team Implementation Checklist (TIC): Results from the June 2008 TIC (see Appendix A) were extracted from the online database and reviewed for their overall implementation scores. The overall score was an average of the following six implementation indicators: establish commitment, establish and maintain team, self-assessment, establish school-wide expectations, establish information system, and build capacity for function-based support. The TIC charts were also reviewed for evidence of quarterly TIC scores, an indicator of progress monitoring of PBS action plans. Office Discipline Referrals: To determine the baseline office referral rate for each school, data from the PBS StarWeb site were extracted at the end of the 2007-08 school year. The yearly rate (100 students/year) was computed based on the following formula: Total Office Referrals ÷ (Total Enrollment ÷ 100). For example, a yearly rate of 10.0 indicates that 10 office referrals were made for every 100 students in a school.



Individual Interview: The Behavior Support Coordinator was interviewed to develop a clearer picture of the actual implementation of the program. The questions addressed the following topics: program awareness and involvement, training of coaches, school needs and goals, coaching support, ongoing communication and support, and intra-/inter-organizational collaboration. Data from the interview were used to describe the program in the background information section. Satisfaction Survey: Data from completed workshop evaluation forms, in the format used by the Staff Development Office, were analyzed and aggregated for overall satisfaction scores, based on a scale of 1 (Strongly Disagree) to 5 (Strongly Agree). The average overall threshold for satisfaction was set at 4.0 (Agree). Note that the format of the survey did not allow for disaggregation of data by school team. Program Records: For the remaining performance indicators, program documentation was reviewed, including the action plans (district- and school-level), PBS website, bi-monthly PBS newsletters, PBS brochure, attendance report (initial two-day training), and agendas and minutes from the coaches’ meetings.

Results The findings below are categorized by process and outcome indicators in accordance with the conceptual framework of an effective PBS implementation in its first year. Process Indicators The data supplied by the Behavior Support Coordinator provided strong supporting evidence for all four PBS process indicators set at the beginning of the school year (see Tables 4 and 5). At the school level, the level of participation of the PBS teams in the initial two-day training exceeded the 90% threshold (100% total average), and their overall average satisfaction level with the training exceeded the 4.0 target (4.82 overall average). In addition, all PBS coaches reported that they had met at least monthly with their school teams to provide support for data-based decision-making, planning, and implementation. With respect to implementation at the program level, the Behavior Support Coordinator provided six job-embedded training sessions for the PBS coaches as evidenced by the meeting agendas and minutes. All four newsletters created and disseminated by the Coordinator contained one or more examples of promising PBS practices from the participating pilot schools (e.g., photographs of behavioral expectations displayed in different school settings), as well as excerpts from related PBS conferences, workshops, and the research literature. As a member of the district-level PBS Coordination Team, the Coordinator also assisted with completing the district self-assessment and action plan template provided by the National Technical Assistance Center on Positive Behavioral Interventions and Supports. Outcome Indicators The data provided for the school-level outcomes provided strong supporting evidence for four of the five outcome indicators set at the beginning of the school year (see Table 6). With respect to the first supported indicator, all five school teams completed their PBS action plans during the initial two-day training, and reviewed progress toward implementation of the critical features as evidenced by the completion of quarterly TIC charts. By the end of the school year, all school teams reported summary scores in excess of 60% on their TIC charts (min. 74%, max. 92%). In addition, the SET data provided by the Behavior Support Coordinator revealed that the end-of-year SET scores for all school teams were in excess of the 60% target (min. 87%). It is noteworthy, however, that the end-of-year TIC and SET scores did not align for two of the five school teams. Finally, all teams calculated their baseline office discipline referral rates at the end of the school year, based on data reports from the PBS StarWeb website. The findings on the year-end SET provided mixed evidence that school teams were presenting referral data to their staff monthly in order to share successes and elicit feedback. Only one school team reported during the SET interviews that they had presented office referral data updates to school staff on a monthly basis. The other teams reported less frequent use of the referral data, ranging from one to four instances. Note that the data source for the office discipline referrals was the PBS StarWeb website, on which all school teams had completed initial training in November 2007.



Conclusions Evidence collected from document reviews, databases, and individual interviews revealed strong supporting evidence of an effective first-year implementation of PBS in LCPS during the 2007-08 school year. More specifically, analyses of the data highlighted the following findings for the process indicators: (a) all five school teams fully participated and were highly satisfied with their initial PBS training; (b) all PBS coaches followed up the initial training with monthly technical support, (c) the PBS Coordinator facilitated communication and dissemination of PBS best practices as intended, and (d) the PBS Coordination Team was actively planning to sustain the program. With respect to the outcome indicators, the evidence revealed that the school teams: (a) reviewed progress toward their implementation goals quarterly, (b) exceeded their target PBS implementation scores as verified by the PBS Coordinator, (c) reviewed school referral data on an intermittent basis, and (d) set baseline measures for their office referrals. These findings support one of the critical objectives of the PBS Coordination Team, which is to develop a core group of PBS schools that serve as local demonstrations of process and outcomes. In conclusion, this embedded, single-case study revealed that both the PBS Coordination Team and school-based teams attained eight of the nine implementation and outcome indicators set at the beginning of the school year (see Table 7). One school-level outcome indicator had a mixed level of supporting evidence. It is important to note, however, that the program’s logic model used to develop these short-term goals is subject to change as the PBS initiative is expanded into the second and third tiers of the framework. Therefore the indicators of an effective PBS implementation may similarly evolve given the nature of continuous process improvement.

Recommendations 1. Develop an administrative interface for the PBS StarWeb site: To encourage use of the PBS StarWeb site

by behavior support teams on a monthly basis, consider working with the software developer to build an administrative interface so that daily usage statistics can be tracked by the Behavior Support Coordinator and/or other program staff. Program staff could then perform random data audits unobtrusively and follow up with technical support as needed.

2. Improve the process for completing end-of-year TICs: The final TIC and SET scores did not align for two of

the five school teams. To improve the validity of the TIC scores, the Behavior Support Coordinator should review the checklist items with the PBS coaches during training sessions to discuss common misconceptions with respect to the level of implementation of critical PBS indicators. In addition, the TIC completion process should be updated to ensure that PBS coaches reach consensus on year-end TIC scores with members of their school team.

3. Improve the representativeness of the PBS Coordination Team: The district-level team was comprised of a

disproportionate number of central office administrators from the Department of Pupil Services. In order to secure resources and support for the program’s mid- and long-term goals (e.g., student achievement outcomes in reading), the team should solicit input from relevant stakeholders in the Department of Instruction (e.g., Reading Supervisor, Steps to Literacy Specialist, School Assistance and Remediation Programs Coordinator). In addition, at least one representative from the PBS teams should be recruited to integrate a school-level perspective into the committee’s decision-making process.

4. Develop an instrument to track school climate over time: The PBS program was implemented in LCPS in

part to address the following School Board goal: “LCPS will strengthen programs that support safe learning environments free of bullying, teasing, and physical violence.” To track progress toward this long-term goal, a standardized instrument could be developed based on the district’s conceptualization of “safe learning environments” within a school (Horner, Sugai, & Lewis-Palmer, 2005). Completion of this instrument by multiple stakeholders over time could reveal positive or negative trends in school climate for resource allocation purposes, as well as the potential effects of implementing PBS with fidelity over time.

5. Track measures of cost-effectiveness for the pilot schools: One of the primary benefits of the PBS program is

its potential to save time for both administrators and students through reductions in the number of office



discipline referrals. Consider tracking this indicator for the pilot schools in the 2008-09 school year. For example, the amount of time saved for students (i.e., instructional days) could be used for correlation analysis with student achievement measures, such as reading and math.

References Association for Positive Behavior Support. (2007). Mission and vision statement. Retrieved from the APBS website at http://www.apbs.org/about_APBS.htm Horner, R., Sugai, G., & Lewis-Palmer, T. (2005). School-wide positive behavior support evaluation template, October, 2005. Eugene, OR: University of Oregon. National Technical Assistance Center on Positive Behavioral Interventions and Supports. (2004). School-wide positive behavior support implementers’ blueprint and self-assessment. Eugene, OR: University of Oregon. Spaulding, S. A., Horner, R. H., May, S. L., & Vincent, C. G. (2008). Implementation of School-wide PBS across the United States (Evaluation Brief No. 2). National Technical Assistance Center on Positive Behavioral Interventions and Supports. Retrieved February 27, 2009, from http://www.pbis.org/evaluation/evaluation_briefs/nov_08_(2).aspx Sprague, J. R. & Golly, A. (2004). Best behavior: Building positive behavior support in schools. Longmont, CO: Sopris West Educational Services. Sugai, G., Horner, R. H., Dunlap, G., Hieneman, M., Lewis, T. J., Nelson, C. M., Scott, T., Liaupsin, C., Sailor, W., Turnbull, A. P., Turnbull, H. R., III, Wickham, D. Reuf, M., & Wilcox, B. (2000). Applying positive behavioral support and functional behavioral assessment in schools. Journal of Positive Behavioral Interventions, 2(3), 131-143. Sugai, G., Horner, R., & Lewis-Palmer, T. (2001). Team Implementation Checklists version 2.2, August, 2002. Educational and Community Supports, University of Oregon. Sugai, G., Lewis-Palmer, T., Todd, A., & Horner, R. (2001). School-wide Evaluation Tool version 2.0, November, 2001. Educational and Community Supports, University of Oregon. Yin, R. K. (2003). Case study research: Design and methods. Thousand Oaks, CA: Sage Publications.



Appendix A: PBS Data Collection Instruments

School-wide Evaluation Tool (SET)*

Purpose of the SET The School-wide Evaluation Tool (SET) is designed to assess and evaluate the critical features of school-wide effective behavior support across each academic school year. The SET results are used to:

1. Assess features that are in place 2. Determine annual goals for school-wide effective behavior support 3. Evaluate on-going efforts toward school-wide behavior support 4. Design and revise procedures as needed 5. Compare efforts toward school-wide effective behavior support from year to year.

Information necessary for this assessment tool is gathered through multiple sources including review of permanent products, observations, and staff (minimum of 10) and student (minimum of 15) interviews or surveys. There are multiple steps for gathering all of the necessary information. The first step is to identify someone at the school as the contact person. This person will be asked to collect each of the available products listed below and to identify a time for the SET data collector to preview the products and set up observations and interview/survey opportunities. Once the process for collecting the necessary data is established, reviewing the data and scoring the SET averages takes two to three hours. Checklist of Products to Collect

1. Discipline handbook 2. School improvement plan goals 3. Annual Action Plan for meeting school-wide behavior support goals 4. Social skills instructional materials/ implementation time line 5. Behavioral incident summaries or reports (e.g., office referrals, suspensions, expulsions) 6. Office discipline referral form(s) 7. Other related information

Using SET Results The results of the SET will provide schools with a measure of the proportion of features that are 1) not targeted or started, 2) in the planning phase, and 3) in the implementation/ maintenance phases of development toward a systems approach to school-wide effective behavior support. The SET is designed to provide trend lines of improvement and sustainability over time.

* School-wide Evaluation Tool version 2.1, June 2005; © 2001 Sugai, Lewis-Palmer, Todd & Horner; Educational & Community Supports, University of Oregon; Revised 06-29-2005 NKS.



SET Implementation Guide

School ________________________________________ Date __________

District _______________________________________ State ___________

Step 1: Make Initial Contact A. Identify school contact person & give overview of SET page with the list of products needed. B. Ask when they may be able to have the products gathered. Approximate date: _________ C. Get names, phone #’s, email address & record below. Name _________________________________ Phone ____________________

Email ____________________________________________________________ Products to Collect 1. _______ Discipline handbook 2. _______ School improvement plan goals 3. _______ Annual Action Plan for meeting school-wide behavior support goals 4. _______ Social skills instructional materials/ implementation time line 5. _______ Behavioral incident summaries or reports (e.g., office referrals, suspensions, expulsions) 6. _______ Office discipline referral form(s) 7. _______ Other related information

Step 2: Confirm the Date to Conduct the SET A. Confirm meeting date with the contact person for conducting an administrator interview, taking a tour of the

school while conducting student & staff interviews, & for reviewing the products. Meeting date & time: __________________________

Step 3: Conduct the SET A. Conduct administrator interview. B. Tour school to conduct observations of posted school rules & randomly selected staff (minimum of 10) and

student (minimum of 15) interviews. C. Review products & score SET.

Step 4: Summarize and Report the Results A. Summarize surveys & complete SET scoring. B. Update school graph. C. Meet with team to review results.

Meeting date & time: _________________________



SET Scoring Guide School ________________________________________ Date __________

District _______________________________________ State ___________

Pre ______ Post ______ SET data collector ________________________________

Feature Evaluation Question

Data Source (circle sources used)

P= product; I= interview;

O= observation

Score: 0-2

1. Is there documentation that staff has agreed to 5 or fewer positively stated school rules/ behavioral expectations? (0=no; 1= too many/negatively focused; 2 = yes)

Discipline handbook, Instructional materials Other ______________

P

A. Expectations

Defined 2. Are the agreed upon rules & expectations publicly posted in 8 of 10 locations? (See interview & observation form for selection of locations). (0= 0-4; 1= 5-7; 2= 8-10)

Wall posters Other ______________ O

1. Is there a documented system for teaching behavioral expectations to students on an annual basis? (0= no; 1 = states that teaching will occur; 2= yes)

Lesson plan books, Instructional materials Other ______________

P

2. Do 90% of the staff asked state that teaching of behavioral expectations to students has occurred this year? (0= 0-50%; 1= 51-89%; 2=90%-100%)

Interviews Other ______________ I

3. Do 90% of team members asked state that the school-wide program has been taught/reviewed with staff on an annual basis? (0= 0-50%; 1= 51-89%; 2=90%-100%)


4. Can at least 70% of 15 or more students state 67% of the school rules? (0= 0-50%; 1= 51-69%; 2= 70-100%)

Interviews Other ______________

I

B. Behavioral

Expectations Taught

5. Can 90% or more of the staff asked list 67% of the school rules? (0= 0-50%; 1= 51-89%; 2=90%-100%)


1. Is there a documented system for rewarding student behavior? (0= no; 1= states to acknowledge, but not how; 2= yes)

Instructional materials, Lesson Plans, Interviews Other ______________

P

2. Do 50% or more students asked indicate they have received a reward (other than verbal praise) for expected behaviors over the past two months? (0= 0-25%; 1= 26-49%; 2= 50-100%)


C.

On-going System for Rewarding Behavioral

Expectations 3. Do 90% of staff asked indicate they have delivered a reward (other than verbal praise) to students for expected behavior over the past two months? (0= 0-50%; 1= 51-89%; 2= 90-100%)


1. Is there a documented system for dealing with and reporting specific behavioral violations? (0= no; 1= states to document; but not how; 2 = yes)

Discipline handbook, Instructional materials Other ______________

P

2. Do 90% of staff asked agree with administration on what problems are office-managed and what problems are classroom–managed? (0= 0-50%; 1= 51-89%; 2= 90-100%)


3. Is the documented crisis plan for responding to extreme dangerous situations readily available in 6 of 7 locations? (0= 0-3; 1= 4-5; 2= 6-7)

Walls Other ______________ O

D. System for

Responding to Behavioral Violations

4. Do 90% of staff asked agree with administration on the procedure for handling extreme emergencies (stranger in building with a weapon)? (0= 0-50%; 1= 51-89%; 2= 90-100%)




Feature Evaluation Question

Data Source (circle sources used)

P= product; I= interview;

O= observation

Score: 0-2

1. Does the discipline referral form list (a) student/grade, (b) date, (c) time, (d) referring staff, (e) problem behavior, (f) location, (g) persons involved, (h) probable motivation, & (i) administrative decision? (0=0-3 items; 1= 4-6 items; 2= 7-9 items)

Referral form (circle items present on the referral form)

P

2. Can the administrator clearly define a system for collecting & summarizing discipline referrals (computer software, data entry time)? (0=no; 1= referrals are collected; 2= yes)

Interview Other ______________ I

3. Does the administrator report that the team provides discipline data summary reports to the staff at least three times/year? (0= no; 1= 1-2 times/yr.; 2= 3 or more times/yr)


E. Monitoring &

Decision-Making

4. Do 90% of team members asked report that discipline data is used for making decisions in designing, implementing, and revising school-wide effective behavior support efforts? (0= 0-50%; 1= 51-89%; 2= 90-100%)


1. Does the school improvement plan list improving behavior support systems as one of the top 3 school improvement plan goals? (0= no; 1= 4th or lower priority; 2 = 1st- 3rd priority)

School Improvement Plan, Interview Other ______________

P I

2. Can 90% of staff asked report that there is a school-wide team established to address behavior support systems in the school? (0= 0-50%; 1= 51-89%; 2= 90-100%)


3. Does the administrator report that team membership includes representation of all staff? (0= no; 2= yes)


4. Can 90% of team members asked identify the team leader? (0= 0-50%; 1= 51-89%; 2= 90-100%)


5. Is the administrator an active member of the school-wide behavior support team? (0= no; 1= yes, but not consistently; 2 = yes)


6. Does the administrator report that team meetings occur at least monthly? (0=no team meeting; 1=less often than monthly; 2= at least monthly)


7. Does the administrator report that the team reports progress to the staff at least four times per year? (0=no; 1= less than 4 times per year; 2= yes)


F. Management

8. Does the team have an action plan with specific goals that is less than one year old? (0=no; 2=yes)

Annual Plan, calendar Other ______________ P

1. Does the school budget contain an allocated amount of money for building and maintaining school-wide behavioral support? (0= no; 2= yes)


G. District-Level

Support 2. Can the administrator identify an out-of-school liaison in the district or state? (0= no; 2=yes)


A = /4 B = /10 C = /6 D = /8 E = /8 Summary Scores: F = /16 G = /4 Mean = /7



SET Administrator Interview Guide

Let’s talk about your discipline system

1) Do you collect and summarize office discipline referral information? Yes No If no, skip to #4. 2) What system do you use for collecting and summarizing office discipline referrals? (E2)

a) What data do you collect? __________________ b) Who collects and enters the data? ____________________

3) What do you do with the office discipline referral information? (E3) a) Who looks at the data? ____________________ b) How often do you share it with other staff? ____________________

4) What type of problems do you expect teachers to refer to the office rather than handling in the classroom/ specific setting? (D2)

5) What is the procedure for handling extreme emergencies in the building (i.e. stranger with a gun)? (D4)

Let’s talk about your school rules or motto

6) Do you have school rules or a motto? Yes No If no, skip to # 10. 7) How many are there? ______________ 8) What are the rules/motto? (B4, B5) 9) What are they called? (B4, B5) 10) Do you acknowledge students for doing well socially? Yes No If no, skip to # 12. 11) What are the social acknowledgements/ activities/ routines called (student of month, positive referral,

letter home, stickers, high 5's)? (C2, C3)

Do you have a team that addresses school-wide discipline? If no, skip to # 19

12) Has the team taught/reviewed the school-wide program with staff this year? (B3) Yes No 13) Is your school-wide team representative of your school staff? (F3) Yes No 14) Are you on the team? (F5) Yes No 15) How often does the team meet? (F6) __________ 16) Do you attend team meetings consistently? (F5) Yes No 17) Who is your team leader/facilitator? (F4) ___________________ 18) Does the team provide updates to faculty on activities & data summaries? (E3, F7) Yes No

If yes, how often? ______________________ 19) Do you have an out-of-school liaison in the state or district to support you on positive behavior support

systems development? (G2) Yes No If yes, who? ___________________

20) What are your top 3 school improvement goals? (F1) 21) Does the school budget contain an allocated amount of money for building and maintaining school-wide

behavioral support? (G1) Yes No



SET Additional Interviews

In addition to the administrator interview questions, there are questions for Behavior Support Team members, staff and students. Interviews can be completed during the school tour. Randomly select students and staff as you walk through the school. Use this page as a reference for all other interview questions. Use the interview and observation form to record student, staff, and team member responses.

Staff Interview Questions Interview a minimum of 10 staff

1) What are the __________________ (school rules, high 5's, 3 bee’s)? (B5) (Define what the acronym means)

2) Have you taught the school rules/behavioral expectations this year? (B2)

3) Have you given out any _______________________ since _______________? (C3)

(rewards for appropriate behavior) (2 months ago)

4) What types of student problems do you or would you refer to the office? (D2) 5) What is the procedure for dealing with a stranger with a gun? (D4)

6) Is there a school-wide team that addresses behavioral support in your building?

7) Are you on the team?

Team Member Interview Questions

1) Does your team use discipline data to make decisions? (E4)

2) Has your team taught/reviewed the school-wide program with staff this year? (B3)

3) Who is the team leader/facilitator? (F4)

Student interview Questions Interview a minimum of 15 students

1) What are the _________________ (school rules, high 5's, 3 bee’s)? (B4)

(Define what the acronym means.)

2) Have you received a _______________________ since ________________? (C2) (reward for appropriate behavior) (2 months ago)


Program Evaluation: PBS Program (First Year)

SET Interview and Observation Form

Staff questions (Interview a minimum of 10 staff members) Team member questions

Student questions

What are the school

rules? Record the # of rules known.

Have you taught the

school rules/ behave. exp. to students this year?

Have you given out

any _____ since

_______? (2 mos.)

What types of student

problems do you or would

you refer to the office?

What is the procedure for dealing with a stranger with a

gun?

Is there a team in your school

to address school-wide

behavior support systems?

Are you on the team? If yes, ask team questions

Does your team use discipline

data to make decisions?

Has your team taught/ reviewed SW

program w/staff this

year?

Who is the team

leader/ facilitator?

What are the (school rules)?

Record the # of rules known

Have you received a ________

since ________?

1 Y N Y N Y N Y N Y N Y N 1 Y N 2 Y N Y N Y N Y N Y N Y N 2 Y N 3 Y N Y N Y N Y N Y N Y N 3 Y N 4 Y N Y N Y N Y N Y N Y N 4 Y N 5 Y N Y N Y N Y N Y N Y N 5 Y N 6 Y N Y N Y N Y N Y N Y N 6 Y N 7 Y N Y N Y N Y N Y N Y N 7 Y N 8 Y N Y N Y N Y N Y N Y N 8 Y N 9 Y N Y N Y N Y N Y N Y N 9 Y N 10 Y N Y N Y N Y N Y N Y N 10 Y N 11 Y N Y N Y N Y N Y N Y N 11 Y N 12 Y N Y N Y N Y N Y N Y N 12 Y N 13 Y N Y N Y N Y N Y N Y N 13 Y N 14 Y N Y N Y N Y N Y N Y N 14 Y N 15 Y N Y N Y N Y N Y N Y N 15 Y N Total X Total

Location Front hall/

office Class 1 Class 2 Class 3 Cafeteria Library Other setting

(gym, lab) Hall 1 Hall 2 Hall 3

Are rules & expectations posted? Y N Y N Y N Y N Y N Y N Y N Y N Y N Y N Is the documented crisis plan readily available?

Y N Y N Y N Y N Y N Y N Y N X X X



Effective Behavior Support Team Implementation Checklists (TICs)* (Monthly) School Date of Report

District County State

INSTRUCTIONS: The EBS team should complete both checklists monthly to monitor activities for implementation of EBS in the school. EBS Team Members

Person(s) Completing Report

Checklist #1: Start-Up Activity

Complete & submit Monthly. Status: Achieved, In Progress, Not Started

Date:

(MM/DD/YY)

Establish Commitment

1. Administrator’s support & active involvement. Status:

2. Faculty/Staff support (One of top 3 goals, 80% of faculty document support, 3 year timeline). Status:

Establish & Maintain Team

3. Team established (representative). Status:

4. Team has regular meeting schedule, effective operating procedures. Status:

5. Audit is completed for efficient integration of team with other teams/initiatives addressing behavior support. Status:

Self-Assessment

6. Team/faculty completes EBS self-assessment survey. Status:

7. Team summarizes existing school discipline data. Status:

8. Strengths, areas of immediate focus & action plan are identified. Status:

* Team Implementation Checklists version 2.2, August 2002; © 2001 Sugai, Horner & Lewis-Palmer; Educational & Community Supports, University of Oregon; Revised 04-14-2003 DP.



Establish School-wide Expectations

9. 3-5 school-wide behavior expectations are defined. Status:

10. School-wide teaching matrix developed. Status:

11. Teaching plans for school-wide expectations are developed. Status:

12. School-wide behavioral expectations taught directly & formally. Status:

13. System in place to acknowledge/reward school-wide expectations. Status:

14. Clearly defined & consistent consequences and procedures for undesirable behaviors are developed. Status:

Establish Information System

15. Discipline data are gathered, summarized, & reported.

Status:

Build Capacity for Function-based Support

16. Personnel with behavioral expertise are identified & involved.

Status:

17. Plan developed to identify and establish systems for teacher support, functional assessment & support plan development & implementation.

Status:

Checklist #2: On-going Activity Monitoring

Complete & submit Monthly. Status: Achieved, In Progress, Not Started

1. EBS team has met at least monthly. Status:

2. EBS team has given status report to faculty at least monthly.

Status:

3. Activities for EBS action plan implemented. Status:

4. Accuracy of implementation of EBS action plan assessed.

Status:

5. Effectiveness of EBS action plan implementation assessed.

Status:

6. EBS data analyzed. Status:

Additional Observations/Comments/Questions:



Action Plan for Completion of Start-Up Activities

Activity Activity Task Analysis Who When

a.

b.

c.

d.

1. Establish Commitment • Administrator • Top 3 goal • 80% of faculty • Three year timeline

e.

a.

b.

c.

d.

2. Establish Team • Representative • Administrator • Effective team operating

procedures • Audit of teams/initiatives e.

a.

b.

c.

d.

3. Self-Assessment • EBS survey • Discipline data • Identification of strengths,

focus • Action Plan developed • Action Plan presented to

faculty e.

a.

b.

c.

d.

4. School-wide Expectations • Define 3-5 school-wide

behavioral expectations • Curriculum matrix • Teaching plans • Teach expectations • Define consequences for

problem behavior e.

a.

b.

c.

d.

5. Establish Information System • System for gathering useful

information • Process for summarizing

information • Process for using information

for decision-making e.

a.

b.

c.

d.

6. Build Capacity for Function-based Support • Personnel with behavioral

expertise • Time and procedures for

identification, assessment, & support implementation e.



Appendix B: Tables

Table 1: PBS Schools by School Year and Type

Type Year School Elementary Middle

Ball’s Bluff X John W. Tolbert, Jr. X Rosa Lee Carter X Stone Hill X

2007-08

Sully X Table 2: Matrix of PBS Professional Development Offerings for School-based Teams

Year

Workshop Training

On-site School Team Coaching

Information Sharing Sessions across Teams

Min. Hours

2007-08

3 sessions (24 hours) comprising: • Initial PBS School Team Training

(12 hrs; July) • End-user training on PBS StarWeb

site (3 hrs; Oct.-Dec.) • Booster Session with Dr. George

Sugai, Center on Positive Behavioral Interventions and Supports (3 hrs; Nov.)

• PBS in the Classroom (6 hrs; Jan.)

10 monthly follow-up coaching sessions (10 hrs)

N/A (dissemination via bi-monthly PBS Newsletters)

34 hours



Table 3: Operationalization of the Conceptual Framework

Category Indicators of an Effective PBS Implementation (First Year) Method Data Source • Participation of school behavior support teams in two full-

day training sessions will be 90%. Participants will be satisfied with the quality of training provided.

document review, survey

attendance report, satisfaction survey

• PBS coaches will attend monthly school team meetings, providing support for data-based decision-making, planning, and implementation.

individual interview

question on year-end SET

• The PBS Coordinator will facilitate communication and dissemination of PBS best practices to school teams through regular contact with the coaches and bi-monthly newsletters.

document review

meeting agendas and minutes, newsletters

Process

• The PBS Coordination Team will complete a district PBS action plan to sustain the growth of the program. document

review district PBS self-

assessment, action plan

• Using the results of the School-wide Evaluation Tool, school teams will complete an action plan and review progress quarterly.

document review, database

school action plans, quarterly TIC

scores • School teams will attain a 60% summary score at a

minimum on their PBS Team Implementation Checklists. database year-end TIC scores

• School teams will present school referral data to their staff monthly in order to share successes and elicit feedback. individual

interview question on

year-end SET • School teams will attain a 60% summary score at a

minimum on the year-end School-wide Evaluation Tool. database year-end SET scores

Outcome

• School teams will determine a baseline office referral rate (per 100 students/year) for benchmarking purposes. database PBS StarWeb

website Table 4: School-level Process Indicators and Results* School

Participation of School Teams in Initial

Two-Day Training

Overall Satisfaction of School Teams with Initial PBS Training (1-5 scale)

Frequency of Team Meetings Attended

by PBS Coaches Ball’s Bluff 100% Monthly Rosa Lee Carter 100% Monthly Sully 100% Monthly Tolbert 100% Monthly Stone Hill 100%

4.82

Biweekly * School teams comprised 6-8 stakeholders, typically an administrator, PBS coach, special and general education teachers, teaching assistants, and parents. Table 5: Program-level Process Indicators and Results Performance Indicator Result Number of Job-embedded Training Sessions for PBS Coaches 6 Number of Newsletters Highlighting the Implementation of PBS Best Practices 4

Completed District PBS Self Assessment & Action Plan? Yes



Table 6: School-level Outcome Indicators and Results School

Developed

Action Plan?

Reviewed Progress of Action Plan Quarterly?

TIC Score (June ‘08)

Frequency of Referral Data Presented to

Staff

SET Score (June ‘08)

Base Office Discipline

Referral Rate Set?

Ball’s Bluff Yes Yes 92% 1-2 times 89% Yes Rosa Lee Carter Yes Yes 74% 3-4 times 91% Yes Sully Yes Yes 91% 4 times 88% Yes Tolbert Yes Yes 68% Monthly 94% Yes Stone Hill Yes Yes 85% 4 times 87% Yes Table 7: Conclusions Based on Supporting Evidence*

Category

Indicators of an Effective PBS Implementation (First Year)

Level of Supporting Evidence

1) Participation of school behavior support teams in two full-day training sessions will be 90%. Participants will be satisfied with the quality of training provided.

Strong

2) PBS coaches will attend monthly school team meetings, providing support for data-based decision-making, planning, and implementation. Strong

3) The PBS Coordinator will facilitate communication and dissemination of PBS best practices to school teams through regular contact with the coaches and bi-monthly newsletters.

Strong

Process

4) The PBS Coordination Team will complete a district PBS action plan to sustain the growth of the program. Strong

5) Using the results of the School-wide Evaluation Tool, school teams will complete an action plan and review progress quarterly. Strong

6) School teams will attain a 60% summary score at a minimum on their PBS Team Implementation Checklists. Strong

7) School teams will present school referral data to their staff monthly in order to share successes and elicit feedback. Mixed

8) School teams will attain a 60% summary score at a minimum on the year-end School-wide Evaluation Tool. Strong

Outcome

9) School teams will determine a baseline office referral rate (per 100 students/year) for benchmarking purposes. Strong

* Rating scale: Strong (++), Moderate (+), Mixed (+/-), Weak (-), Insufficient (--).



Appendix C: Figures

Figure 1: Continuum of Behavioral Support

Figure 2: School-level PBS Implementation Model

Specialized Individual Interventions(Individual StudentSystem)

Specialized GroupInterventions(At-Risk System)

Universal Interventions (School-Wide SystemClassroom System)

Students withoutSerious ProblemBehaviors (80 -90%)

Students At-Risk for Problem Behavior(5-15%)

Students withChronic/IntenseProblem Behavior(1 - 7%)

Primary Prevention

Secondary Prevention

Tertiary Prevention

All Students in School

Agreements

Team

Data-based Action Plan

ImplementationEvaluation



Figure 3: District-level PBS Implementation Model

Funding Visibility Political Support

Training Coaching Evaluation

Local School Teams/Demonstrations

Leadership TeamActive Coordination



Appendix D: Sample Artifacts from the First Year

Testimonials from School Principals on PBS Website PBS Newsletter – March/April 2008, Vol. 1, Issue 3 Elementary School Referral Form (2-Sided) Stone Hill Middle School Referral Form

“PBS has helped create a culture that stresses high behavioral and academic expectations for all students. As a new school, PBS has given us a common language in which we communicate with students and with each other. Students know that every adult in the building will have the same response to behavioral and academic challenges. Consequently, students have stepped up to the challenge.”

Rodney S. Moore, Principal, Stone Hill Middle School

“PBS has been a wonderful addition to the Tolbert Elementary Community. Our staff is united in our efforts, expectations, and practices. The difference between our school and others who do not have such a program has been noticed by our visitors! We love PBS!” Elaine Layman, Principal, John Tolbert Elementary School

“PBS has allowed Sully to build a culture that the students are completely vested. Our theme of "Be Great Today" is related to every portion of the school day and the students are proud of the accomplishments that are recognized daily.” Tim Martino, Principal, Sully Elementary School

“PBS has allowed Sully to build a culture that the students are completely vested. Our theme of "Be Great Today" is related to every portion of the school day and the students are proud of the accomplishments that are recognized daily.” Michele Freeman, Principal, Rosa Lee Carter Elementary

“The students seem to enjoy the different components of PBS. They're learning to R.O.A.R. (Respect, On Task, Always Responsible) throughout the school.” Jerry Hill, Principal, Balls Bluff Elementary School

WWWhhhaaattt aaarrreee LLLCCCPPPSSS ppprrriiinnnccciiipppaaalllsss sssaaayyyiiinnnggg aaabbbooouuuttt PPPooosssiiitttiiivvveee BBBeeehhhaaavvviiiooorrr SSSuuuppppppooorrrttt aaattt ttthhheeeiiirrr sssccchhhoooooolllsss???

March/April, 2008 Volume 1, Issue 3

The 5th Annual Conference for PBS “The Expanding World of PBS:

Science, Values and Vision” Several members of PBS school teams from Loudoun County Public Schools attended the 5th Annual Conference for PBS in Chicago March 26 – 28, 2008. Valuable information was gathered for school-wide PBS implementation, as well as targeted interventions for our students in the “yellow and red zones.” Excerpts from several sessions are provided in this newsletter.

Targeted Social Intervention for Individual Students

Stop, Observe, Decide, and Act (SODA) is a social behavioral learning strategy created by Brock (2001) for individuals who experience difficulty interpreting social situations. This strategy teaches individuals to pay attention to relevant social cues, consider their meaning, and select an appropriate response. This strategy is an effective tool to be taught to our students in the “red or yellow zones.”

SODA What can I do? What can I ask myself? STOP THINK

Take a deep breath What is in the room? What is the activity? What is the routine? What does my schedule say?

OBSERVE LOOK Ask

Who’s here” What are they doing? What are they saying? Are they playing safely? Is everybody having fun?

DECIDE THINK Choose the best and safest solution

What should I do? What should I say? Will my feelings get hurt? Will I hurt their feelings? Will I get into trouble?

ACT DO Introduce yourself Listen to others Stay on topic, Join the activity OR Get help from an adult when things don’t look or feel so good

Am I enjoying myself? Do I feel included? Are the others being respectful toward me? Am I being respectful toward others?

SODA is also discussed during MANDT training.

Pep Rallies and Booster Sessions:

Sully Celebrates Successes Sully Elementary celebrated reaching their second goal for Being Respectful, Responsible and Ready for whatever comes their way! The “Travel-Around-the-World” incentive program recognizes students who exhibit exemplary behavior through acts of kindness, caring and courtesy. Sully Hornets are respectful, responsible and give their best effort. Staff members recognize students for respectful, responsible or ready behavior with a hornet coupon on a regular basis. These coupons are taped to a path around the cafeteria which goes from one continent to another. When the coupon path reaches a continent, everyone in the school celebrates. This celebration was a school-wide B-I-N-G-O game. Everyone participated and celebrated in Sully’s school spirit!

Students received a goody bag when they had B-I-N-G-O. Michelle Walthour, assistant principal, called letters and numbers with the upper grades in the gym, while Tim Martino, principal, called out the letters and numbers in the cafeteria with the primary grades.

PBS Newsletter Page 2

What are Teachers Doing in Effectively Managed Classrooms?

Active supervision High rates of positive reinforcement for

expected behavior Predictable schedules and routines Respectful, caring interactions Consistent responses to problem behavior Use of effective instruction and curriculum

An effective relationship may be the keystone that allows the other aspects to work well. If a teacher has a good relationship with students, then students accept the teacher’s rules, procedures, and disciplinary actions. Without the foundation of a good relationship, students commonly contest them. (Marzano, 2003)

Giving Effective Praise

Is delivered contingently Convincingly, specifies the particulars of the

accomplishment being praised Rewards attainment of specified performance

criteria Provides information to students about their

competence or the value of their accomplishments

Uses students’ own prior accomplishments as the context for describing present accomplishments

Is given in recognition of noteworthy EFFORT or success at difficult (for the student) tasks

Focuses students’ attention on their own task-relevant behavior…Does not focus attention on the teacher as an external authority who is manipulating them.

(Effective Classroom Management Strategies for All Students, Gatlinburg, TN PBS training)

Rosa Lee Carter Elementary School

Students, staff and several parents reviewed Rosa Lee Carter’s expectations and procedures on April 4, 2008 with a very enthusiastic pep rally. The Coyote paw is a signal for all students to be silent and ready to listen. PBS team members dressed up as place settings for the cafeteria incentive program. Each grade level gave a PBS cheer for Respect Self, Respect Others and Respect Property.

John W. Tolbert Elementary School

Tolbert instituted a new incentive program for Respecting Property with the “Golden Plunger Award!” Classrooms that are neat, clean, and have chairs placed on the desks will receive the Golden Plunger from the custodian. On the first day back from Spring Break, the students were reminded of school rules and procedures by reviewing the Tolbert Rules: Respect Self, Others and Property for all areas of the school. Students were also entertained by watching teacher relay races.

Diane Newcomer, assistant principal, displays “Golden Plunger” award.

PBS Newsletter Page 3

Let us know what you are doing at your schools! You are welcome to submit any

articles or suggestions to be considered for the next PBS Newsletter. Please submit

ideas to Cathy Shwaery at [email protected].

Second Semester Kick-off at Stone Hill Middle School

All resource teachers at Stone Hill Middle School engaged students in a PBS Jeopardy game developed for the smart boards to review the school rules. Reports from teachers and students indicated that the lessons were a great success! Students are recognized in the cafeteria on Fridays for respectful behavior. Students may choose from: Going to the head of the lunch line, adding a point to a test grade (with teacher permission), or an ice cream. Students have shown great enthusiasm for PBS in the Cafeteria! Students also participated in a poster contest. Visit Stone Hill and see the posters displayed around the cafeteria.

One of the student-made posters

displayed in the cafeteria

Websites to check out:

http://caughtyoubeinggood.googlepages.com/lessonplans. There are several lesson plans from Griffin Middle School (Smyrna, GA)

Dates to remember:

April 28, 2008: Data Support group for administrators and coaches; 8:30 a.m.

July 15 & 16, 2008: for New School Team training

July 17, 2008: Returning School Team Training

To download flyers for summer training, please go to www.loudoun.k12.va.us. Click on Pupil Services and then Positive Behavior Support. The information is available under “Professional Development.”

Balls Bluff Elementary Recognizes Classes on Their “Wall of Fame”

Students at Balls Bluff Elementary attach their “Caught Being Good” coupons on the strips of paper for their grade level and watch the participation grow!

Comprehensive Teaching Model for Students with Autism and

Developmental Disabilities in PBS in East Rochester, NY

Why didn’t 50% of students with autism and developmental disabilities “get” PBS? Behavioral expectations were abstract. Reinforcement was frequently delayed. Incentive activities were socially based. No opportunities were given to practice. Verbal/abstract approach to teach what

was expected. How did they meet the special needs of these students? Teaching strategies were enhanced to meet the needs of this population. Each area was broken down into visual

steps. The students read a social story in class,

in the environment, and on the bus. The students were exposed to the story

multiple times every day. While the rest of the school did an

expectation area a month, they worked on an area until it was mastered.

Visual cues around the school were changed to the familiar Boardmaker symbols.

Students were recognized for specific behavior on a frequent basis by all staff members.

Elementary School Referral Form (2-Sided)

Loudoun County Public Schools Office Referral

Date_________ Grade_______ Teacher/Team___________ Time____________ Day of Week: M T W R F (circle one) Student(s) Involved________________________________ Reporting Staff Person_____________________________ Incident: (check one)

____Disrespect (D1C) ____Defiance/Refuses Request (D2C) ____Disruptive Behavior (D5C) ____Inappropriate Language/Gesture (D6C) ____Lying/Cheating (S2V) ____Other _____________________________________________ Location: ____Bus Stop (BST) ____Hallway (HWY) ____Bus (BUS) ____Library (LIB) ____Cafeteria (CAF) ____Office (OFF) ____Classroom (CRM) ____Playground (PLG) ____Computer Lab (LAB) ____Restroom (RRM) ____Field Trip (FT) ____Stairwell (STW) ____Gym (GYM) ____Other _______________

Teacher Action: _____Proximity ____Time-out in class _____Redirection ____Counselor Contact _____Re- teach Rule ____Parent Contact _____Reminders When__________ _____Reprimand/Warning How___________ _____Modifications ____Other_____________ _____Conference with Student Possible Motivation: Trying to Access/Obtain___________________________________ Trying to Avoid/Escape____________________________________ Staff Comments: Administrative Response: ____Counsel/Mediate (110) ____Removal from class (106) ____Reprimand/Warning (109) ____Out-of-School Susp. (002) ____Parent Phone Call (111) ____Bus Susp. (107) ____Parent Conference (111) ____Assigned bus seat (108) ____Loss of Privilege (111) ____Write Apology (111) ____Restitution (111) ____Other____________ (111) Administrative Signature_______________________________

Stone Hill Middle School Classroom Report or Office Referral (Circle one)

Student: __________________________ Teacher: ___________________________ Grade: 6 7 8

Disciplinarian: _____________________ Time: ________ Date: _______Day of week: M T W TH F Location: Athletic Fields Auditorium Bus Bus Stop Restroom

Cafeteria Hallway Gym/Locker Room Library Locker Area Other _________

Possible Motivation:

Escape/Avoid (who or what)________________ Obtain/Access (who or what)________________ Teacher Response:

Warn and re-teach Redirection Modifications Parent Contact (when__________; phone or email) Counselor contact Detention (Please check: a.m.___; p.m.____; lunch____) Conference w/ student Time-out in class Other_________________________________

Teacher Comments: …………………………………………………………………………………………………………………….…………………………… Administrator Response:

Reprimand/Warning Loss of Privilege Conference w/ student Contact parents Counsel/Mediate Detention (Please check: a.m.___; p.m.____; lunch____) In-school restriction (AIA) Out of school suspension Saturday School Other

___________________________________________________________________________________________________

White Copy - Administrator Yellow Copy - Teacher

Classroom Report

Disrespect/Walking Away (D1C) Defiance/Refuses Request (D2C) Inappropriate Language (D6C) Frequent Tardies (A1T)

Misrepresentation: Lying, cheating (S2V) Unauthorized Tech Use (T1C) Other _____________________________

Office Referral

Harassment (HR1) Obscene Language (D6C) Significant Violation

Self: _____________________________ Others: ___________________________ Property: _________________________

Chronic Report Violations ________________

Documents

PBS Year 1 Report Final - Loudoun County Public SchoolsFinal).pdf · Positive Behavioral Support ... The findings of this study provided strong supporting evidence that the PBS