Upload
noel-green
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Understanding and Administering the School-Wide
Evaluation Tool (SET)
Agenda for today• Understanding of the SET
and its purpose• Process for scheduling and
completing• Administering the 4 parts of
the SET• Sign up for dates
In your packet…
• Slides• Supporting Handouts• Blank SET Documents• Activities
Please make note of time limits and watch your clocks!
• Trainer will raise his/her hand.
• Finish your thought/comment.
• Participants will raise a hand and wait quietly.
Attention Signal
Be Responsible• Return promptly from breaks
• Be an active participant
• Use laptop computers appropriately
Be Respectful• Maintain cell phone etiquette
• Listen attentively to others
• Limit sidebars and stay on topic
Be Kind• Enter discussions with an open mind
• Respond appropriately to others’ ideas
Participant Expectations
Understanding and Planning for the School-Wide Evaluation Tool
(SET)
Understanding and Planning: What is the SET?
• The SET is an empirically validated tool used to assess level of PBS implementation.
• It was developed at the University of Oregon for the PBIS National Technical Assistance Center as a research tool.
• SET scores are the national standard used to measure PBS implementation (universals).
Understanding and Planning: Why Administer the SET?
• Provides objective information to schools about effectiveness of implementation
• Gives teams specific areas to focus on for improvement
• Establishes a baseline to compare implementation over time
• Allows WCPSS PBS team to analyze trends in implementation in order to modify strategies of support
• Helps WCPSS to understand discrepancies in school outcomes
Understanding and Planning: How is the SET Administered?• There are four components of
the SET– Administrator interview
– Staff and student interviews
– Observations in all settings
– Document review and scoring
• Takes about 2 hours to complete the SET
• Usually completed between 9:30 am and 11:30 am.
Understanding and Planning: Who administers the SET?
• SET is completed in pairs.
• Primary interviewer does the majority of the interviewing and is responsible for completing all required tasks.
• Secondary interviewer’s role is to check for agreement on scoring, and to discuss any discrepancies with the primary interviewer.
• Secondary interviewers can ask follow up questions if they need more information to document the answer.
Understanding and Planning: Scheduling Process
The SET Scheduling Coordinator will:
• Create and maintain master schedule
• Notify all schools of the assigned date and obtain confirmation
• Re-confirm close to the SET date
• Handle all re-schedules
• Troubleshoot any day-of challenges
• Ensure that all materials are returned to the assigned PBS coach
Understanding and Planning: Scheduling Process
• Master schedule is on google docs, on laptops.
• Sign up for dates and choose primary or secondary interviewer.
• In order to meet our goal, 4 SETS must be completed every day between March 22rd and April 20th.
Understanding and Planning: Process for Completing the SET• Once you have dates, Javier will be
your main contact.
• Day of SET:– Show up – Complete the Administrator Interview first.
– Complete the staff and student interviews while touring and doing observations.
– Review documents and score on site.
• After SET– Write a brief plus/delta of observations
you would like to share with the school.
– Return all documents and notes to Javier.
– Accept our gratitude!
Administering the SET
Administrative InterviewStudent and Staff Interviews
ObservationsDocument Review and Scoring
Administering the SET: Administrator Interview
• The administrator interview must be completed prior to staff and student interviews.
• Get clear answers to each question as staff interviews are scored based on agreement with administrator answers.
• Interview typically takes 20-30 minutes.
Administering the SET: Administrator Interview Tips
• At the beginning of each section, it is helpful to orient the administrator to the types of questions coming.
• It is ok to ask follow up questions if you need more information or to re-word if they don’t understand what you are referring to.
• If the administrator answers no to certain questions you will be prompted to skip ahead.
Administering the SET: Administrator Interview Tips
• Question #5: Find out how staff would refer to the procedure
• Question #6: If unclear, reference the acronym
• Question #11: Focus on the reinforcers that are most frequent, and find out how staff and students refer to them
• Question #12: If yes, follow-up with when and how procedures were reviewed with staff.
• Question #20: Key word in this question is goals. Improving or sustaining behavioral practices must be part of a goal
• Question #21: If yes, follow-up with how
Activity 1: SET Matching Exercise
• On your own, complete the SET matching exercise sheet by matching the interview questions with the evaluation questions.
• Compare answers with a partner.
• Refer to answer key if needed.
Administering the SET: Student and Staff Interviews
• Prior to beginning additional interviews and observations, fill in the blanks on the top of the interview and observation form based on administrator answers.
• Write the administrator’s answers to questions 4, 5 and 17 on the interview and observation form in the corresponding column.
• If you skipped questions on the administrator interview, you need to agree on which additional interview questions you can complete.
Administering the SET: Student and Staff Interviews
• Move throughout the school to conduct interviews of staff members and students.
• Strive to obtain a representative sample of students across grade level, sex, and race.
• Interview staff members with various roles in the school.
• Four of the staff interviews need to be PBS team members so that they can answer additional questions.
Administering the SET: Student and Staff Interviews• When you approach staff, you may have to remind
them of your purpose and ask if they have time to answer a few questions
• Staff Interviews:
– 7 questions with 3 additional for PBS team members
– Interview one person at a time
– Try to locate staff in various grade levels
– Attempt to minimize interruption to direct instruction or supervision
– Record number of expectations known(3/4, 1/3, 5/5, etc.)
– Circle Y or N for yes/no questions
– Put “+” when staff response matches administrator and a “-” when it does not.
Administering the SET: Student and Staff Interviews• Tell the student that you are conducting a
survey and would like to ask them 2 questions. Move on if the student is hesitant.
• Student Interviews:– Interview one person at a time. If students are
standing in a group, ask only one student to answer or ask the student to move over to the side with you.
– Attempt to minimize interruption to direct instruction or supervision
– Record number of expectations known(3/4, 1/3, 5/5, etc.)
– Circle Y or N for yes/no question
Activity 2: Role Play
• Form groups of two or three.• Practice approaching both
“staff” and “students” in role play.
• Note any issues or trouble spots.
Administering the SET: Observations
While interviewing staff and students, complete the required observations in 10 areas.
• Look for posted rules and expectations that are specific to the area you are in
• Ask for documentation of the crisis procedures in the required areas (notebook, red bag containing procedures, cards in nametag, etc.)
Administering the SET: Document Review and Scoring
• After completing the observations and student and staff interviews take all notes and documents to a space on-site.
• Complete the scoring guide with the secondary interviewer.
• Review documents as needed
• For items requiring documents to score there will be a P in the “data source” column.
Administering the SET: Document Review and Scoring
• The role of documents is to support sustainable implementation of PBS.
• Documents should contain details clear enough for future training of staff and students.
• There are many different ways to document procedures that meet the criteria.
Administering the SET: Scoring Guidelines• Most items are easy to score and
have clear criteria for 0,1,2.• Some items may require
discussion between the primary and secondary interviewer to determine score.
• Many items require multiple data sources to determine score.
• Use the “data source” column on the scoring guide to assist you.
Administering the SET: Scoring Guidelines
Common Scoring Challenges• SIP• Action Plans• Budget
When in doubt leave an item blank, make copies of necessary documents and send to Javier for scoring
Activity 3: Case Study
• On your own, review all the materials from the provided case study and complete the scoring guide.
• Compare answers with a partner and discuss any discrepancies.