Upload
belinda-harrison
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
SCREENING AND PROGRESS MONITORING: THE HOW TO’S…..
BEAVERTON SCHOOL DISTRICT10-23-09
Dean Richards, Lisa Bates, Jon Potter
Objectives
Provide the philosophy behind universal screening
Provide overview of why to use progress monitoring and its essential features
Show how to use progress monitoring Developing an aimline and phaselines
How to use the information in Tier 1 and Tier 2 meetings
Managing Complex Change
= Change
Confusion=
=
=
=
=
+ + + +Vision Incentives ResourcesAction Plan
+ + + +Skills Incentives Resources Action Plan
+ + + +Vision
Incentives ResourcesAction Plan
+ + + +Vision Skills Resources
Action Plan
+ + + +Vision Skills Incentives Action
Plan
+ + + +Vision Skills Incentives Resources
Adapted from Knoster, T.
Anxiety
Resistance
False Starts
Frustration
Skills
The Process is Ongoing and Long-Term
CONSENSUS
Oregon RTI project’s role today Create knowledge of one component of
the RTI infrastructure (assessment) Provide knowledge and skills of Universal
Screening and Progress Monitoring
Expectations
Let us know when you need a break Write questions on sticky notes provided
and we will answer them when we can
The Importance of Data
Data are important
Assessments
Universal Screening Tools Why? What?
What is CBM?
Curriculum Based Measures are usually composed of a set of standard directions, a timing device, set of materials, scoring rules, standards for judging performance, and record form or charts.
These are also called General Outcome Measures (GOM)
Purposes of CBMs
Evaluate overall effectiveness of program Select students who need additional
support Monitor progress of students
A universal screener should over-identify students who might need something more!
Essential Features of CBM Tools• Robust indicator of academic health • Brief and easy to administer• Can be administered frequently• Must have multiple, equivalent forms
(If the metric isn’t the same, the data are meaningless)
• Must be sensitive to growth
Universal Screening Quick general outcome measures Should occur for ALL students 3x per
year Used for data-based decision making
about: How to create instructional change for
ALL Changes for Tier 1
Which students need a closer look and/or intervention Changes for Tier 2
Why use a Universal Screener to Track Progress?
• Reading trajectories are established early.
• Readers on a low trajectory tend to stay on that trajectory without intervention.
• Students on a low trajectory tend to fall further and further behind.
• Early identification leads to early intervention. The later children are identified as needing support, the more difficult it is to catch up!
Maze EasyCBM AIMSweb ORF DIBELS Math Computation Math Applications Math Tests of Early
Numeracy Writing (Total Words
Written) Writing (Correct Word
Sequences)
Quick Phonics assessment
QRI-IV CORE Multiple Measures
Assessment DRA2 Fountas and Pinnell Report cards Meeting OAKS standards Read Well Unit Tests,
core curriculum weekly tests on skills that are learned
Screeners Not Screeners
•The major purpose for administering diagnostic tests is to provide information that is useful in planning more effective instruction.
Purpose of Diagnostic Assessments
• Diagnostic tests should only be given when there is a clear expectation that they will provide new information about a child’s difficulties learning to read that can be used to provide more focused, or more powerful instruction.
OAKS Passing OAKS 3rd grade (204) places a child in the 17th percentile
Passing 3rd grade is not enough.
By 5th grade, if these students stay at the same percentile they will fail OAKS
By 10th grade, if these students stay at the same percentile they will fail OAKS
This student will score a 236 and pass at the 37th percentile
Based on 2008-2009 ODE percentiles
Storytown Assessments
Assesses what was taught that week/unit The metric changes each week (i.e.
inferencing is more difficult than summarizing)
Not enough questions to truly guide individual student instruction
Oral Reading Fluency
• Same measure middle of first grade through eighth grade
• ORF is not designed to provide an exhaustive assessment.
• You can be fluent enough, unless you want to be an auctioneer!
• Strong link to comprehension• Accuracy matters!
Oral reading fluency and comprehension activity• Meet a new friend. Find a person who
you do not know in the room. (This is important because this is how these assessments are often given to students.)
• The person with the next birthday is the Test Administrator, paper A. The other person will be the Test Taker, paper B.
• Test Administrator, read your directions to yourself. Give the test and score.
• Test Taker, read the passage aloud and be prepared to answer questions.
Oral Reading Fluency
95% 98% 99%
The Secret Life of Bees
18.5 7.4 3.6
My Brother Sam is Dead
15 6 3
The Magic School Bus
6 2.4 1.2
Oral Reading Errors per Page at Different Levels of Accuracy
Richard Allington, 2009
Evaluate Overall Effectiveness of Program Are 80% of students reaching
benchmarks and “on track” for next goal? 90 minutes of reading daily? Protected allocated reading time each
day? Skill grouping by class or grade? Core and supplemental programs
implemented with fidelity? More professional development needed?
Indicators of the health of the core
Assessment Walls
13
10 3
16
121 3
66
6 6000
Intensive at Beginning of the year Strategic at Beginning of the year Benchmark at Beginning of the year
Which students do you need to take a closer look at? Lowest 20% (?) of students What do your resources allow?
Find the Students Belowthe Proficiency Guide Point
About 65% Meeting Minimum Proficiency
Lowest 20% identified to receive interventions
5th Grade MathMultiplication & Division 0-9
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 9910
010
110
210
310
410
510
610
710
810
911
0
Dig
its C
orre
ct T
wo
Min
utes
Fifth Grade Math
About 60% Meeting Minimum Proficiency
Screening Indicates Math Problem in Grade 5 Given The Standard – Do We Have a Problem With Our Core?
Lowest 20% identified to receive interventions
Make a School Plan
• Who will conduct Universal Screening? • Who will train the screeners?• Who will prepare materials?• Who will organize at the school?• Where will the data go?• Who will organize the data and present it
to teaching teams?• Who will keep track of which students are
in interventions?
Purpose of a Tier 1 meeting
•To determine the effectiveness of the core program•Make necessary adjustments if the core is not meeting the needs of most students
Tier One Meeting agenda
Purpose of meeting: To determine the effectiveness of the core program and make necessary adjustments. Determine if your system is moving children toward benchmark goals.
Review team norms Review purpose for meeting
1) Is our core program sufficient for most students? a) Review and analyze benchmark screening data.
i) How does this round of screening data compare to screening data from earlier in the year or last year? ii) What trends do you see in these data?
b) Review and analyze core program assessments. i) Are students making adequate progress? ii) What is the class average score in each category? iii) When analyzing weekly tests, look at the total questions given weekly in the unit and compute percentage
passed. Are 80% of students passing at 80%? c) Review and analyze OAKS assessments (in Fall review final spring data, Winter and Spring
progress made thus far) i) Review percentile needed for passing at the grade level. ii) Review of grade level percentage passing in the first try, second try, and third try. iii) Which strands need additional instruction?
2) What instructional adjustments are needed to improve the health of the core?
a) Discuss grade level wide opportunities to make core program more robust for this cohort. i) What agreements can the grade level make on common instructional strategies?
(1) How are you going to make the instruction more explicit? (2) How are you going to provide extra guided practice when needed? (3) How are you going to provide more corrective feedback to students? (4) How and when are you going to have students practice skills independently?
ii) What agreements can the grade level make on common active engagement strategies? How will you increase active engagement to increase the effectiveness of the instruction?
iii) What has worked? iv) What can you do to improve your teaching of the core to fidelity? v) How is your team using fidelity checklists to improve the instruction within the core? vi) What are the professional development needs?
b) What are the common instructional needs of the students in this grade level? c) Establish an end of the year goal to work toward for the percentage of student you would like
to see in each tier based on assessment data.
3) How can we support the students for whom the core is not sufficient? a) Identify the percentage of students who are falling in each tier based on the assessment data.
i) What trends do you see? ii) Are students in need of strategic instruction progressing towards benchmark standards? iii) Are students in intensive instruction progressing towards strategic or benchmark standards?
b) Identify the initial list of students who should be monitored more frequently than 3x per year using fluency measures.
c) Discuss what you are doing to challenge your highest skilled students.
Talk Time
Directions for 3-2-1 Activity
3 connections between the video and what you heard today
2 things you think would work in your school
1 question
Oregon RTI Video Clip
Talk Time
Directions for 3-2-1 Activity
3 connections between the video and what you heard today
2 things you think would work in your school
1 question
Assessments
Progress Monitoring Tools Why? What?
Maze EasyCBM AIMSweb ORF DIBELS Math Computation Math Applications Math Tests of Early
Numeracy Writing (Total Words
Written) Writing (Correct Word
Sequences)
Quick Phonics Screener QRI-IV CORE Multiple Measures
Assessment DRA2 Fountas and Pinnell Report cards Meeting OAKS standards Read Well Unit Tests, core
curriculum weekly tests on skills that are learned
Progress Monitoring Tools Not Progress Monitoring Tools
Progress Monitoring: Key Features
• Administration is standardized • Student performance is measured frequently (e.g.,
once a week, or every other week) and results in quantitative data
• Progress is monitored toward an observable, measurable, and ambitious goal
• Progress is graphed and viewed regularly• Data decision rules are used consistently
Essential Features of Reliable and Valid Progress Monitoring Tools
• Robust indicator of academic health • Brief and easy to administer• Can be administered frequently• Must have multiple, equivalent forms
(If the metric isn’t the same, the data are meaningless)
• Must be sensitive to growth
Why Monitor Progress?
It’s the GPS for educators It improves achievement It’s the law
Where are we?What is our goal?What course should we follow?How are we doing?
ActualCourse
DesiredCourse
Our Goal
We are Here
Effects of Progress Monitoring
• Progress monitoring has been extensively researched in Special Education (Fuchs & Fuchs, 1986)
• Students showed improved reading scores when teachers:• monitored their progress (+.70 effect size; ≈ 25th 50th %ile. Like it!)• graphed their reading scores (+.80 effect size. Love it!)• used decisions rules to determine whether to make a
change in instruction (+.90 effect size. Gotta have it!)
Progress Monitoring leads to good things…..
It’s the Law!
Teams must include for all SLD evaluations“data that demonstrate that prior to or as
part of the referral process the child was provided appropriate instruction in regular education settings, delivered by qualified personnel; and
Data based documentation of repeated assessments of achievement at reasonable intervals, reflecting formal assessment of student progress during instruction, which was provided to the child’s parents.”
This information is to be used to prompt evaluation as appropriate.
Sources for Reliable and Valid Monitoring Tools
National Center on Student Progress Monitoring (studentprogress.org)
DIBELS (dibels.uoregon.edu - Free to Oregon Schools)
AIMSWEB (www.aimsweb.com) Easy CBM (www.easycbm.com) Monitoring Progress of Basic Skills (Fuchs &
Fuchs; Reproducible masters) The ABC’s of CBM (Hosp, Hosp,& Howell)
TTSD EBIS DECISION RULES - Grades 1-5
So.. How do you progress monitor?
Key components Aimlines
Tell you where you need to go
Phase lines Tell you that you made a change
Trendlines Tell you if you are o the right track
Aimlines: Tell you where you need to go
Step 1: Aimlines
1. Plot the student’s score from the screening data using a X.
2. Draw a red line from the screening score to the end of year benchmark score (bullseye).
Brandon, first grade/Identify Goal and Draw Aimline
The aimline connects where you are to where you want to get to, and shows the rate of progress needed to get there.
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
M ayScores
JuneScores
60
50
Aimline
Cor
rect
wor
ds p
er M
inut
eAimlines
Step 2: Phase Lines
1. Plot the data on the chart by week 2. When there are four data points in a
row below the aimline (TTSD rule, BSD will make own decision rules) you need to make a change in your intervention.
Step 2: Phaselines tell you when to make a change
Whoops! Time to make a change!
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Cor
rect
Pho
nem
es
Aimline
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
MarchScores
AprilScores
MayScores
JuneScores
60
50
Aimline
Determine Response to InterventionWhen a student fails to make adequate progress we should change their instruction to help her meet the goal
Let’s Practice
19151812
18172129
374346
Step 3: Trendlines
Why do we need trendlines? To tell you where you are To tell you how intensive of a
modification/individualization you need to make to get back on track
Step 3: Draw a Trendline
1. Start with at least 7 data points2. Approximately divide the data points into
three equal sections using vertical lines. The outer two sections should have at least three data points
3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).
4. Mark the points on the graph where the two values intersect.
5. Connect the points to draw a trendline.
Trendline Example: Tukey Method
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Cor
rect
Pho
nem
es
Aimline
10 data points after baseline data divide points
Step 3: Draw a Trendline
1. Start with at least 7 data points2. Approximately divide the data points into
three equal sections using vertical lines. The outer two sections should have at least three data points
3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).
4. Mark the points on the graph where the two values intersect.
5. Connect the points to draw a trendline.
Trendline Example: Tukey Method
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Cor
rect
Pho
nem
es
Aimline
Mid-date & Mid-Rate
Step 3: Draw a Trendline
1. Start with at least 7 data points2. Approximately divide the data points into
three equal sections using vertical lines. The outer two sections should have at least three data points
3. In the first and third sections, calculate the mid-date (draw a vertical line) and mid-rate (draw a horizontal line).
4. Mark the points on the graph where the two values intersect.
5. Connect the points to draw a trendline.
Trendline Example: Tukey Method
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Cor
rect
Pho
nem
es
Aimline
Connect the points wiith a trendline
Trendlines
302730
34
3328
3247
283931
221920
22253038
31413245
5155
RN 30 min
+ Phonics for Reading
Trendline
Sam
Trendline FYIs
Some research suggests that you should have at least 14 data points to develop a reliable trendline
Step 4: Redraw aimline
• When an intervention needs to be changed, the aimline needs to be redrawn
1. Using the last three data points, find the intersection of the mid-date & mid-rate.
2. Draw a new aimline from this point to the end of the year benchmark.
3. Draw a phase line to reflect that the intervention has changed (after that point).
Redrawing Aimlines
302730
34
3328
3247
283931
Use mid-date & mid-rate of last 3 data points & redraw aimine
Progress Monitoring Chant
Remember, you have to share this information with the rest of your school.
This is a GLAD (Guided Language Acquisition Design) strategy
Progress Monitoring Chant
We all know ‘cause we’ve been toldProgress monitoring is worth its weight in goldWatch the data, I’ll tell you whyIt won’t leave you high and dry
Progress (Progress)Monitoring (Monitoring)Make informed instructional decisions
everyday (everyday)
Progress Monitoring Chant
When we monitor, students learn moreImproved decision making for all we adoreIt doesn’t measure just one skillMonitoring measures pace towards the goals we
need to fulfill
Progress (Progress)Monitoring (Monitoring)Make informed instructional decisions everyday
(everyday)
Important Factors to Examine Match between skill need and
intervention Group Size is small
provides enough explicit instruction and feedback
Time in program is sufficient
Organizing Fluency Data:Making the Instructional
Match
Group 1: Accurate
and Fluent
Group 2:Accurate but Slow
Rate
Group 3:Inaccurate and Slow
Rate
Group 4:Inaccurate but High
Rate
Regardless of the skill focus, organizing student data by looking at accuracy and fluency will assist teachers in making an appropriate instructional match!
Organizing Fluency Data:Making the Instructional Match
Group 1:
Accurate and Fluent
Group 2:
Accurate but Slow
Rate
Group 3:
Inaccurate and Slow
Rate
Group 4:
Inaccurate but High
Rate
Group 1: Dig Deeper in the areas of reading comprehension, includingvocabulary and specific comprehension strategies.Group 2: Build reading fluency skills. (Repeated Reading, Paired Reading, etc.) Embed comprehension checks/strategies.Group 3: Conduct an error analysis todetermine instructional need. Teach to the instructional need paired with fluency buildingstrategies. Embed comprehension checks/strategies.Group 4: Conduct Table-Tap Method. If studentcan correct error easily, teach student to self-monitor reading accuracy. If reader cannot self-correct errors, complete an error analysis to Determine instructional need. Teach to the instructional need.
Purpose of a Tier 2 Meeting
• To determine which students are in need of intervention, select appropriate interventions based on protocol, schedule intervention groups, and schedule progress monitoring.
• Determine effectiveness of interventions• To review progress monitoring for all
students in interventions (20% group) and make decisions about whether to proceed with, discontinue, or change an intervention.
Reading Protocol
Contacts
Dean Richards [email protected] 503-431-4135
Lisa Bates [email protected] 503-431-4079
Jon Potter [email protected] 503-431-4149