Upload
marcus-garrison
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Gibbs 2011 1
Dr. Denise P. Gibbs, DirectorAlabama Scottish Rite Foundation Learning
PSTRtISTUDENTSUCCESS
RTI Implementation: Assessment Components
Gibbs 2011 2
This presentation is provided at no cost to Alabama schools by the Alabama
Scottish Rite Foundation. The philanthropy of the Alabama Scottish
Rite Foundation began in the 1950’s in Alabama and continues today. The
mission of the Alabama Scottish Rite Foundation is to provide help to
Alabama Schools as they work with students who struggle in reading -
particularly those students with dyslexia.
Gibbs 2011 3
Anticipation Guidethen…Turn and Talk
As soon as you are seated, please As soon as you are seated, please complete the Anticipation Guide included complete the Anticipation Guide included
in your handout.in your handout.
Then…turn to your neighbor and talk Then…turn to your neighbor and talk about your initial answersabout your initial answers
Bell ringer activityBell ringer activity
Gibbs 2011 4
Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001)
Type of Strategy Effect size
%ile gain
# of studies
1. Identifying similarities and differences
1.61 45 31
2. Summarizing and note taking 1.00 34 179
3. Reinforcing effort and providing recognition
.80 29 21
4. Homework and practice .77 28 134
5. Nonlinguistic representations .75 27 246
Gibbs 2011 5
Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001)
Type of Strategy Effectsize
%ilegain
# of studies
6. Cooperative learning .73 27 122
7. Setting objectives and providing feedback
.61 23 408
8. Generating and testing hypotheses
.61 23 63
9. Questions, cues, advance organizers
.59 22 1251
Gibbs 2011 6
Today’s High Impact Strategies 9. Questions, cues, and advance organizers –
Anticipation Guide, session outcomes, RTI Vocabulary, Double Bubble
6. Cooperative learning – Turn and Talks 2. Summarizing and note taking – Power Point
slides, your notes, 3-2-1, 1. Similarities and differences – CBM vs CAT -
Double Bubble 5. Nonlinguistic representations - VVWAs 4. Homework and practice – utilize at least one
of these strategies in your work
Gibbs 2011 7
Session outcomes…. Today, you will:
Practice high impact strategiesIdentify uses of high stakes tests such as
the ARMT or AHSGE in the RTI screening process.
Analyze essential criteria to guide selection or development of RTI screening and progress monitoring tools.
Use the National Center for Response to Intervention's (NCRTI) tools chart to compare some commercially available screening and progress monitoring tools.
Gibbs 2011 8
RTI Essential Vocabulary: Assessment
Screening process vs tool Progress monitoring process vs tool Classification accuracy
False positives? False negatives? Reliability and predictive validity of the
performance level and slope of improvement
Reliability and predictive validity of the slope of improvement
Sensitivity to student improvement Efficiency!!!!!!
Gibbs 2011 9
Screening Process and Examples of Some Tools
Gibbs 2011 10
Designing your screening process Commercially available screening tools
Curriculum-Based Measures (CBM) Computer Adapted Testing (CAT) Standards-Connected Assessments
High stakes tests ARMT Alabama High School Graduation Exam End of course tests
Other variables Grades Behavior Attendance
Gibbs 2011 11
Using High Stakes Tests in the Screening Process
A starting point in screening older students could be to use the results of the ARMT or the AHSGE. Students who score a I or II on the ARMT in
reading or in math could be considered as at risk and could proceed to the next step in the district’s screening process
AHSGE results could be analyzed to determine the relative severity of the student’s deficiencies
Determine if the student would benefit from remediation classes or is in need of a more basic skill-level intervention class
Gibbs 2011 12
Double Bubble Time! As an advanced organizer, compare and contrast
CBM and CAT
Gibbs 2011 13
Three types of screening measures Curriculum Based Measures
Probes which look at skills which predict that a student will not be successful in reading or math.
Computer Adapted TestingAnswers determine subsequent questions.
Can learn more about skill levels with fewer test items.
Standards Connected AssessmentsUsually a fixed set of questions that mirror
standards-based state assessments and provide practice and performance prediction.
Gibbs 2011 14
Independent Review of Screening Tools National Center for Response to
Intervention has charts summarizing their review of various screening tools.
Some of the tools reviewed as of Spring 2011 are:AIMSweb - CBMDIBELS - CBMSTEEP - CBMSTAR - CATDiscovery Education (Think Link) –
Standards-connected
Gibbs 2011 15
Curriculum-Based Measures (CBM)
Was initially synonymous with RTI. If concerned about reading, have
student complete various reading screening probes
If concerned about math, have student complete math computation and math reasoning/problem solving screening probes.
AIMSweb, DIBELS, and STEEP are all based on CBM
Gibbs 2011 16
A look at some reading CBM probes
Early Literacy MeasuresLetter Naming FluencyLetter Sound FluencyPhonemic Segmentation FluencyNonsense Word Fluency
R-CBM (8th grade example)
Grades 1-12 Mazes (7th grade example)
Grades 2-12
Gibbs 2011 17
A look at some math CBM probes Early Numeracy Measures
Quantity discriminationMissing NumbersOral CountingNumber Identification
Computation (M-CBM) (1st – 6th grade example)Grades 1-12
Concepts and Applications (M-CAP) (7th grade example)Grades 2-8
Focal points (STEEP)
Gibbs 2011 18
Computer Adapted Testing (CAT) The best computer adapted testing is based
on Item Response Theory (IRT) IRT is a statistical framework in which
examinees can be described by a set of one or more ability scores that are predictive, through mathematical models, linking actual performance on test items, item statistics, and examinee abilities.
Test items are determined by responses to questions
Score is reported as a scaled score. STAR is an example of CAT based on IRT
Gibbs 2011 19
Computer Adapted Testing (CAT) STAR Early Literacy
PreK-Grade 3 41 skills in 7 domains 10 minutes
STAR Reading Grades 1-12 10 minutes
STAR Math Grades 1-12 10-15 minutes
Gibbs 2011 20
Standards Connected Assessments
Test includes a fixed number of grade-specific questions reflecting common core or state standards.
Can serve as a screening tool for students who might not be likely to perform satisfactorily on state assessments.
Can serve as a practice test for state assessments.
Generally, will not be able to show growth over time and monitor progress.
Gibbs 2011 21
Reports from Screening Tools
Some of the most helpful screening tools provide reports which reflectStudent’s percentile scoresStudent’s expected performance on high
stakes tests (ARMT!)Longitudinal data for a group of studentsLevel of proficiency on key common core
standards Instructional grouping/planning
suggestions
Gibbs 2011 22
Turn and Talk:Where are we in terms of
screening process?
In the bags, quick fixes, long-term growth
opportunities!
Gibbs 2011 23
Progress Monitoring Process and Examples of
Some Tools
Gibbs 2011 24
What progress to monitor?
Growth of skills in areas recognized as deficientFoundational academic skills in reading or
math Growth of appropriate social and
behavior skills Growth of content knowledge and
information
Gibbs 2011 25
Progress Monitoring Process
Use of a formal commercially available progress monitoring tool or
Use of a formal teacher/school developed progress monitoring tool
Should also consider additional information which can reflect progress Use of work samples reflecting baseline
and current workObservation of classroom behavior and
participation
Gibbs 2011 26
NCRTI Progress Monitoring Tools Chart
Provides administration information Compares tools according to key
reliability, validity, slope, and improvement criteria.
Many of the screening tools have also been favorably reviewed as progress monitoring tools.
But some screening tools were NOT found to have favorably reviewed progress monitoring probes.
Gibbs 2011 27
Key progress monitoring criteria
Reliability of the performance level scorethe extent to which the score (or
average/median of 2-3 scores) is accurate and consistent.
Validity of the performance level scorethe extent to which the score (or
average/median of 2-3 scores) represents the underlying construct
Gibbs 2011 28
Key progress monitoring criteria
Reliability of the slope of improvement an indicator of how well individual differences in
growth trajectories can be detected using a particular measure
Predictive validity of the slope of improvement the extent to which the slope of improvement
corresponds to end-level performance on highly valued outcomes.
Sensitivity to student improvement the extent to which a measure reveals
improvement over time, when improvement actually occurs.
Gibbs 2011 29
Progress Monitoring Tools
Need to be efficient. Need to be able to provide a reliable
baseline. Need to be able to reflect growth over
time. Need multiple, equivalent probes. Results need to be easily graphed.
Gibbs 2011 30
Progress Monitoring Decisions
Skills to be progress monitored? Do you have the probes you need? Frequency of progress monitoring
Weekly? Bi-weekly? Monthly? Who will conduct progress monitoring Who will score? Who will enter data in
computer? Print reports?
Gibbs 2011 31
Some reading skills to be targeted…
Word-level needs General benchmark should be 95%
accuracy for grade-level texts To reach desired accuracy, may need
to target both regular (decodable) words and irregular (learned) words
Rate needs to be that designated for student’s grade or 90-110 words correct per minute (whichever is less)
Prosody/inflection patterns!!!!
Gibbs 2011 32
Some reading skills to be targeted…
Comprehension needs Vocabulary Active reading strategies
Before, During, After Magnificent Seven!
Metacognitive strategies Fix-up strategies
Gibbs 2011 33
Math skills to be targeted… Computation automaticity
I am learning the facts I can think of the answer I can not avoid seeing the answer
when I see the problem. Grade-specific problem solving and
applications Concrete-representational-abstract
continuum Subject specific skills
Gibbs 2011 34
Selection of Progress Monitoring Tools Tools should be able to be administered weekly,
should be efficient and should be sensitive to growth If improved reading comprehension is the goal, then
you may elect to use mazes or computer adapted testing
If improved reading accuracy is the goal, then you may elect to use oral reading fluency passages and graph percentage of accuracy
If improved math problem solving skills is the goal, then you may elect to use timed problem solving probes or computer adapted testing
If improved math calculation skills is the goal, then you may elect to use timed calculation probes
If improved behavior is the goal, then you may elect to use behavior report card points earned .
Gibbs 2011 35
Verbal and Visual Word Association
SensitivityEfficiency
Gibbs 2011 36
Developing Progress Monitoring Probes Interventioncentral.com equips you to
generateMaze passages – from your own textsR-CBM passages – from your own textsWordlist fluency probes Math fact fluency probesBehavior report cards!
Establish baseline and set goal
Gibbs 2011 37
Developing Progress Monitoring Probes To monitor progress in learning
content, you can develop a set of vocabulary matching probes (Espin, et al., 2005).Comprehensive list of terms and
definitions for the course/subjectRandomly generated probes with 25 or
more terms given as frequently as desired.
Graph number or percentage correct
Gibbs 2011 38
About norms for locally developed PM measures
You could initially use norms included in ABCs of CBM and other resources to give you a “ballpark idea” of student’s performance level
But remember, since you are measuring the student’s progress from baseline, you can determine growth without much attention to norms!
Goal setting will require some creativity!
Gibbs 2011 39
Goals setting with locally developed PM measures
You could determine the performance levels on your probes of your most proficient students and use those levels in developing goals for students in interventions.
You could use “norms” included in ABCs of CBM and other sources to set goals
Over time, you could determine scores which correspond to successful performance on ARMT or AHSGE and set those as goals.
Gibbs 2011 40
Graphing progress monitoring data Commercially available progress
monitoring tools are equipped with graphing features.
Results of teacher or school developed progress monitoring probes can be graphed with a number of graphing tools including the Data Management Tool available on the ALSDE website and at this Dropbox URL
http://dl.dropbox.com/u/24788238/GraphingtoolGibbs.xls
Gibbs 2011 41
Some valuable resources
Gibbs 2011 42
The ABCs of CBM: A practical guide to curriculum-based measurement
(Hosp, et al., 2007)
Instructions regarding creating progress monitoring probes
R-CBM, Mazes, Math Computation, Math Problem Solving, Written Expression, Spelling, Early Literacy, Early Numeracy
Expected levels of performance for grades 1-8
Gibbs 2011 43
Assessing Reading Multiple Measures for All Educators Working to Improve Reading
Achievement (2nd ed, 2008)
Phonics Screener Phoneme Deletion Test Phonological Segmentation Test Phoneme Segmentation Test Graded High Frequency Word Survey San Diego Quick Assessment of Reading
Ability Maze passages Oral reading fluency passages
Gibbs 2011 44
3-Minute Reading Assessments: Word Recognition, Fluency, and Comprehension
(Rasinski and Padak, 2005)
Includes a 16 point multidimensional fluency scale!!Expression and volumePhrasing and intonationSmoothnessPace
Gibbs 2011 45
Mathematics RTI: A Problem-Solving Approach to Creating An Effective Model
(Allsopp, et al., 2010)
Includes specific recommendations for analyzing mathematics curriculum as it relates to struggling students.
Includes discussion of responsive mathematics teaching practices.
Gibbs 2011 46
RTI in Mathematics (Riccomini and Witzell, 2010)
Excellent tools to facilitate assessment, instruction, and intervention forNumber senseWhole numbersFractions and decimalsProblem solvingMathematical vocabulary!
Gibbs 2011 47
Anticipation Guide Revisited
Gibbs 2011 48
THANK YOU!RTI for Early Readers: Implementing
Common Core Standards in Your K-5 RTI Model (LRP, 2011)
RTI for Middle and High Schools: Strategies and Structures for Literacy Success
(LRP,2008)
Leading the Dyslexia Challenge: An Action Plan for Schoolwide Identification and
Intervention (LRP, 2004)