CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model Tom Jenkins, Ed.D. Educational Consultation Services, LLC

CBM, Charting, Progress Monitoring, and Data Based

Instructional Decision Making in a RTI Model

Tom Jenkins, Ed.D.

Educational Consultation Services, LLCWilmington, NC

What you can expect today

• CBM, CBA, CBE

• Instruction on progress monitoring

• Examples and hands-on activity

• Final thoughts and conclusions

What is CBM?

• Curriculum-based measurement– Data collection tools derived directly from the

curriculum that student is expected to learn

CBM

• CBM is believed to reduce the gap between assessment and instruction

• Aides teachers in generating superior student achievement

• Improved communication• Higher level of sensitivity• Enhancement of the database• Administration time is shorter• More cost effective

CBM

• The simplicity of CBM allows for quick and easy peer referencing

• Normative data can be collected• This allows for comparison of a student’s

performance to his/her actual peer group• More representative geographically,

culturally, ethnically, and has been exposed to similar instructional environment

CBM

• CBM has been shown to posses high levels of reliability

• 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between .90-.99 with just three one-minute administrations (Jenkins, 2002)

CBM

• Discriminant Validity• Several studies have demonstrated the

ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

CBM Procedures

– Remember different modalities– Given the nature of reading probes they must

be administered individually– Math and written expression may be

administered in a group setting

CBM Procedures

• Scoring– One minute administration time except for

written expression– Number ID is scored corrects for minute;

however, math computations are scored digits correct per minute

– Written expression is scored according to correct sequences

Example

• Administration example

• Work in pairs

• Each person gets a chance to adminster

CBM Procedures

• Written expression examples

Progress Monitoring

• Basic Principle #8 of PSM– Progress monitoring an essential aspect of

the intervention phase

• Basic Principle #9 of PSM– Decision making in regards to the

effectiveness of an intervention is based on analysis of progress monitoring data in relation of goal

Progress Monitoring

• Essential for four reasons– There is no guarantee that interventions will be

successful, thus the intervention must be “tested” to evaluate effectiveness

– Increased emphasis of specific outcomes for students, data base must be generated to guide intervention decision making

– Pre/post testing has be shown to be unreliable (small amount of data) and provides too little data to allow for instructional decision making – progress monitoring allows for evaluation of level of performance and rate of learning

– Research has shown that progress monitoring is associated with improved educational outcomes

Progress Monitoring

• Definition– Frequent and repeated data collection (no

less than 2-3 times a week) and analysis of student performance

– Data is collected during intervention and provides basis for intervention effectiveness

Progress Monitoring

• Essential components that must be in place for successful progress monitoring– A well-defined behavior– A measurement strategy– Identification of student’s current level of performance

(baseline)– Intervention– Goal– Graph– Decision-making plan

Well Defined Behavior

• Target behavior, observable, measurable, and specific

• Focus on enabling skills– Skills that are prerequisite skills for more

complex skills– Deficiencies in enabling skills often adversely

affects performance on global assessments


• Enabling skills for reading– Phonemic awareness– Alphabetic understanding– Fluency– Sight words– Comprehension


• Enabling skills for math– Number sense– Facts– Computation– Applications– Problem solving

• Enabling skills for written expression– Mechanics– Expression


• Enabling skills for behavior– Social skills– Work completion– Compliance– Problem solving skills

Measurement Strategy

• Frequent and repeated collection of data

• Time and cost efficient

• Sensitive to changes over short periods of time

Current Level of Performance (Baseline)

• Gathered prior to intervention

• Repeatable

• Provides comparison for progress data

• Helps set goal

• Median score – why?

Intervention

• Match intervention to problem

• Humans tend to employ interventions with which they are comfortable instead of intervention that the student needs

• Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data

• No intervention works all of the time for every student

Goal

• Standard against which progress can be compared• Allows for aimline to be established• Possible goals

– Norms– Percentile cutoff– Realistic growth rates– Ambitious growth rates– Minimum celeration– Local growth rates

Computing Growth Rates

• Winter Norm minus Fall Norm, divided by number of weeks between norming projects

• Gives you a growth expectancy for each week of school year

• Allows for obtaining student’s baseline then monitoring progress while comparing to growth expectancy

Computing Growth Rates

• Example – first grade, ID words in sentences

• Winter Norm minus Fall Norm, divided by number of weeks between norming projects

• 67.9119-35.2694 / 10 = 3.2643 words per week

Growth Rates based on research by Doug and

Lynn Fuchs• Realistic Growth Rates• Gr 1 2 words/week• Gr 2 1.5 wrds/week• Gr 3 1 words/week• Gr 4 .9 words/week• Gr 5 .5 words/week

• Ambitious Growth Rates

• Gr 1 3 words/week• Gr 2 2 words/week• Gr 3 1.5 wrds/week• Gr 4 1.1 wrds/week• Gr 5 .8 words/week

Goal

• 1.25 minimum celeration, Precision Teaching

Graph

• Provides a visual representation of a large amount of data

• A visual representation of student’s acquisition of skills and allows for easier analysis of progress

• Semi-log chart– Equal interval charts can misrepresent data,

depending on how axis is quantified– Equal interval charts assumes equal amounts of

progress between all data points– Precision Teaching

Decision Making Plan

• Facilitates interpretation of data

• Should include– Rule for raising performance goal – 6/4

consecutive data points above the aimline– Rule for altering the intervention due to lack of

progress – 3/4 consecutive data points below the aimline

– Deno/Allison


• If the decision is to adjust an intervention, small changes or refinements are recommended before major changes

• However, changes should be substantial enough that it has a possibility to result in improved student performance

• If making an adjustment, do not make two at the same time. It may result in the team being unable to determine what caused increased student performance


• As student performance approaches goal, team must decide– Raise the goal– Begin work on another target behavior– Discontinue PSM process

• If in level four consider– Discontinuation of EC services

Charting Activity

• A means of interpreting large amounts of data

• Allows for relating performance trends to desired performance

• Research has shown the charting also facilitates positive student outcomes (Fuchs, 1989)

Charting Activity

• Definition– Visual depiction of the student’s performance

data, relative to the goal and aimline– Includes baseline data, goal, aimline, and

progress monitoring data

Charting Activity

• Procedures– Write the prediction/goal statement– Depict baseline data collection phase on the

chart, indicate the median score, identify with a heavy dark line

– Depict the goal at the end of the anticipated intervention phase

– Depict the aimline by connecting the baseline median with the goal, this gives you expected rate of progress

Charting Activity

• Activity– Plot the baseline data

• Monday – 14• Tuesday – 10• Wednesday – 6• Thursday – 5• Friday – 10

– Plot the median and signify with a heavy dark line

Charting Activity

• Plot the goal at the end of eight weeks– 40

• Indicate the aimline

Charting Activity

• Performance trends should be analyzed periodically– Trend above aimline – raise the goal– Trend below aimline – adjust intervention

• If changes are made to the intervention, indicate change on the graph with a squiggle line

• Describe the changes on the back of the chart• This allows for understanding of specific

instructional adjustments that were successful/unsuccessful

Charting Activity

• Plot the first two weeks progress monitoring data– Week One

• Tuesday – 16• Thursday – 14

– Week Two• Monday – 10• Wednesday – 18• Friday – 16

• Make an informed decision regarding the effectiveness of the intervention

Charting Activity

• Remember– Rule for raising performance goal – 6/4

consecutive data points above the aimline– Rule for altering the intervention due to lack of

progress – 3/4 consecutive data points below the aimline

– Deno/Allison

Charting Activity

• Plot progress monitoring data for next two weeks– Week Three

• Tuesday – 14• Thursday - 10

– Week Four• Monday – 15• Wednesday – 16• Friday – 16


Charting Activity

• Phase two or three of the intervention?– Plot the progress monitoring of the next two weeks– Week Five


– Week Six• Monday – 26• Wednesday – 24• Friday – 26


Charting Activity

• Phase two or three of the intervention?– Plot the progress monitoring of the next two weeks– Week Seven


– Week Eight• Monday – 32• Wednesday – 36• Friday – 38


• Final decision?

Charting Activity

• More information than you wanted to know– Trend lines

• Line that you draw through a series of data points that represents the student’s actual rate of progress

• If trend line slope is flatter than aimline slope, then adjust intervention

• If trend line slope is steeper than aimline slope, then adjust goal

• If slopes are the same, make no change

Charting Activity

• Trend line procedures– Draw vertical line in middle of graph, half data

points on one side, half on the other – if odd number of data points, put line through middle point

– Draw a vertical line in first half of data to separate data half and half

– Draw a vertical line in second half of data to separate data half and half

Charting Activity

• Trend line procedures– Draw a horizontal line through median of first

half of data – form an intersection with vertical line

– Draw a horizontal line through median of second half of data – form an intersection with vertical line

– Connect the two intersections– Make your decision, is it the same decision?

Thoughts and Conclusions

• What if we took it a step further?

• What if we used the intervention/progress monitoring/charting process to move students towards a CBM reading fluency score that we knew predicted success on EOG’s and EOC’s?


• Espin et al, 2002 found correlation between CBM reading fluency scores and Minnesota grade 8 state standards test scores to be .78

• Grade 5 correlation between CBM reading fluency scores and Minnesota Comprehensive Assessments - .77


• Similar results were found in Minnesota for grades 3 and 5

• Similar results have been found in different states


• Similar results have been found in Oregon and Washington


• What does all this mean?– Implement a school-wide model of progress

monitoring– Screen all students in the fall– Identify students with CBM fluency scores that

make them “at risk” on the standards based testing

– Intervene and monitor progress of the students “at risk” – move them towards a CBM reading fluency score that predicts a higher likelihood of success on the standards based testing


• Goals can be set based on local norming data – find CBM reading fluency for those that did meet standard

• Growth rates – realistic or ambitious• Fluency benchmarks set by research (Deno)

– 1st grade – 60 wpm– 2nd grade – 90 wpm– 3rd grade – 120 wpm– 4th grade – 130 wpm– 5th grade – 140 wpm– 6th grade – 150 wpm


• How can I progress monitor with all the other things that I have to do?– School improvement occurs when focus is

placed on student improvement outcomes, priorities are going to have to be changed so that progress monitoring is seen as just as important as instruction


• We have standards based tests to measure student progress, why do more?– These tests are usually administered once a

year and show student success in meeting a criterion. Progress monitoring enables learning, shows if a student is making progress towards success on standards based tests, and allows for informed instructional decision making that helps the student reach the goal of success on standards based tests.

Questions?

• Dr. Tom Jenkins, Director• Educational Consultation Services, LLC• Wilmington, NC• (910) 367-7209• [email protected]• www.educationalconsultationservices.com

mailto:[email protected]

Documents

CBM, Charting, Progress Monitoring, and Data Based Instructional Decision Making in a RTI Model Tom Jenkins, Ed.D. Educational Consultation Services, LLC