77
PSM/RtI Formal Evaluation Level 3 at Elementary School Curriculum Based Assessment and/or Behavior Data

PSM/RtI Formal Evaluation Level 3 at Elementary School

Embed Size (px)

DESCRIPTION

PSM/RtI Formal Evaluation Level 3 at Elementary School. Curriculum Based Assessment and/or Behavior Data. Objectives. Define and discuss the following tools used in Responsiveness to Instruction: Curriculum-based Assessment and Norms Baseline data, Goals and Aimlines, Progress Monitoring - PowerPoint PPT Presentation

Citation preview

Page 1: PSM/RtI Formal Evaluation  Level 3 at Elementary School

PSM/RtI Formal Evaluation Level 3 at Elementary School

Curriculum Based Assessment

and/or Behavior Data

Page 2: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Objectives

Define and discuss the following tools used in Responsiveness to Instruction:– Curriculum-based Assessment and Norms– Baseline data, Goals and Aimlines, Progress

Monitoring– Graphing data– Decision-making rules– Revisions of Hypotheses/Interventions (Level 2 e)

Page 3: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Curriculum Based Assessment (CBA)

Data collection tools derived directly from the curriculum that student is expected to learn

Page 4: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Curriculum Based Assessment (CBA)

CBM is believed to reduce the gap between assessment and instruction

Aides teachers in improving instruction Improved communication Higher level of sensitivity Administration time is shorter More cost effective

Page 5: PSM/RtI Formal Evaluation  Level 3 at Elementary School

To Use CBAs in RtI: Normative Data is Required

Why do we do it? How is it useful?– To provide comparison data on skill levels of students

with the same curriculum and demographics of our county

When or how often do we do it?– Every 5 years

How do we do it?– Stratified Random Sample– Fall, Winter and Spring Windows

Page 6: PSM/RtI Formal Evaluation  Level 3 at Elementary School

ACADEMICS

Reading, Math and Written Language Probes; e.g., CBAs

Page 7: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Probes: Measuring Academic Skills

Reading: The Big Five Areas a Hierarchy– Phonemic Awareness– Phonics– Fluency– Vocabulary– Comprehension

Page 8: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Probes: Measuring Academic Skills

Math– Number Sense– Calculation– Reasoning

Written Expression– Conventions– Spelling

Page 9: PSM/RtI Formal Evaluation  Level 3 at Elementary School

EVALUATION THATINFORMS INSTRUCTION

Baseline Data: How do the student’s skills compare to those of peers (normative sample)

Goals and Aimlines: Where would we like the student’s skills to be at the end of the designated intervention period?

Progress Monitoring: Is the student developing the skills we are teaching?

Decision Rules: Do we need to change what we are doing?

Page 10: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Baseline Data: Comparison to Peers

How is Baseline Data Collected? • Student is given the grade level probes used

in norming• Administration starts with the first item on the

page each time• Over 3 days in one week• Median scores in each probe area are

compared to the grade level norms

Page 11: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Determining Where to Intervene

Skills of student compared to peers

Lowest level skills (the building blocks)

Page 12: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Back Sampling and Diagnostic Assessment

Back Sampling– If the student’s skills in the lowest skill areas on

grade level are weak, then baseline data is collected on the probes for the previous grade level to inform instruction.

Diagnostic Assessment– Can be completed by reviewing actual skills in an

area. Ex. Which sounds/blends/digraphs does the student know? Which math facts are known?

Page 13: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Skills: Discrete or Complex

Discrete Skills are generally taught by practice and over learning. These might include as phonemic awareness, blends, sight recognition, fluency, math calculation, etc.

Complex Skill Sets required a combination of skills to perform. These might include fluency, vocabulary, reading comprehension, math concepts, written expression, etc

Page 14: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Fluency: Discrete or Complex?

Fluency might be considered– Complex if it is the highest level skill on which the

staff are intervening with the student because the student is still working on building blocks or prerequisite skills such as phonological awareness, phonics and/or sight words

– Discrete if it is the lowest level skill on which the staff are intervening because the requisite skills are there, but the student needs practice for speed

Page 15: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Plotting Baseline Data

Student A--0, 5, 3– What is the median score?– Where do you plot it on the graph?

Student B—25, 38, 40 What is the median score?– What is the median score?– Where do you plot it on the graph?

Student C—0, 0, 10– What is the median score?– Where do you plot it on the graph?

Page 16: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 17: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 18: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 19: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Goals: What should we use?

Where would we like the student’s skills to beat the end of the designated interventionperiod? Considerations:• Is the skill a building block to higher level

skills?• Is the skill discrete or complex? • Is it likely to take more or less time and/or

intensity to bring the skills up to proficiency?

Page 20: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Types of Goals

1. District (NHCS) Norms

2. District Behavior Standards

3. Growth Rates• NHCS Data - Can be done for any skill, any

grade level for which norm data is available• Realistic/Ambitious (Fuchs and Fuchs)-Fluency

Only

4. Class Norms

5. Minimum Celeration Finder

Page 21: PSM/RtI Formal Evaluation  Level 3 at Elementary School

1. Using District Norms

Goals are selected from the norms tables by grade level. The norms tables are located in the Special Education and Related Services manual on the web.

Often the 25th percentile is chosen, but there are times when that is too ambitious for a student and the 13th is used instead.

Goals be selected by the date of the end point of the intervention. Example: If the baseline data was from the week of 10/26/09 (Fall Norms) but the goal is set for 6 weeks later (12/7/09), the goal should be selected from the Winter Norms.

Page 22: PSM/RtI Formal Evaluation  Level 3 at Elementary School

2. NHCS Behavior Standards:

Will be addressed later in the presentation

Page 23: PSM/RtI Formal Evaluation  Level 3 at Elementary School

3. Growth Rate: Using Growth Rate to Set a Goal

NHCS– Determine the number of weeks of intervention.– Multiply growth rate from chart by number of

weeks of intervention– Add to baseline median– Plot goal at number of weeks on the chart.

Page 24: PSM/RtI Formal Evaluation  Level 3 at Elementary School

3. Growth Rate: How Do We Calculate It?

• Take a seasonal norm (fall to winter or winter to spring)

• Subtract the earlier seasonal norm, and divide by 10 (for NHCS probes) or by 18 for DIBELS

• For example; Winter Norm minus Fall Norm, divided by ten weeks

Page 25: PSM/RtI Formal Evaluation  Level 3 at Elementary School

3. Growth rate calculation: Fall to Winter

CALCULATE: – First grade, ID words in sentences– Winter Norm minus Fall Norm, divided by ten weeks– 76.97- 48.15 / 10 = 2.82 words per week USE GROWTH RATE TO SET GOAL– For this reading fluency measure, students are

acquiring a little under 3 words per week, thus if a intervention plan is for six weeks then the student should acquire a little over 18 words in the six weeks

– Add 18 to baseline median to set goal.

Page 26: PSM/RtI Formal Evaluation  Level 3 at Elementary School

3. Growth Rate Calculations: Fall to Spring

CALCULATE: • First grade, ID words in sentences• Spring Norm minus Fall Norm, divided by 20 weeks• 99.07- 48.15 / 20 = 2.54 words per week USE GROWTH RATE TO SET GOAL– For this reading fluency measure, students are

acquiring about 2.5 words per week, thus if a intervention plan is for six weeks then the student should acquire a little over 15 words in the six weeks

– Add 15 to Baseline Median to set the goal.

Page 27: PSM/RtI Formal Evaluation  Level 3 at Elementary School

3. Using Growth Rates

Realistic/Ambitious Growth Rates for Fluency (Fuchs and Fuchs)– The process is the same as with NHCS Growth

Rates

Page 28: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Realistic/Ambitious Growth Rates (Fuchs and Fuchs)--Fluency

Realistic Growth Rates Gr 1 2 words/week Gr 2 1.5 wrds/week Gr 3 1 words/week Gr 4 .9 words/week Gr 5 .5 words/week

Ambitious Growth Rates Gr 1 3 words/week Gr 2 2 words/week Gr 3 1.5 wrds/week Gr 4 1.1 wrds/week Gr 5 .8 words/week

Page 29: PSM/RtI Formal Evaluation  Level 3 at Elementary School

4. Classroom Norms: Why?

You may choose to norm a class or grade level on a probe that was not used for state or local norming. Most likely when probes from previous grade levels need to be used to determine entitlement.

To do class or grade level norms, the school must:– Give the probe(s) to the entire group,– Administer each probe 3 times in a week.

You will probably want to do this Fall, Winter and Spring.

Page 30: PSM/RtI Formal Evaluation  Level 3 at Elementary School

4. Classroom Norms: How do you get one?

To do class or grade level norms, the school must:– Give the chosen probe(s) to a sample that is

representative of the school populationIn a school with heterogeneous grouping, the

student’s classroom will doIn a school with homogeneous grouping you

may have to balance out the student’s classroom with another or even norm the entire grade

– Administer each probe 3 times in a week.

Page 31: PSM/RtI Formal Evaluation  Level 3 at Elementary School

4. Classroom Norms: How do you get one?

Once you have the class or grade level data– Add all the students’ scores for all 3 days

together.– Compute the Mean score by dividing by the

total of the students’ scores by the total number of probe administrations. Ex. 3 administrations with 25 students

would equal a divisor of 75. You may want to do a mean for more than one

season: Fall and Winter, Winter and Spring

Page 32: PSM/RtI Formal Evaluation  Level 3 at Elementary School

4. Using Classroom Norms

Setting Goals: – Class or Grade Level Mean– Minimum Celeration Finder

Determining Whether a Student’s Skills are Discrepant After Intervention:– Divide the class or grade level Mean by the

Median of the Student’s last 3 data points.– Class Mean ÷ Student’s Median ≥ 2

Page 33: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Computing Two Times Discrepant

Ex. Class Mean is 40. Student Median is 20. 40 ÷ 20 = 2

Student is discrepant.

Non Ex. Class Mean is 40. Student Median is 22.

40 ÷ 22 = 1.89

Student is not discrepant.

Page 34: PSM/RtI Formal Evaluation  Level 3 at Elementary School

5. Minimum Celeration Finder

Celeration rates from the Precision Teaching program can be used for goal setting consideration

The lines on the overlay indicate different celeration slopes or rates of progress

Depending on how ambitious the team wants the goal to be different rates of progress can be used

Page 35: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Minimum Celeration

Page 36: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Plotting Goals and Aimlines

Student A--0, 5, 3 (Grade 2, Sight Words, Fall to Winter)– District Norm

– NHCS Growth RateExample uses 13th Percentile growth rateMay use the growth rate at 25th or 50th

percentile

Page 37: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 38: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 39: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Plotting Goals and Aimlines

Student B—25, 38, 40 (Grade 2, Words in Passage, Winter to Spring)– District Norm – Realistic/Ambitious Growth Rate

Page 40: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 41: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 42: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Progress Monitoring: Why?

Time and cost efficient Sensitive to changes over short periods of

time Frequent and repeated data collection

(dependent upon student’s level/tier) and analysis of student performance

Use data to inform instruction in specific skills.

Page 43: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Progress Monitoring

How does it differ from collection of baseline data?– Use alternative probes with the baseline probe

included as the cycle of alternative probes allows.– For probes such as phonemes, blends, sight

words, or math calculation, start at the line after the last line used on the third baseline probe.

– Alternate probes for reading fluency (Skill Builders and DIBELS); for reading comprehension (Blue binder books by grade level from Fuchs and Fuchs) and Math Concepts (Black line Masters—Orange binder book)

Page 44: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Progress Monitoring: When?

• Tiers 1 and 2: • May use DIBELS, ClassScapes, Reading or

Math Assessment, Probes, etc. • Generally less frequent or even pre-intervention

and post-intervention.

Page 45: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Progress Monitoring: When?

• Tier 3: • Grade Level Probes Normed by the County, • DIBELS K-2 (Local Norms), and/or Probes

normed in the classroom or grade level. • At least two probe areas • Must be given a minimum of 5 times in 2

weeks.

Page 46: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Type of Progress Monitoring

NHCS

Tier 1Depends on Intervention Plan

Evaluation Design

Tier 2Depends on

Intervention Plan Evaluation Design

Tier 3Normed

Probes in 2 Areas:

Page 47: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Frequency of Progress Monitoring

NHCS

Tier 1Depends on Intervention Plan

Evaluation Design

Tier 2Depends on

Intervention Plan Evaluation Design

Tier 3Every Other Day

Page 48: PSM/RtI Formal Evaluation  Level 3 at Elementary School

REVISIONS OF HYPOTHESES/INTERVENTIONS

How do we know when to change the intervention?

Page 49: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Decision-making rules: What type of skill is involved?

Discrete Skills such as phonemic awareness, blends, sight recognition, fluency, math calculation:– Consider 3 to 4 below the line to change

hypothesis/intervention– 4 to 6 above the line to raise goal and aimline or

discontinue (25th percentile) and move on the next skill in the hierarchy

Page 50: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Decision-making rules: What type of skill is involved?

Complex Skill Sets such as fluency, vocabulary, reading comprehension, math concepts, written expression, etc:– Consider how many days/weeks it might take to

show improvement or be certain skill is developed Given probe administration 5 times in two weeks:

How many data points below the line should be considered before changing hypothesis/intervention?

How many data points above the line should be considered before changing the goal or considering a change back to Tier 1 or 2?

Page 51: PSM/RtI Formal Evaluation  Level 3 at Elementary School

REVISIONS OF HYPOTHESES/INTERVENTIONS

How do you know what to change?

Page 52: PSM/RtI Formal Evaluation  Level 3 at Elementary School

REVISIONS OF HYPOTHESES/INTERVENTIONS

• Intervention Intensity• Does the data show progress toward the goal?

• Intervention Strategy/Hypothesis• Does the data NOT show progress toward the

goal?

Page 53: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Graphing Progress Monitoring and Revision of Hypotheses/Interventions

Level 3e

Page 54: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Graphs

Student A– Baseline 0, 5, 3– Week 1 3, 1, 6– Week 2 4, 5– Week 3 6, 6, 7

What does this data tell you? Would you change the hypothesis/intervention? If so, how? If not, why?

Page 55: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 56: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Graphs

Student B– Baseline 25, 38, 40 – Week 1 30, 15, 25– Week 2 30, 40, 25

What does this data tell you? Would you change the hypothesis/intervention? If so, when and how? If not, why?

– Week 3 35, 60– Week 4 50, 55, 60

What hypotheses would you have about the improvement given the choices you made at the end of Week 2?

Page 57: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 58: PSM/RtI Formal Evaluation  Level 3 at Elementary School
Page 59: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Trendlines

Another Measure of Growth Rate

Page 60: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Trend lines: How Do They Show Growth?

Line that you draw through a series of data points that represents the student’s actual rate of progress

If the trend line slope is flatter than the aimline, then adjust intervention

If the trend line slope is steeper than the aimline then adjust goal or area of intervention

If the slopes are the same make no change.

Page 61: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Trend Line Procedures: Can be done in Excel

- Draw a horizontal line through median of first half of data – form an intersection with vertical line

– Draw a horizontal line through median of second half of data – form an intersection with vertical line

– Connect the two intersections– Make your decision, is it the same

decision?

Page 62: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Trend Lines and Entitlement Criteria

Used to determine how long it will be before the student’s skills are likely to be proficient– Required for Behavior Entitlement– Can be used for Academics as well

Page 63: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Alternatives

– In academics, can add student’s current growth rate to the median of the last 3 data points to determine how many weeks it will take the student to meet proficiency standards.

Page 64: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Example: Short Term Intervention Needed

Student is in Grade 4 and is receiving intervention in reading fluency

On 3/1/10, the median of the last 3 data points for the student is 68

The student’s growth rate is 3 words per week The 25th percentile for Spring is 87 In 6 - 7 weeks, the student is likely to have

reached proficiency on this probe:– 87 – 68 = 19– 3 X 6 = 18

Page 65: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Example: Long-term Intervention Needed

Student is in Grade 4 and is receiving intervention in reading fluency

On 3/1/10, the median of the last 3 data points for the student is 68

The student’s growth rate is 0.5 words/week The student will need 36 to 37 weeks to become proficient

– 87 – 68 = 19– 0.5 x 36 = 18

By that time the student will be into the next school year and, possibly, in the next grade level with even more difficult reading passages to master.

Page 66: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Example: Long-term Intervention Needed

By that time the student will be into the next school year and, possibly, in the next grade level with even more difficult reading passages to master.

Page 67: PSM/RtI Formal Evaluation  Level 3 at Elementary School

WHAT DOES THE DATA SHOW?

Comparison to peers, e.g. normative data– Is the student above or below the 13th, 25th/30th or

50th percentile for academics– Has the student met behavior goals?

Page 68: PSM/RtI Formal Evaluation  Level 3 at Elementary School

WHAT DOES THE DATA SHOW?

Growth rate and trend lines– How does the growth rate compare to students in the

normed sample at the 13th, 25th, and 50th percentiles?– When does the trend line indicate that the student is

going to achieve proficiency?Academic—25th/30th percentileBehavior—75 percent or 100 percent

Services at Level III that approximate special education

Page 69: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Measuring Behavior

BASICS:

Attention Issues

Disruptive Behaviors

Page 70: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Setting Goals

For example a student that is exhibiting difficult maintaining attention to task, the intervention plan should be aimed at increasing his on task behavior to 75% of the time

A student that displays disruptive/dangerous behaviors should have an intervention plan that aims at increasing appropriate replacement behavior 100% of the time

Page 71: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Data: Interval Observation

 

• A certain time span is set aside for an observer to take data in the classroom.•The time span is divided into intervals of 10, 30, or 60 seconds. •A check is made in the box, each time the replacement behavior is exhibited within the interval.                                                   Peer                                    Interval  17   18   19   20   21   22   23   24 Target                Peer                                    Interval  25   26   27   28   29   30 Target            Peer                           Summary:___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Page 72: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Data: Star Charts

•Teacher records intervals without incidents (when the student is exhibiting replacement behaviors) in the intervals designated during the day.

•One example might be12 thirty minute intervals during each day. •Simply use checks on a chart for intervals without incident.

•Record your data on a Progress Monitoring Graph.

Page 73: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Behavior Data Collection Rules = Academic Data Collection Rules

A baseline of 3 using the median is required. Goals are set by 75 or 100 percent. Aimline is drawn. Progress Monitoring Data is taken Decision making rules and changes in

hypotheses/interventions apply

 

Page 74: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Progress MonitoringProgress MonitoringName: Toby Toms

Target Behavior:

Time without Assaults

Baseline (Median): 4

Goal: 12 or 100%

Median at end of 3 weeks: 6

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Days

12

10

8

6

4

2

M T W TH F M T W TH F M T W TH FBaseline = 54%

Day 1 = 4

Day 2 = 4

Day 3 = 3

Day 4 = 4

Day 5 = 5

Day 6 = 4

Day 7 = 5

Day 8 = 5

Day 9 = 6

Day 10 = 6

Day 11 = 6

Day 12 = 6

Day 13 = 6

Day 14 = 6

Day 15 = 6

Tim

e S

eg

men

ts W

ith

ou

t A

ssau

lts

Beh

avio

r

Week 1

Week 2

Week 2

Page 75: PSM/RtI Formal Evaluation  Level 3 at Elementary School

What Does the Data Show?

Comparison to peers Growth rate by trend line

– When will the trend line intersect with the goal line:

Less than 18 weeksMore than 18 weeks

Services similar to special education

Page 76: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Teaching Replacement Behaviors

What’s the difference between teaching a behavior and a reward system

Features of reward systems– Successive approximations– Positive and Negative Reinforcement

Page 77: PSM/RtI Formal Evaluation  Level 3 at Elementary School

Nancy Kreykenbohm, Ph.D.

Coordinator, PSM/RtI Program

School Psychologist Level III

Email: [email protected]

Phone: 910-616-3382