AIMSWEB CURRICULUM BASED MEASUREMENTS DPS District RtI Team

AIMSWEB AIMSWEB CURRICULUM BASED CURRICULUM BASED MEASUREMENTSMEASUREMENTS

DPS District RtI Team

OutcomesOutcomes

To know the four purposes for assessmentTo understand what makes an assessment a

Curriculum Based Measure (CBM)To be able to differentiate between a

Mastery Measure and a CBMTo know why CBMs offer a unique way to

progress monitor student growth

Permission to reprint granted with appropriate acknowledgement ©2009 DPS

What is What is ReliableReliable Assessment? Assessment?

Reliability of the assessment refers to the stability or consistency of test scores.

To have confidence in assessment, we would expect a similar score if the students were tested:a) On a different day.b) By a different tester.c) A set of comparable items.


What is What is ValidValid Assessment? Assessment?

Validity of assessment refers to evidence that the test measures what it is supposed to measure.

Validity is established by showing that the test agrees with or predicts scores on other accepted measures.

Screening tests must have predictive validity. ◦ For example, DIBELS Oral Reading Fluency is

predictive of performance on outcome assessments such as CSAP


What is What is EfficientEfficient Assessment?Assessment? It takes a minimum amount of time to give.

It is not expensive.

The data is easy to interpret and manage.

Key skills have been selected for measurement.

There is a big pay-off in terms of the usefulness of the information gathered.


Why do we give tests?Why do we give tests?

States, districts, schools, and teachers need to know whether they are:

a) reducing the number of “failing” students;

b) increasing the number of students reading at grade level or above; and

c) increasing the percentages of students in ethnic, racial, and low-income populations who are reading at grade level or above.


An effective, comprehensive An effective, comprehensive assessment plan includes assessment plan includes assessments to accomplish four assessments to accomplish four purposes:purposes: Outcome—to provide a bottom-line

evaluation of the effectiveness of the instructional program.

Screening—to determine which children are at risk for academic difficulty and who will need additional intervention.

Diagnosis—to help teachers plan instruction by providing in-depth information about students’ skills and instructional needs.

Progress Monitoring—to determine if students are making adequate progress or need more intervention to achieve grade level outcomes.


What is Screening?What is Screening?

All children are assessed.Each child’s assessment is brief, 10

minutes or less.Screenings are given at the beginning,

middle, and end of the year.The purpose is to flag children who are at

risk for not being at grade level.Examples: CBMs (DIBELS , AimsWeb,DRA-2

(CBLA data), Spelling Inventories, SRI


Use of Screening DataUse of Screening DataScreening is useful for predicting who is

likely to have academic difficulty.

The primary purpose of screening is to identify children before they fail and make sure the instruction is appropriate.

Identification is not enough! Screening is only valuable if it leads to differentiated instruction.


What is Progress What is Progress Monitoring?Monitoring?

Assessing a child every few weeks on alternate forms of a task, such as oral passage reading fluency, to measure growth resulting from instruction

Why Monitor Progress?Why Monitor Progress?Children’s responses to instruction cannot

be assumed.

Students with true disabilities can be identified by poor response to instruction; refer them for additional evaluation.

We need to document objectively whether students are on the right track toward closing the gap.


Progress: The Teacher’s MapProgress: The Teacher’s Map


10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Aimline

A change in intervention

p. 16

2 Types of Progress 2 Types of Progress MonitoringMonitoringMastery Based

◦Running Records◦Unit Tests (Teacher created and publisher

created)

◦Spelling TestsCurriculum Based Measures

◦DIBELS◦AimsWeb


Defining Mastery Based Defining Mastery Based MeasurementMeasurement

MASTERY MEASUREMENT Teachers have been

doing it for years Tracks Mastery of a

Discrete Skill Follows an set sequence

of skills Uses a criterion-

referenced test for each skill.

Once the skill is mastered, teachers move to the next instructional skill in the sequence.

2nd Grade Example

Phonics

◦ cvc patterns ◦ cvce patterns ◦ cvvc patterns . . .

Sight Vocabulary Comprehension

◦ Identification of who/what/when/where

◦ Identification of main idea

◦ Sequence of events Fluency


Defining Curriculum Based Defining Curriculum Based MeasurementMeasurement

Curriculum-Based Measurement

Informs how students are progressing in reading

Result of 30 years of research

Used across the country

Demonstrates strong reliability, validity, and instructional utility

2nd Grade CBM Example Students read aloud from

a grade-level passage. Each passage has the

same difficulty. As each student reads,

teacher marks errors. The number of words read

correctly is counted. Scores are put on a

graph. Progress is tracked

frequently to shape instructional decisions


Curriculum Based Curriculum Based MeasurementMeasurement

Curriculum-Based Measurement (CBM) is a method teachers use to find out how students are progressing in basic academic areas such as math, reading, writing, and spelling.

Result of 30 years of researchUsed across the countryDemonstrates strong reliability,

validity, and instructional utility


Research on CBMsResearch on CBMsThe CBM score is an overall indicator of reading

competenceStudents who score high on CBM

− Are better decoders− Are better at sight vocabulary− Comprehend better

CBM is sensitive to student improvement.CBM corresponds strongly with high-stakes tests. When teachers use CBM to inform their

instructional decisions, student achievement improves.

CBMs to be valid for ELLs demonstrating .64 -.77 correlation with comprehension skill levels (Baker & Good, 1995; Fien et.al., 2008)

CBM’s Can Be Used For…CBM’s Can Be Used For…Universal Screening As a part of a BOE to create

instructional groups.Progress monitoring

◦Required at Targeted and Intensive Levels

Engaging students in their own progress!


Understanding the Role of Understanding the Role of CBMs in the DPS RtI CBMs in the DPS RtI Assessment SystemAssessment SystemCBM’s are a necessary addition to our

tool box.They do not replace Mastery

Measurement; They are an additional tool for more

standardized, frequent information regarding a student’s progress toward goals.


AIMSweb Reading AIMSweb Reading www.aimsweb.com

OutcomesOutcomesUnderstand what each subtest is

measuring.Understand why each test is

administered.Understand how to administer each

subtest.Understand how to score each

subtest.

Height and Weight are Height and Weight are IndicatorsIndicators of of Physical DevelopmentPhysical Development

How Do We Use AIMSweb? How Do We Use AIMSweb? Types of AssessmentTypes of Assessment

Benchmark assessment◦ At Grade Level◦ All students 3-4 times per year

Progress monitoring◦ At Instructional Level◦ Students who need support more frequently

Middle and Low Trajectories Middle and Low Trajectories for Second Gradersfor Second Graders

Wo

rds

Pe

r M

inu

te

Identify Students Early

Reading trajectories cannot be identified by reading measures until the end of first grade.

Wo

rds

Pe

r M

inu

te

Identify Students EarlyIdentify Students EarlyNeed for AIMSwebNeed for AIMSweb

Wo

rds

Pe

r M

inu

te

We We CANCAN Change Change TrajectoriesTrajectories

How?Identify students early.Focus instruction on Big Ideas of

literacy.Focus assessment on indicators of

important outcomes.

GPS On Where are we?

You are Here

Port

Where do we want to be ?

How do we get there?

How are we doing?

ActualCourse

DesiredCourse

AIMSweb Probes are the GPS AIMSweb Probes are the GPS for Educatorsfor Educators

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Cor

rect

Pho

nem

es

Aimline

The AIMSweb Subtests

Reading-CBMReading-CBM(Oral Reading Fluency – (Oral Reading Fluency – ORF)ORF)

Mid First Grade – 12th Grade

What Is Reading Fluency?What Is Reading Fluency?Fluent readers read words with:

◦Automatic recognition.◦Quick and accurate decoding.◦Comprehension.

Reading fluency can be measured by:

◦Rate and accuracy.◦Words read correctly per minute.

p. 130

Is ORF just word calling?Is ORF just word calling?If wake up music, put on rock denim, speak orange jeans juice you for breakfast, and drink English, those of are part your things culture. If eat you bread for flat speak breakfast, Arabic, and long wear a to cotton protect robe you the from hot, those sun things part are of culture your.

p. 138

(Original passage excerpted from Rewards, page 250,

by Anita Archer, Sopris West)

What do you think?What do you think?If you wake up to rock music, put on denim jeans, drink orange juice for breakfast, and speak English, those things are part of your culture. If you eat flat bread for breakfast, speak Arabic, and wear a long cotton robe to protect you from the hot sun, those things are part of your culture. When some people hear the word culture, they think of priceless paintings and classical symphonies. Culture, as used in geography, is the way of life of a group of people who share similar beliefs and customs. These people may have the same language and religion.

p. 138

(Original passage excerpted from Rewards, page 250,

by Anita Archer, Sopris West)

Reading Fluency:Reading Fluency:

Reading passageAdministered individually

Timed for one minute

Errors marked as student reads three (3) stories

Teacher records middle score

Administration RulesAdministration Rules Time for a continuous minute At the end of one minute place a bracket (]) Emphasizing Words Read Correctly (WRC). 3-Second Rule.

◦ If a student stops or struggles with a word for 3 seconds, tell the student “the word” and mark it as incorrect. •

Discontinue Rule. ◦ If the passage that is read is so hard that the student reads less than 10

words correctly in 1 minute, discontinue administration of any other passages from that level and use this WRC score.

Speed Reading. ◦ You may encounter some excellent readers who may view R-CBM as a

“speed reading test” (i.e., read the passage very fast and without expression) in their first R-CBM testing.

◦ When this occurs, interrupt the student, saying “This is not a speed reading test. Begin again, and be sure to do your best reading.”

37

Let’s watch a video of the test being administered.

General Scoring Rules General Scoring Rules

WHAT IS A WORD READ CORRECTLY (WRC)?◦ Correctly Pronounced Words within context◦ Self-Corrected Incorrect Words within about 3

seconds

WHAT IS AN ERROR?◦ Mispronunciation of the Word or Substitutions◦ Omissions◦ 3-Second Pauses or Struggles

WHAT IS NOT INCORRECT (NEITHER A WRC OR ERROR)?◦ Repetitions◦ Dialect Differences◦ Insertions (Consider them Qualitative Errors)

Now let’s try a few Now let’s try a few togethertogetherModeling of test administration &

ScoringGuided Practice

Reading MAZEReading MAZE

Mid First Grade – 12th Grade

MAZE:MAZE:Reading passageGroup

AdministeredTimed for three

minutesScored after

assessment in completed

Teacher records total correct out of total attempted

DirectionsDirections1. Pass Maze task out to students. Have students write

their names on the Cover Sheet so they do not start early. Make sure they do not turn the page until you tell them to.

2. Say to the students:

"When I say 'Begin' I want you to silently read a story. You will have 3 minutes to read the story and complete the task. Listen carefully to the directions. Some of the words in the story are replaced with a group of three words. Your job is to circle the 1 word that makes the most sense in the story. Only 1 word is correct."

3. Decide if practice test is needed. (If needed, follow detailed practice script)

Directions (cont.)Directions (cont.)4. Start the testing by saying…

“When I say 'Begin' turn to the first story and start reading silently. When you come to a group of three words, circle the 1 word that makes the most sense. Work as quickly as you can without making mistakes. If you finish a/ the page/first side, turn the page and keep working until I say 'Stop' or you are all done. Do you have any questions?” (Answer student questions)

5. Then say, “Begin.” Start your stop watch.

6. Monitor students to make sure they understand that they are to circle only ONE word.

7. If a student finishes before the time limit, collect the student's Maze task and record the time on the student's test booklet.

8. At the end of 3 minutes say: “Stop. Put your pencils down. Please close your booklet.”

9. Collect the Maze tasks.

General Scoring Rules General Scoring Rules What is Correct? Student circles the word that matches the correct word on the

scoring template.

What is Incorrect? An answer is considered an error if the student:

a. circles an incorrect word.

b. omits word selections prior to last word circled in 3 minutes

Making Scoring Efficient

1. Count the total number of items up to the last circled word.

2. Compare the student answers to the correct answers on the scoring template.

3. Mark a slash [/] through incorrect responses.

4. Subtract the number of incorrect answers from the total number of items attempted.

4. Record the total number of correct answers on the cover sheet followed by the total number of errors (e.g., 35/2, 45/0)

AIMSweb

Early Literacy

Administration and ScoringAdministration and Scoring

Nonsense Word FluencyNonsense Word Fluency

Why Nonsense Words?Why Nonsense Words?Assess student rapid accuracy with

association of sounds with symbols.

Assess how students attack unknown or multisyllabic words.

Research has shown this subtest to be valid for ELLs demonstrating a .64 -.77 correlation with comprehension skill levels (Baker & Good, 1995; Fien et.al., 2008)

• A Page of 2 to 3-letter nonsense words

• Timed for one minute

• Number of correct sounds recorded

• All short vowels!!!

Nonsense Word FluencyNonsense Word Fluency

• Student copy of NWF (not numbered)

• Examiner copy of NWF (numbered)

• Clipboard

• Timer

• Student practice page. (bim lat)

Nonsense Word FluencyNonsense Word Fluency Materials Materials

Practice ItemsPractice Items

bim lat

Standard Directions“Look at this word (point to the first word on the

practice probe). It’s a make-believe word, not a real word. All the letters have sounds: (point to the letter “b”) /b/, (point to the letter “i”) /i/, (point to the letter “m”) /m/. Altogether the sounds are /b/ /i/ /m/ (point to each letter) or “bim” (run your finger fast through the whole word). Remember, it is a made up word. You can say the sounds of the letters, /b/ /i/ /m/ (point to each letter), or you can say the whole word “bim” (run your finger fast through the whole word). Be sure to say any sounds you know.

“Ready? Lets try one. Read this word the best you can (point to the word “lat”). Point to each letter and tell me the sound or tell me the whole word.”

Place the student copy in front of the child.

“Here are some more make-believe words (point to the student probe). When I say “begin”, start here (point to the first word), go across the page (point across the page), and read the words the best you can. Remember, you can tell me the sound of the letter or read the whole word. Put your finger underthe first word. Ready, begin.”


Directions for ScoringDirections for Scoring

1. Underline each letter sound the student provides correctly, either in isolation or in the context of the nonsense word.

2. Put a slash (/) over each letter sound read incorrectly.

3. Do not mark any letter sounds omitted, added, or repeated.

55

WHAT IS A CORRECT SOUND?

The number of correct sounds are counted.

• Correct Letter Sounds. ( z a t )

• Correct Words. ( zat )

• Self-Corrections. (within 3 seconds)

WHAT IS AN INCORRECT SOUND?

• Incorrect Letter Sounds. Put a slash (/) through errors.

• 3-Second Rule.◦ Say the sound/word ,

◦ Mark it as incorrect by drawing a slash (/) through it.

◦ Point to the next sound and say “what sound/word?

OTHER RULES

• Repeated Sounds. Letter sounds pronounced twice while sounding out the word are given credit only once.

• Insertions. Insertions are not scored as incorrect. Discontinue. If the student does not get any sounds correct in

words 1-5, discontinue the task and record a score of 0.

• Skipped Row. If a student skips an entire row, draw a line through the row and do not count the row in scoring.

General General Scoring Scoring RulesRules



Administration and ScoringAdministration and Scoring

Phonemic Segmentation Phonemic Segmentation FluencyFluency

What Is a Phoneme?What Is a Phoneme?Phonemes are the smallest oral segments into which a word can be divided.Spoken words are combinations of speech sounds.

shoe = /sh/ /ū/ (2 phonemes)stove = /s/ /t/ /ō/ /v/ (4 phonemes)

Highly predictive of later reading outcomes

(Adams, 1999)p. 70

• Single word is read by adult (oral)

• Student breaks word into individual sounds

• Timed for one minute

• # of correct sounds are marked

Phonemic Segmentation FluencyPhonemic Segmentation Fluency Materials Materials

• Examiner copy of PSF (numbered)

•Clipboard

• Timer

Phonemic Segmentation FluencyPhonemic Segmentation Fluency Materials Materials

“I am going to say a word. After I say it, I want you to tell me all the sounds in the word. So, if I say, “Sam,” you would say /s/ /a/ /m/. Let’s try one. . Tell me the sounds in “mop.”

Correct ResponseCorrect Response Incorrect ResponseIncorrect Response

Very good.Very good. The sounds in “mop” are /m/ /o/ /p/. The sounds in “mop” are /m/ /o/ /p/. Your turn. Tell me the sounds in “mop”. Your turn. Tell me the sounds in “mop”.

Phonemic Segmentation Fluency

Practice Items

“OK, here is your first word.” (say word and start timer)

Administration RulesAdministration Rules

3 SECOND WAIT RULE ◦ Maximum time for each sound segment is 3

seconds. ◦ If the student does not say the next sound

segment within 3 seconds, say the next word. DISCONTINUE RULE:

◦ If a student has not given any sound segments correctly in the first 5 words, discontinue the task and record a score of zero (0).

63


Scoring RulesScoring RulesWHAT IS A CORRECT SEGMENT?There are multiple ways a correct segment can be scored. Students may receive credit for any different, correct, part of the

word represented by sounds that correspond to the word part. Students may receive credit for

◦ Complete segmentation,

◦ Incomplete segmentation, and

◦ Overlapping segmentation.

◦ Schwa sounds,

◦ Additions, and

◦ Elongated sounds

WHAT IS NOT A CORRECT SEGMENT?• Omissions. The student does not receive credit for sound segments

that are not produced. If student

• No segmentation: If the student repeats the entire word, no credit is given for any correct parts.

• Segment mispronunciation: The student does not receive credit for sound segments that are mispronounced.



Administration and Scoring Administration and Scoring Letter Naming FluencyLetter Naming Fluency

Letter Naming is anLetter Naming is anAdded Measure of Added Measure of RiskRisk

Serves as an indicator of risk in conjunction with scores on other AIMSweb measures.

The skill of letter naming has a high correlation with later reading outcomes (Adams, 1999)

Random upper and lower case letters


Errors marked as student reads

Letter Naming Fluency Letter Naming Fluency

1. Student copy (not numbered)

2. Examiner copy (numbered)

3. Clipboard

4. Timer

Letter Naming Fluency: Letter Naming Fluency:

Materials Materials

DirectionsDirections1. Place the student copy in front of the student.2. Place the examiner copy on a clipboard and

position it so the student cannot see what the examiner records.

3. Say these specific directions to the student:“Here are some letters (point to the student copy).

Begin here, (point to first letter) and tell me the names of as many letters as you can. If you come to a letter you don’t know, I’ll tell it to you. Are there any questions? Put your finger under the first letter. Ready, begin.”

4. At the end of 1 minute, place a bracket ( ] ) after the last letter named and say, “Stop.”

Administration RulesAdministration Rules3 SECOND WAIT RULE

◦ Maximum time for each sound segment is 3 seconds. ◦ If a student does not provide the next letter with 3 seconds, tell the

student the letter name and mark it as incorrect. Point to the next letter and say, “what letter?” This prompt can be given only one time.

DISCONTINUE RULE: ◦ If the student does not get any correct letter names within the first

10 letters (1 row), discontinue the task and record a score of 0.PROMPTING RULE:

◦ “Remember to tell me the letter name, not the sound it makes.” This prompt may be provided once during the administration.


Scoring RulesScoring RulesWHAT IS A CORRECT LETTER NAME?• A correctly named letter.• Confused I’s and L’s as a function of font. For these letters, either name is considered

correct.• Self-Corrections. Within 3 seconds, write “SC” above the letter and do not count it as an error.

WHAT IS AN INCORRECT LETTER NAME?• Substitutions of a different letter for the stimulus letter (e.g., “P” for “D”).• Omissions of a letter.• Stops or struggles with a letter for more than 3 seconds.

74



Administration and ScoringAdministration and Scoring Letter Sound FluencyLetter Sound Fluency

Random lower case letters


Errors marked as student reads

Letter Sound FluencyLetter Sound Fluency

• Student copy of LSF (not numbered)

• Examiner copy of LSF (numbered)

•Clipboard

• Timer

Letter Sound Fluency:Letter Sound Fluency: Materials Materials

DirectionsDirections1. Place the student copy in front of the student.2. Place the examiner copy on a clipboard and

position it so the student cannot see what the examiner records.

3. Say these specific directions to the student:“Here are some letters (point to the student copy).

Begin here, (point to first letter) and tell me the sounds (with emphasis) of as many letters as you can. If you come to a letter you don’t know, I’ll tell it to you. Are there any questions? Put your finger under the first letter. Ready, begin.”

4. At the end of 1 minute, place a bracket ( ] ) after the last letter named and say, “Stop.”

Administration RulesAdministration Rules3 SECOND WAIT RULE

◦ Maximum time for each sound segment is 3 seconds. ◦ If a student does not provide the next letter with 3 seconds, tell the

student the letter name and mark it as incorrect. Point to the next letter and say, “What sound?”

DISCONTINUE RULE: ◦ If the student does not get any correct letter names within the first 10

letters (1 row), discontinue the task and record a score of 0.PROMPTING RULE:

◦ If the student says the letter name rather than the letter sound say, “Remember to tell me the sound (with emphasis) the letter makes, not its name”.


Scoring RulesScoring RulesWHAT IS A CORRECT LETTER SOUND?• Students must provide the most COMMON sound of the letter.

(pronunciation guide for most common sounds is in the Administration Manual appendix)

• Confused I’s and L’s a function of font. For these letters, either sound is considered correct.

• Self-Corrections. Within 3 seconds, write “SC” above the letter and do not count it as an error.

WHAT IS AN INCORRECT LETTER SOUND?• Substitutes a different sound for the stimulus letter sound.

• Substitutes the less common sound for the stimulus letter sound.

• Omission of a letter sound

• Stops or struggles with a letter sound for more than 3 seconds.Skipped Row - draw a line through the row and do not count the

row in scoring.



General considerationsGeneral considerationsAdministration, timing, and

scoring need to be exact, according to standardized instructions.

This is the key to maintain the reliability and validity of the assessment!

The Plan From Here…The Plan From Here…

Complete benchmark testing at grade level with your identified students

Use data for . . . . .◦Determining instructional needs of individual students

◦Grouping decisions◦Deciding level of intensity for intervention

Documents

AIMSWEB CURRICULUM BASED MEASUREMENTS DPS District RtI Team