Using the Right Data to Change Instruction and Improve ...attachments.aplus.agora.com/t13_live/PEFE34.handout.8BB54.pdfUsing the Right Data to Change Instruction and Improve Reading

© 2012 Readsters, LLC 1

1 © 2012 Readsters, LLC 1 © 2012 Readsters, LLC

Using the Right Data to Change Instruction and Improve Reading Scores Presented at

National Title I Conference

Nashville, Tennessee January 23, 2013

Presented by

Amy Siracusano Calvert County Public Schools (MD) [email protected] &

Linda Farrell Readsters

[email protected]

2 © 2012 Readsters, LLC 2

Objectives

• To show specific data that led one school to change reading instruction for Title I students

• To show the results of changing reading instruction

• To provide two assessments that drove the changes in instruction


Topics

1.  Introduction

2.  Title I Data & Instruction: Before

3.  Catalysts for Change (Research)

4.  New Focus and New Data

5.  Two New Assessments

6.  Accuracy before Rate

7.  Title I Instruction: After

8.  Steps To Create Change

9.  One Student


1.  Introduction



Introduction

•  Calvert County Public School District (MD)

–  13 elementary schools

–  4 Targeted Assistance Title I Schools with a total of 8 Title I Teachers.

•  Barstow Elementary is one of the Title I schools

–  40% FaRMS

–  560 students PreK – 5

–  Located in Calvert County, MD

•  Amy became a Title I teacher in the 2009-10 school year. She is the only Title I teacher at Barstow this year.

•  Amy and Linda met in 2005 at a LETRS workshop.


2.  Title I Data & Instruction: Before


Data Used in 2008 through 2011 To Identify and Group Title I Students for Instruction

•  Used to Identify Students: –  DIBELS Oral Reading Fluency (ORF)

–  Words Correct Per Minute score –  Retell Score

–  County reading benchmark scores –  Maryland State Assessment scores

•  Used Only for Grouping Students: –  Developmental Spelling Inventory

(from LETRS Module 8)


Before: Grouping Spreadsheet

Sample: Grade 4 Rankings Beginning of Year

SCORES POINTS

TOTAL POINTS

DIBELS ORF

MSA Bench-‐mark

Teacher Rec?

DIBELS ORF

MSA Bench-‐mark

Teacher Rec? Name WCPM Retell Retell % of

WCPM WCPM Retell

V. Hugo 29 4 14% 336 26 y 3 1 3 3 1 11

E. John 42 13 31% 362 48 n 3 0 3 3 0 9

G. Ford 52 17 33% 374 48 n 3 0 2 3 0 8

C. Rock 66 10 15% 361 70 n 2 1 3 1 0 7

T. Fey 76 8 11% 366 70 n 1 1 2 1 0 5

J. Galt 68 28 41% 405 52 y 2 0 0 2 1 5

R. Hayes 74 48 65% 374 57 n 1 0 2 1 0 4

L. Byrd 54 22 41% 423 83 y 3 0 0 0 1 4

W. Earp 99 11 11% 381 87 n 0 1 2 0 0 3

G. Bush 72 10 14% 405 87 n 1 1 0 0 0 2

M. Twain 71 10 14% 406 83 n 1 1 0 0 0 2

J. Cash 67 20 30% 427 87 n 2 0 0 0 0 2

A. Ashe 61 45 74% 453 83 n 2 0 0 0 0 2

H. PoYer 101 80 79% 397 70 n 0 0 0 1 0 1

Selected for Title I Instruction



Instructional Programs

•  Fundations: phonics for students –  At Risk or Some Risk based on DIBELS data

–  Significant deficits on Developmental Spelling Inventory

–  Not proficient on state and county comprehension assessments.

•  Great Leaps: fluency for students –  At Risk or Some Risk based on DIBELS data


Grouping & Programs for Instruction

•  Fundations –  Title I teachers worked with groups of no more

than 8 students for 30 minutes daily

–  40 students serviced in grades 2 - 4

•  Great Leaps –  Para professionals worked 1-to-1 for maximum of

10 minutes

–  90 students serviced in grades 2 - 4


Before: Analysis of Results

•  Looked only at DIBELS

ORF rate score.

•  Improvement was sporadic

•  A few students were getting instruction that improved their rate.

•  Most students showed little or no improvement in rate.

Sample: Grade 4 Results Middle of Year

DIBELS ORF WCPM

Name Program Fall Winter Change

V. Hugo FundaZons 29 27 -‐2

E. John FundaZons 42 44 +2

G. Ford FundaZons 52 88 +36

C. Rock FundaZons 66 64 -‐2

T. Fey Great Leaps 76 82 +6

J. Galt Great Leaps 68 82 +14

R. Hayes Great Leaps 74 71 -‐3

L. Byrd Great Leaps 54 51 -‐3

W. Earp Great Leaps 99 87 -‐12


3.  Catalysts for Change: 1. The Simple View of Reading 2. Scarborough Rope Model



The Simple View of Reading (Gough & Tumner, 1986)

•  Reading difficulties can be classified into one of three categories: 1.  decoding 2.  language comprehension 3.  both

•  Students with decoding weaknesses have different instructional needs than students with language comprehension issues.

•  If students have both weaknesses, decoding instruction needs to be separate from language instruction.


The Simple View of Reading (Gough & Tunmer, 1986)

A formula introduced by Gough and Tunmer in 1986

Decoding (D) Language Comprehension (LC) =

Reading Comprehension (RC)

Note: Scores for D & LC are between 0 and 1.

D x LC = RC

•  The formula was demonstrated to work by as described in Hoover and Gough’s study, published in 1990. Correlations were very high, ranging from .84 in first grade to .91 in third and fourth grades.

•  The essence has been replicated in research many times since.

x

x


Relationship among Decoding, Language Comprehension & Reading Comprehension

Scores obtained: •  Decoding (D): Read nonsense words orally

•  Language Comprehension (LC): Listen to a story, retell the story, answer orally asked questions not covered in retell.

•  Reading Comprehension (RC): Read a story independently, retell the story, answer orally asked questions not covered in retell.

D x LC = RC

Given a decoding score and a language comprehension score, the reading comprehension score can be estimated by multiplying the two scores.


Simple View Depends on the Multiplication of Fractions

•  Each variable has a value of 0 (no competence) to 1 (perfect competence), or 0% to 100%.

•  The product will be less than either variable (unless both variables are 1 or 100%)

–  Example: D x LC = RC .90 x .40 = .36

•  A perfect RC score would require perfect D and LC scores:

1.0 x 1.0 = 1.0

Decoding (D) x Language Comprehension (LC) = Reading Comprehension (RC)



A Reading Comprehension Score Does Not Specify the Weakness

•  Two students with the same reading comprehension scores can have different underlying weaknesses.

D x LC = RC

.90 x .40 = .36

.40 x .90 = .36

Decoding (D) x Language Comprehension (LC) = Reading Comprehension (RC)

LC is weak

D is weak

RC is very weak for both students

Student 1

Student 2

D is strong

LC is strong


More Information on the Simple View

• Download the free article “The Simple View of Reading: Research of Importance to All Educators” 1.  Go to: www.Readsters.com

2.  Point to: “Resources” in upper right corner

3.  Click on: “Newsletter” in drop down menu

4.  Click on: “Readsters Reader, Vol. 1. No. 1, Winter 2010” near bottom of page

5.  Scroll down to third topic, Research Review, and click on “Read more…”


Lessons from the Simple View

•  Achieving reading comprehension requires strong decoding skills and strong language comprehension. –  We need to teach and test both decoding and language

separately for students with low literacy.

•  Strong language comprehension skills cannot compensate for low decoding skills. –  We need to teach decoding even when students with weak

decoding can answer questions based on their background knowledge.

•  Simply testing reading comprehension does not inform us whether the underlying weakness is decoding or language comprehension. –  We need to assess students with low DIBELS or MSA scores

further to determine the appropriate intervention.


New Questions from the Simple View of Reading

We answer these questions before deciding on intervention:

1.  Is the problem decoding?

2.  Is the problem language comprehension?

3.  Is the problem both?



The Scarborough Rope Model (Scarborough, 2001) Many strands that are woven together in skilled reading


Components of the Rope Model

Provides various components of word recognition and language comprehension that may be appropriate for instruction:

Language Comprehension Abilities –  Background Knowledge

–  Vocabulary

–  Language Structures

–  Verbal Reasoning

–  Literacy Knowledge

Word Recognition –  Phonological awareness

–  Decoding •  Onset rime units •  Syllable patterns •  Morpheme chunks

(prefixes, roots, suffixes)

–  Sight recognition of familiar words


Lessons from the Rope Model 1.  Word recognition skills and language comprehension

skills are different.

2.  Language comprehension improves largely through oral instruction when students are learning to decode.

3.  Language comprehension improves largely through reading when students are more skilled decoders.

4.  Strong readers have strong comprehension skills and strong word recognition skills.

5.  Any single skill that is weak can contribute to comprehension weaknesses.

6.  Students of any age can be missing basic skills in either language comprehension or in word recognition.


The Rope Model and Fluency

• Fluency results when students decode with automaticity and have internalized strategic reading skills. (Scarborough, 2001)

• Teaching fluency will not remediate underlying decoding or language comprehension weaknesses.

• Weaknesses in word recognition or in language comprehension can result in poor fluency and poor comprehension.



4.  New Focus and New Data


New Focus: Decoding

•  Diagnosis: –  Identify students with decoding

weakness (phonemic awareness or phonics)

•  Prescription & Implementation: –  Provide targeted instruction based on

level and area(s) of decoding weakness

Note: Comprehension and fluency instruction were addressed through differentiated instruction in classrooms.


Data Used in 2011 To Group Title I Students for Instruction

•  DIBELS Oral Reading Fluency (ORF) WCPM score

–  Added Accuracy scores

•  Added Decoding Surveys

•  Added Pre-Reading Probes for students with low decoding scores

•  County reading benchmark scores

–  Broke into two pieces: Word Work & Comprehension

•  Maryland State Assessment scores

–  Broke into two pieces: Word Work & Comprehension


5.  Two New Assessments: Decoding Surveys Pre-Reading Probes



Assessing Decoding

• Diagnostic Decoding Surveys (Really Great Reading)

–  Free copy and video available at www.rgrco.com.

• Administered to all students considered for Title I.

30 © 2012 Readsters, LLC 30 (c) Really Great Reading Company 2006 30

22 one-syllable words in sentences: short vowels & high frequency words

Beginning Decoding Survey

50 Very Easy One-Syllable Words

Words and Sentences to Read

5 high frequency words

5 real words with short vowels & 3 letters

5 real words with short vowels & 4 letters (digraphs)

5 real words with short vowels & 4 letters (blends)

•  4 with 3 letters

8 nonsense words with short vowels

•  4 with 4 letters (digraphs)

31 © 2012 Readsters, LLC 31 (c) Really Great Reading Company 2006

3rd grade student’s Beginning Decoding Survey

The student is a guesser:

•  8 sounds added or omitted

•  7 final consonant errors

The student has short vowel and blend weaknesses

•  9 short vowel errors

•  4 blend errors

2 8 9 32 © 2012 Readsters, LLC 32

(c) Really Great Reading Company 2006 32

Advanced Decoding Survey

30 Harder Words

8 nonsense words with short vowels

12 nonsense words with advanced vowels

4 nonsense words with other vowels

4 nonsense words with long vowels

3 nonsense words with r-controlled vowels

5 nonsense multi-syllable words

5 real multi-syllable words

10 multi-syllable words


33 © 2012 Readsters, LLC 33 (c) Really Great Reading Company 2006

4th grade student’s Advanced Decoding Survey

Has minor difficulties with:

•  short vowels (2 errors)

•  digraphs (ph, and ch on the beginning decoding survey)

•  advanced vowels (4 errors)

•  unfamiliar multi-syllable words (2 errors)


Assessing Pre-Reading Skills

•  Pre-Reading Probes –  Free copy and directions available at readsters.com.

•  Administered to all students with low decoding scores.


6.  Accuracy before Rate


A Question for You? Two fourth grade students. DIBELS benchmark is 105 WCPM. •  Sally

–  117 WCPM (well above benchmark) –  95% accuracy (6 errors) –  Added 3 words not in the passage and not counted as errors.

•  Billy –  87 WCPM (18 below benchmark) –  100% accuracy –  Read exactly what is on the page.

The question: You have $500 to bet on which student will score better on comprehension tests. Who do you bet on?



Mother told us that she has had straight A’s until this year, and she is struggling to get B’s now. Teacher complained: “I worry about her comprehension. She decodes well most of the time. She understands beautifully sometimes, and sometimes she doesn’t get the concept at all.”

4th Grade Student


4th Grade DIBELS Passage (6th edition) 167 WCPM (Benchmark = 105)

98% accuracy (7th edition benchmark =


Accuracy Is Important because Errors Change Meaning

Read as a question

Changed meaning

Not read as a question Changed meaning


7.  Title I Instruction: After



Identification Process

1.  SCREENING: All students entered into a ranking matrix based on: –  DIBELS ORF: WCPM, Accuracy & Retell scores –  Word Work scores on county and state assessments

2.  DIAGNOSIS: Lowest 30 – 50 students identified as potential Title I students who take: –  Decoding Surveys –  Pre-reading Probe, if needed

3.  SELETION: 25 students with poor decoding selected for Title I intervention


Better Data Led to Better Instructions and Better Results

• Focus on elements of reading scores, not just the composite score, from standardized tests.

• Added accuracy scores on DIBELS.

• Assessed decoding skills.

• Assessed pre-reading skills.


New Instruction for Decoding

• Programs for Decoding Weakness

–  Phonics Boost (Really Great Reading) + decodable books and customized spelling lessons: for students with moderate to significant weaknesses.

–  Phonics Plug-In (draft from Readsters) and various decodable books for lowest students.


New Instruction for Pre-Reading Skills

•  Instruction for Pre-Reading Skills Weaknesses –  Letter Name and Sound Identification

•  Used materials we made or purchased to practice letter names and letter sounds to automaticity.

–  Phonological Awareness •  Used draft edition of Readsters’ Phonological

Awareness Notebook (product will be available soon for purchase) for instruction at syllable, onset-rime, and phoneme levels. We taught students to manipulate phonemes, in addition to blending and segmenting.



Grouping Sizes and Instructional Time

•  Maximum 4 students per group

•  1-to-1 with very low students

•  30 minutes daily


8.  Steps To Create Change


Major Steps Barstow Took To Create Change Academic Year 2010-2011

SUMMER 2010 1.  Developed a research-based framework for thinking about and dealing with

reading problems.

2.  Learned how to administer diagnostic assessment for students identified as At Risk by DIBELS.

3.  Identified phonics and phonemic awareness instructional materials. SEPTEMBER 2010 4.  Assessed At Risk students using diagnostic assessments and analyzed data

5.  Selected 25 Title I students based on decoding weaknesses OCTOBER 2010– MAY 2011 6.  Implemented phonics instruction and early reading skills instruction DECEMBER 2010–MARCH 2011 7.  Gave 4 full-day workshops about assessment and instruction for struggling

readers to teachers with Title I students MAY 2011 8.  Assessed students with DIBELS and decoding surveys or pre-reading probes


Teacher Comment

I have truly enjoyed, as well as benefitted from the training I have received over the past year and a half in conjunction with Title I/Readsters.

Since the focus has been on fluency and phonics, I have been able to diagnose gaps my students may have and get a more in-depth view of their abilities. Once I have diagnosed weak areas, I am able to use fluency techniques and recommendations that I have learned in Readsters.

Because I have implemented these techniques, my students are showing significant gains in reading fluency, and with increased fluency, they are showing gains in comprehension as well.

With the gains, I am also seeing a boost in student self-esteem.

I have learned a tremendous amount and this program has filled in some gaps that were not covered in my undergraduate or graduate programs. Carol Corken 4th Grade Teacher Barstow ES



9.  One Student


Brandon – DIBELS and Decoding Survey scores, December 1st grade

• Brandon moved to Calvert County in December and was enrolled in 1st grade at Barstow Elementary.

• The principal identified him as a student with reading problems that needed special attention. She consulted the Title I teachers about how to help Brandon.

• Brandon became a Title I student in 2nd grade.


Brandon’s DIBELS Scores Were Low: December 1st grade

DIBELS, middle of 1st grade

•  PSF . . . 46 – above 35 benchmark

•  NWF. . . 38 – below 50 benchmark (all letter sounds)

1 – word recoded correctly

•  ORF. . . . .1 – below 20 benchmark

Notes: •  PSF = Phoneme Segmentation Fluency •  NWF = Nonsense Word Fluency •  ORF = Oral Reading Fluency


Typical Course of Instruction for Students with Brandon’s Scores

• Fundations to help with phonics.

• Title I teacher would deliver instruction

–  30 minutes daily

–  Group no larger than 8 children



Brandon Needs Further Assessment

DIBELS, middle of 1st grade •  PSF . . . 46 – above 35 benchmark

–  Brandon appears to have adequate phonemic awareness.

•  NWF. . . 38 / 43 letter sounds attempted – below 50 benchmark

1 – word blended / below 8 benchmark

–  Brandon’s 5 errors were all with vowels. He appears not to have mastered short vowel letter sounds.

–  Brandon appears to struggle with blending letter sounds into words.

•  ORF. . . . .1 / 7 words attempted – below 20 benchmark

14% accuracy

–  Brandon struggles immensely with text.


Brandons’ Beginning Decoding Scores January, 1st Grade

Notes: •  t/d confusions at end of words

–  7, 8, 19, 44, 48

•  Short u vowel substitutions –  8, 15, 17

•  Short i usually correct –  7, 18, 23, 29, 36, 40, 45, 48 –  Exceptions: 11, 14, 31

•  Short a correct –  6, 13, 20, 42, 47 –  Exception: 37

•  High Frequency Words: –  Errors: one, they, are, were –  Correct: see, you, the, a, is, my


Brandons’ Pre-Reading Skills Scores: Overview January, 1st Grade

Weaknesses:

•  Orthographic I. Letter names

II. Letter sounds

•  Phonological IV. Phoneme Blending & Segmenting

2. Blending 4 sounds 3. Segmenting 4 sounds

V. Phoneme Chaining


Brandons’ Pre-Reading Skills Scores: Letter Names

Letter Sounds January, 1st Grade

Weaknesses:

•  Letter Names –  j named as i

–  s named as e

•  Letter Sounds –  b sounded as /d/

–  e sounded as long e

–  Did not attempt digraphs



Brandons’ Pre-Reading Skills Scores: Phoneme

Blending & Segmenting

Weaknesses:

•  Blending 4 sounds –  Can hold on only 3 sounds

•  Segmenting 4 sounds –  Puts 2 or 3 sounds together


Brandons’ Pre-Reading Skills Scores: Phoneme

Chaining

Weaknesses:

•  Cannot quickly or accurately identify the one sound that changes between minimal pairs (pairs of words that differ only by one sound).


Brandon: New Course of Instruction First Grade

First grade teachers focused first on pre-reading skills: Orthographic issues:

–  letter confusions: b/d; i/j; p/q; e/s Phonological issues:

–  Blending and segmenting sounds in orally spoken words: first 2 sounds, then 3 sounds, then 4 sounds

–  Identifying and labeling vowel sounds in words –  Chaining sounds in words –  Identifying vowel sounds in spoken words

High frequency words: –  10 h/f words: the, I, a, of, is, you, was, for, to, and


Brandon: Progression of Instruction

First Grade •  Mastered pre-reading skills during in-class intervention. •  Exposed to phonics during Fundations whole-class

instruction.

Second Grade •  Entered Title I. •  Phonics instruction with Plug-In lessons (draft lessons;

available for purchase soon from Readsters) and Power Readers (Sopris West) decodable books.

Third Grade •  Did not qualify for Title I. •  Classroom teacher uses Supercharged Readers (Sopris

West) decodables and targeted phonics in small groups. •  Comprehension is also a focus.



Brandon – DIBELS and Decoding Survey scores, Dec. 1st grade & Sept. 3rd grade

DIBELS ORF Dec. 1st Grade

Sept. 3rd Grade

WCPM 1

at risk Benchmark = 20

61 some risk

Benchmark = 77

Accuracy 14% 95%

Retell NA 33

Decoding Surveys Jan. 1st Grade

Sept. 3rd Grade

Beginning (50 total) 22 49

Advanced (30 total) NA 20


Brandon: DIBELS ORF & NWF Scores Dec. 1st grade - Sept. 3rd grade

1st Grade 2nd Grade 3rd Grade Mid End Beg Mid End Beg Mid

DIBELS ORF

WCPM 1 17 13 38 57 61

Accuracy 13% 74% 62% 88% 95% 95%

Retell 0 18 8 35 21 33

Accuracy before rate!!!!


What Made the Difference in Effectiveness of Intervention? •  Using the right data

–  We pinpointed weak areas in pre-reading skills using the pre-reading skills survey.

–  We continuously analyzed the data to know specifically what to teach.

•  Providing targeted instruction based on data –  Initial intervention focused on the pre-reading skills

that were missing, instead of starting instruction with phonics.

–  Further intervention remained targeted toward weaknesses based on data, rather than just placement in a single program identified for everyone.


References Gough, P., Hoover, W., and Peterson, C. 1996. Some observations on a simple

view of reading. In Cornoldi, C. and Oakhill, J. (Ed.), Reading comprehension difficulties: processes and intervention (pp. 1–13). New York, NY: Routledge.

Gough, P. and Tunmer, W. 1986. Decoding, reading, and reading disability. Remedial and Special Education, 7, 6–10.

Hoover, W. and Gough, P. 1990. The simple view of reading. Reading and writing: an interdisciplinary journal, 2, 127–160.

Moats, L. (2005). Language essentials for teachers of reading and spelling. Longmont, CO: Sopris West.

Scarborough, H. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory, and practice. In S.B. Newman & D.D. Dickinson (Eds.), Handbook of early literacy research (pp. 97–110). New York: Guilford Press.



Assessment Materials

DIBELS •  Universal screening and progress monitoring for

grades K - 6 •  Website: dibels.uoregon.edu

Diagnostic Decoding Surveys

•  Assesses basic and advanced decoding skills •  Website: rgrco.com

Pre-Reading Probes

•  Asseses letter names, letter sounds, and phonological awareness

•  Website: readsters.com


Instructional Program Materials

Wilson Fundations Basic Spelling and Phonics for grades K - 3 www.fundations.com

Phonics Blitz Phonics Intervention for grades 4+ Website: rgrco.com

Phonics Boost Phonics intervention for grades 2+ Website: rgrco.com

Phonics Plug-In (publication scheduled for 2013) Early phonics and phonemic awareness intervention for grades 1 – 3 Website: readsters.com

Phonological Awareness Notebook (publication scheduled for 2013) Activities and word lists for teaching syllables, onset-rime, and

phoneme blending, segmenting and manipulation Website: readsters.com


Decodable Readers

Power Readers by Susan Ebbers Beginner Books for New Decoders Website: readingway.com/index.php/supercharged-readers.htm Supercharged Readers by Susan Ebbers Decodable chapter books Website: readingway.com/index.php/supercharged-readers.htm

Documents

Using the Right Data to Change Instruction and Improve ...attachments.aplus.agora.com/t13_live/PEFE34.handout.8BB54.pdfUsing the Right Data to Change Instruction and Improve Reading