Advanced DIBELS & IDEL: Scientifically Based Reading Research into Practice Roland H. Good III University of Oregon Reading First

Advanced DIBELS & IDEL: Scientifically Based Reading Research into Practice

Roland H. Good IIIUniversity of Oregonhttp://dibels.uoregon.edu

Reading First Sponsored Workshop

Portland, OR

January 13, 2005

Presentation:http://darkwing.uoregon.edu/~rhgood/adv_dibels-idel.pptHandout:http://darkwing.uoregon.edu/~rhgood/adv_dibels-idel.pdfTech Report:http://darkwing.uoregon.edu/~rhgood/curriculum_effectiveness_evaluation.pdf

1/13/05 Portland OR (c) 2005 2

Scientifically Based Reading Research

The term scientifically based reading research' means research that

(A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and

(B) includes research that —

(i) employs systematic, empirical methods that draw on observation or experiment;

(ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;

(iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and

(iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

1/13/05 Portland OR (c) 2005 3

Levels of Evidence

Lowest Level of Confidence Cardiac – I know in my

heart this is the way children learn.

Belief, philosophy, opinion, tradition.

Don’t bother me with evidence, I already know the answer.

Many, many different positions, often held with extreme and emotional conviction.

1/13/05 Portland OR (c) 2005 4

Levels of Evidence

Higher Level of Confidence A controlled, carefully

designed study is conducted to test the proposed component or procedure.

Must have an opportunity not to work.

Peer review enhances our confidence in the findings.

1/13/05 Portland OR (c) 2005 5

Levels of Evidence

Even Higher Confidence Replication - the

controlled, carefully designed study is repeated with different children, different researchers, different interventionists, different procedures.

Peer review enhances our confidence in the findings.

Fewer findings are replicated by many researchers.

1/13/05 Portland OR (c) 2005 6

Levels of Evidence

Even Higher Confidence Meta-Analysis of many

studies under varying conditions with varying threats to conclusions and a strong, robust effect is obtained under all or most conditions.

Again, peer review of the meta-analysis enhances our confidence in the findings.

Very few findings are examined and summarized by meta-analyses.

1/13/05 Portland OR (c) 2005 7

Levels of Evidence

Great Confidence A panel of experts spends 2

years summarizing 115,000 research studies on early literacy. Selects studies that meet high standards of rigor for design and experimental control. Conducts a meta-analysis of the findings. Distills important components of effective early literacy instruction.

5 Core Components meet this level of confidence.

1/13/05 Portland OR (c) 2005 8

Levels of Evidence – See for Yourself

Greatest Confidence You obtain powerful and

persuasive evidence on an ongoing basis regarding the effectiveness of instruction and outcomes for your children.

Your children, your implementation, your setting, your conditions, current information.

Immediate, vivid, personal. Lots of sites can replicate

under lots of conditions.

1/13/05 Portland OR (c) 2005 9

Beginning Reading Core Components#1. Phonemic Awareness – The understanding that individual sounds of

spoken language (phonemes) work together to make words. This allows readers to hear, identify, and manipulate the individual sounds.

#2. Phonics – The relationship between the sounds of spoken language (phonemes) and the letters representing those sounds in written language (graphemes). Skill in phonics helps students to recognize familiar words and decode unfamiliar ones.

#3. Fluency – The skill of reading texts accurately and quickly, which allows readers to recognize and comprehend words at the same time.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Bethesda, MD: National Institute of Child Health and Human Development. Available: http://www.nationalreadingpanel.org/.

1/13/05 Portland OR (c) 2005 10

Beginning Reading Core Components

#4. Vocabulary – The ability to store information about the meaning and pronunciation of words. There are four types of vocabulary: listening, speaking, reading, and writing.

#5. Reading Comprehension – Understanding, remembering, and communicating with others about what has been read. Comprehension strategies help readers to make sense of a text.

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Bethesda, MD: National Institute of Child Health and Human Development. Available: http://www.nationalreadingpanel.org/.

1/13/05 Portland OR (c) 2005 11

Model of Big Ideas, Indicators, and Timeline

Adapted from Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decision-making utility of a continuum of fluency-based indicators of foundational reading skills for third-grade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.

Big Ideas inBeginningReading

DynamicIndicators ofBasic EarlyLiteracy Skills

Benchmark GoalTimeline forAssessing BigIdeas K-3

Accuracy &Fluency with

Connected Text

ReadingComprehension

AlphabeticPrinciple

PhonologicalAwareness

ISF PSF NWF ORF ORF ORF ORF & RTF

Fall Winter Spring Fall Winter Spring Fall Winter Spring

Kindergarten First Grade Second Grade Third Grade

Fall Winter Spring

Vocabulary and Language Development

WUF WUF

1/13/05 Portland OR (c) 2005 12

Indicadores dinámicos del éxito en la lectura 6ta Edición (IDEL) Good, Bank, & Watson (2003)

A “reinvention” of the DIBELS Designed to be indicators of important early literacy

skills in Spanish Directions and Assessment in Spanish Seven measures:

1/13/05 Portland OR (c) 2005 13

The Measures

DIBELS Measure (English) IDEL Measure (Spanish)

Letter Naming Fluency (LNF) Fluidez en el nombramiento de las letras (FNL)

Initial Sound Fluency (ISF) Fluidez en los sonidos inciales (FSI)

Phonemic Segmentation Fluency (PSF)

Fluidez en la segmatación de fonemas (FSF)

Nonsense-Word Fluency (NWF) Fluidez en las palabras sin sentido (FPS)

DIBELS Oral Reading Fluency

(DORF)

Fluidez en la lectura oral IDEL (FLO)

DIBELS Oral Retell Fluidez en el recuerdo oral del cuento (ROC)

Word Use Fluency (WUF) Fluidez en el uso de las palabras (FUP)

1/13/05 Portland OR (c) 2005 14

Two Pathways to Literacy

English – L2

Phonemic Awareness in English

Alphabetic Principle in English

Accuracy and Fluency in English

Reading Comp. in English

Voc

abu

lary

& L

ang.

in E

ngl

ish

Spanish – L1

Phonemic Awareness in Spanish

Alphabetic Principle in Spanish

Accuracy and Fluency in Spanish

Reading Comp. in Spanish

Voc

abu

lary

& L

ang.

in S

pan

ish

Could Follow Both L1 and L2

1/13/05 Portland OR (c) 2005 15


Spanish – L1 English – L2





Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Could Follow L1 and Transition to L2

1/13/05 Portland OR (c) 2005 16







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Could Teach Skills in English (L2)

1/13/05 Portland OR (c) 2005 17







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Could Teach Skills in English (L2) with Support from Transition L1 Skills

1/13/05 Portland OR (c) 2005 18







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Could Teach Skills in English (L2) with Generalization to L1 Skills

1/13/05 Portland OR (c) 2005 19







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Use DIBELS & IDEL, with instructional goals for both DIBELS & IDEL

1/13/05 Portland OR (c) 2005 20







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Use IDEL until Transition to English, with Spanish instructional goals for IDEL

1/13/05 Portland OR (c) 2005 21







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Use DIBELS with English instructional goals for DIBELS

1/13/05 Portland OR (c) 2005 22







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Use DIBELS & IDEL, with English instructional goals for DIBELS

1/13/05 Portland OR (c) 2005 23







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Use DIBELS & IDEL, with English instructional goals for DIBELS

1/13/05 Portland OR (c) 2005 24







Voc

abu

lary

& L

ang.

in S

pan

ish





Voc

abu

lary

& L

ang.

in E

ngl

ish

Not Achieving Literacy in Either is Unacceptable

1/13/05 Portland OR (c) 2005 25

Instructional Goals for Core Components of Beginning Reading

Benchmark Goals to be On Grade Level:Middle K: Phonological Awareness with 25 - 35 on DIBELS

Initial Sound Fluency by mid kindergarten (and 18 on PSF)End K: Phonemic Awareness with 35 - 45 on DIBELS

Phoneme Segmentation Fluency by end of kindergarten (and 25 on NWF)

Middle 1st: Alphabetic principle 50 - 60 on DIBELS Nonsense Word Fluency by mid first grade with at least 15 words recoded (and 20 on DORF)

End 1st: Fluency with 40 - 50 on DIBELS Oral reading fluency by end of first grade (and RTF 25% or more).

End 2nd: Fluency with 90 + on DIBELS Oral reading fluency by end of second grade (and RTF 25% or more)

End 3rd: Fluency with 110 + on DIBELS Oral reading fluency by end of third grade (and RTF 25% or more)

1/13/05 Portland OR (c) 2005 26

http://DIBELS.uoregon.edu

1/13/05 Portland OR (c) 2005 27

1/13/05 Portland OR (c) 2005 28

DIBELS Oral Reading FluencyPlease read this (point)

out loud. If you get stuck, I will tell you the word so you can keep reading. When I say, “stop” I may ask you to tell me about what you read, so do your best reading. Start here (point to the first word of the passage). Begin.

The Robin’s Nest

There was a robin’s nest outside our kitchen window. The

nest was in a tall bush. The mother robin sat in the nest all day

long. One day when I was watching, the mother bird flew

away. I saw the eggs she was sitting on. There were four blue

eggs.

I watched and watched. The eggs moved. I watched some

more. The eggs started to crack. Finally, the eggs hatched. I

saw four baby birds. The baby birds opened their beaks wide.

I heard them peeping. Soon the mother bird came back. Then

the mother robin put worms in their mouths.

Every day I watched the baby birds and their mother.

Pretty soon the babies were so fat there was no room for the

mother. Then one morning the nest was gone from the bush.

DIBELS Oral Reading Fluency First Grade Benchmark 2 © 2001 Dynamic Measurement Group Revised: 03/28/02

1/13/05 Portland OR (c) 2005 29

DIBELS Retell Fluency

Please tell me all about what you just read. Try to tell me everything you can. Begin. Start your stopwatch after you say “begin”.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94

1/13/05 Portland OR (c) 2005 30

Fluidez en la Lectura Oral (FLO)

Por favor lee esto (señale) en voz alta. Si te atoras, te digo la palabra para que puedas seguir leyendo. Cuando digo “para” te puedo preguntar sobre lo que leíste, así que trata de leer lo mejor que puedas. Empieza aquí (señale la primera palabra del pasaje). Empieza.

Por favor cuéntame sobre todo lo que acabas de leer. Trata de contarme todo lo que puedas. Empieza.

Indicador 2 – Pasaje 1 Primer grado Fluidez en la lectura oral IDELTM

Las mañanas en nuestra casa Por la mañana, cuando todavía está oscuro, todos nos despertamos. Primero se despiertan mi abuelita y mi papá. Su radio empieza a tocar música. Luego mi papá sale al pasillo y llama a nuestra puerta. —Es hora de desayunar niños,— nos grita. Luego oigo correr el agua del baño mientras mi abuelita se ducha. Cuando termina, ella llama a nuestra puerta. Lo hace porque sabe que todavía estamos acostados. Mi papá se mete a la regadera. Canta unas melodías del pasado. Me tapo la cabeza con la almohada. Por fin abro los ojos cuando oigo el secador de pelo. Después mi papá baja corriendo las escaleras hasta la cocina. El prepara el desayuno. Sube el volumen del radio aún más. Entonces, mi abuelita llama a la puerta del baño y dice: —¡Tomás, deja un poco de agua caliente para tu hermano! Mi hermano Mario es el último en levantarse. El es perezoso. Ahora estamos todos juntos en la cocina. Mi papá desayuna pan tostado con miel. Le pregunta a mi abuelita si está lista para salir. Nos mira a mi hermano y a mí y nos dice: —No se olviden de hacer la tarea al llegar a casa. Vamos a ir al parque esta tarde. Mi abuelita y mi papá salen juntos. Miro el reloj. En seis minutos yo tengo que estar en la parada del autobús. Meto de prisa mi almuerzo en la mochila. Después, corro por la puerta justo a tiempo. En ese momento, veo llegar el autobús escolar. Por suerte, no llegué tarde.

9

19

31

41

42

53

63

75

86

98

99

109

120

131

141

152

162

175

176

187

201

205

217

227

238

250

256

Palabras recontadas: Total:_________

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 © 2003 Dynamic Measurement Group, Inc. Página 13

1/13/05 Portland OR (c) 2005 31

In one minute, we can obtain a reliable indicator of early reading proficiency. The two students require substantially different instruction toward the goal of being lifelong readers.

I’ve thrown a lot of rocks into the lake by our cabin. Sometimes I think I’ve thrown in enough to fill the whole lake. But it never seems to get full. As you can tell, I like to throw rocks. But throwing rocks is always a lot more fun with Grandpa. He can make anything….

Low risk reader At risk readerDORF 40 to 45 DORF 5 to 10

1/13/05 Portland OR (c) 2005 32

DIBELS Nonsense Word Fluency

Here are some more make-believe words (point to the student

probe). Start here (point to the first

word) and go across the page (point across the page). When I say, “begin”, read the words the best you can. Point to each letter and tell me the sound or read the whole word. Read the words the best you can. Put your finger on the first word. Ready, begin.

kik woj sig faj yis kaj fek av zin zez lan nul zem og nom yuf pos vok viv feg bub dij sij vus tos wuv nij pik nok mot nif vec al boj nen suv yig dit tum joj yaj zof um vim vel tig mak sog wot sav

1/13/05 Portland OR (c) 2005 33

Fluidez en las palabras sin sentido Mira esta palabra (señale la primera palabra en la copia de práctica). No es una palabra verdadera. Es una palabra sin sentido. Observa cómo leo la palabra: /m/ /o/ /s/ /i/ “mosi” (señale cada letra, despúes pase el dedo rápidamente debajo de toda la palabra) Puedo decir los sonidos de las letras, /m/ /o/ /s/ /i/ (señale cada letra), o puedo leer la palabra completa “mosi” (pase el dedo rápidamente debajo de toda la palabra).Ahora te toca a ti leer una palabra sin sentido. Lee la palabra lo mejor que puedas (señale la palabra “lu”). Asegúrate de decir todos los sonidos que sepas.

Indicador 2 Kindergarten Fluidez en las palabras sin sentido IDELTM

b o n a j a g a l a l i p o t a __/16

p i d e t i f o p u p a n o t i __/16

j u m a p o s o d i n e n u m u __/16

j e n e b i c o r i v i __/12

d a v e m e m u f u d e m a v a r i c u __/20

l o s u v e t a z o d e b e n e d o __/18

l a l e v o b i r e l e j o t e v o __/18

n i ñ i f o l a c o s e p o l i __/16

p a p e t e t e t u t o r o d i ñ i __/18

z a l o n i b e j a j o t u l a __/16

Total: ____

_ Tipos de errores: Página 18 © 2003 Dynamic Measurement Group, Inc.

1/13/05 Portland OR (c) 2005 34

Role of Mid First Alphabetic Principle

Odds of being on track with ORF in end of first grade when Established with NWF in middle of first grade is 117 out of 136, or 86%

Odds of being on track with ORF in end of First Grade when Deficit with NWF in middle of first grade is 5 out of 46, or 11%

1/13/05 Portland OR (c) 2005 35

Similar Odds, Different Outcome

Odds of being on track with ORF in end of first grade when Established with NWF in middle of first grade are 12 out of 18, or 67%

Odds of being on track with ORF in end of first grade when Deficit with NWF in middle of first grade are 3 out of 21, or 14%

1/13/05 Portland OR (c) 2005 36




1/13/05 Portland OR (c) 2005 37




1/13/05 Portland OR (c) 2005 38

How do I support my children to learn the alphabetic principle so well they reach NWF of 50?

Foundation of Phonemic Awareness Systematic and Explicit Instruction Start Early and Move in the Direction Practice Assess to inform decisions that change outcomes Monitor Progress Do Something About Lack of Adequate Progress

1/13/05 Portland OR (c) 2005 39

Foundation in Phonemic Awareness with Systematic and Explicit Phonics Instruction

AlphabeticPrinciple

PhonemicAwareness

Skill

Instruction onLetter Sounds

Instruction ona Recoding

Strategy

Instruction onFading the

Scaffold Fluency with Recoding

Instruction

Instruction ona Scaffolded

Recoding

1/13/05 Portland OR (c) 2005 40




AlphabeticPrinciple

NWF



Fall Winter Spring

Alphabetic Principle, Indicator, Goal, and TimelineEarlier Intervention and Prevention are Best


13 25 24

Moving in the Direction of the goal. Focus of early intervention and prevention efforts.

Alphabetic Principle Goal. To have a powerful strategy to encounter an unknown word and confidently obtain a reasonable pronunciation of the word. Phonics and alphabetic principle skills should be so well developed they score at least 50 on DIBELS NWF with at least 15 words recoded.

1/13/05 Portland OR (c) 2005 41

Practice? Should I use DIBELS NWF to practice decoding nonsense words?

Absolutely not. Under no conditions should DIBELS assessment

materials be used for instruction or practice. Reason 1: Children should always be tested cold on

the skills. If they aren’t tested cold we don’t know what their scores mean. We don’t know if they are on track or not.

Reason 2: More important, the DIBELS NWF score is not the point. The alphabetic principle is the point. Our instruction should always focus on the big idea or core component: phonics and the alphabetic principle.

1/13/05 Portland OR (c) 2005 42

Practice what? Should I never have my children practice reading nonsense words? I think practicing decoding and reading words is great:

real words and nonsense words both. Keep in mind the big idea goal:

To have a powerful strategy to encounter an unknown word and confidently obtain a reasonable pronunciation of the word.

Practice should occur in the context of meaningful and important instruction on the alphabetic principle.

Don’t forget recoding: using letter sound knowledge to recover the pronunciation of the whole word.

For example, The Alien Word Game (Source unknown)

1/13/05 Portland OR (c) 2005 43

The Alien Word Game (Source unknown) Start with a set of magnetic or felt letters, a mixture of consonants

and vowels, that the students have been learning and practicing. For example,

a o i m t l p s r n have the students review the sounds of all the letters, group and

individual turns, signal for group response so low kids respond at the same time as the group. Make sure low kids are accurate with the letter sounds.

make a word “tap” and practice reading the word: first sound by sound then say it fast – what word? tap Is it an alien word? No

next switch out one of the letters – trade the p for an n. read the word: sound by sound, say it fast, what word? tan Is it an alien word? No

next switch out another letter – trade t for l. Read the word: sound by sound, say it fast, what word? lan Is it an alien word? Yes, it is an alien word. It is not a real word, it is a make believe word. It might be a new word that someone makes up some day.

- as students develop skill in reading a variety of words with these letters, real and alien, you can fade the sound by sound part so they are reading words and judging what the word is. (i.e., they are recoding the words fluently and automatically)

1/13/05 Portland OR (c) 2005 44

Reading First:Four Kinds/Purposes of Reading Assessment

Outcome - Assessments that provide a bottom-line evaluation of the effectiveness of the reading program.

Screening - Assessments that are administered to determine which children are at risk for reading difficulty and who will need additional intervention.

Diagnosis - Assessments that help teachers plan instruction by providing in-depth information about students’ skills and instructional needs.

Progress Monitoring - Assessments that determine if students are making adequate progress or need more intervention to achieve grade level reading outcomes.

An effective, comprehensive, reading program includes reading assessments to accomplish four purposes:

Source: Reading First Initiative: Secretary’s Leadership Academy

1/13/05 Portland OR (c) 2005 45

Using an Outcomes Driven Model to Provide Decision Rules for Progress Monitoring

Outcomes Driven model: Decision making steps

1. Identifying Need for Support

2. Validating Need for Instructional Support

3. Planning and Implementing Instructional Support

4. Evaluating and Modifying Instructional Support

5. Reviewing Outcomes for Individuals and Systems

Good, R. H., Gruba, J., & Kaminski, R. A. (2002). Best Practices in Using Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in an Outcomes-Driven Model. In A. Thomas & J. Grimes (Eds.), Best Practices in School Psychology IV (pp. 679-700). Washington, DC: National Association of School Psychologists.

1/13/05 Portland OR (c) 2005 46

1. Identifying Need for Support

Key Decision for Screening Assessment: Which children may need additional instructional support to attain

important reading outcomes?

Data used to inform the decision: Compare individual student’s performance to local normative

context or expected performance to evaluate need for additional instructional support. Local normative context: First, choose a percentile cutoff.

20th percentile seems a good place to start, but a district could choose 15th percentile or 25th percentile or other cutoff depending on resources.

Expected performance: A deficit in a foundation skill is a strong indicator that instructional support will be needed to attain later benchmark goals.

1/13/05 Portland OR (c) 2005 47

Beginning of First Grade

1/13/05 Portland OR (c) 2005 48

Considering Initial Skills, Does Slope Add to Predictions of Outcomes? Students with complete data from 2002-2003 in the

DIBELS Data System were examined for level of risk, slope of progress, and reading outcomes.

Beginning NWF NWF Slope

Group N Mean Std Dev N Mean Std Dev

At Risk 20739 5.46 4.23 20739 1.54 1.02

Some Risk 20606 18.08 3.13 20606 1.47 0.97

Low Risk 38082 34.62 7.09 38082 1.23 1.16

Hi AP 12288 70.32 22.55 12288 1.24 1.73

Total 91715 29.09 22.12 91715 1.36 1.19

NWF 0 to 12

NWF 13 to 23

NWF 24 to 49

NWF 50 to 255

Beg

inni

ng F

irst

Gra

de N

WF

1/13/05 Portland OR (c) 2005 49

Utility of Initial NWF Risk Categories Initial skills on NWF are a very strong predictor of

reading outcomes.

Ending ORF

Group N Mean Std Dev Odds of Achieving Benchmark Goal

At Risk 20739 26.52 21.13 22%

Some Risk 20606 42.81 24.47 47%

Low Risk 38082 62.07 28.74 76%

Hi AP 12288 102.19 34.44 97%

Total 91715 55.08 35.68 60%

NWF 0 to 12

NWF 13 to 23

NWF 24 to 49

NWF 50 to 255

Beg

inni

ng F

irst

Gra

de N

WF

1/13/05 Portland OR (c) 2005 50

Decision Utility of DIBELS Fall of 1st LNF >= 37, DIBELS PSF >= 35, DIBELS NWF >= 24

Instructional Recommendation: Benchmark - At grade level. Effective core curriculum and instruction recommended, Odds of reading 40 or more words correct per

minute at the end of first grade: 84% LNF < 25, DIBELS PSF < 10, DIBELS NWF < 13

Instructional Rec: Intensive - Needs substantial intervention: Odds of reading 40 or more words correct per

minute at the end of first grade: 18% (unless given intensive intervention)

Value of knowing the instructional recommendation and the goal early enough to change the outcome: Priceless.

1/13/05 Portland OR (c) 2005 51

2. Validate Need for SupportKey Decision: Are we reasonably confident the student needs instructional

support? Rule out easy reasons for poor performance:

Bad day, confused on directions or task, ill, shy, or similar. More reliable information is needed to validate need for

support than for screening decisions. Data used to inform the decision: Repeated assessments on different days under different

conditions Compare individual student’s performance to local normative

context or expected performance to evaluate discrepancy.

1/13/05 Portland OR (c) 2005 52

Sept.

10

20

30

40

Oct. Nov. Dec. Jan. Feb.

60

70

80

50

Validating Need for Support Verify need for instructional support by retesting with

alternate forms until we are reasonably confident.

No

nse

nse

Wo

rd F

luen

cy

Beginning 1st cutoff at risk

Beginning 1st cutoff low risk

1/13/05 Portland OR (c) 2005 53

3. Planning and Implementing Instructional SupportKey Decisions for Diagnostic Assessment: What are the Goals of instruction?

Where are we? Where do we need to be? By when? What course do we need to follow to get there?

What skills should we teach to get there? Focus on the beginning reading core areas: Phonological

Awareness, Alphabetic Principle, Accuracy and Fluency with Connected Text

Specific skills based on error analysis or additional diagnostic assessment (e.g., CTOPP).

How much instructional support is needed? Intensive Instructional Support Strategic Instructional Support Benchmark Instruction

1/13/05 Portland OR (c) 2005 54

Dec.

10

20

30

40

Jan. Feb. Mar. Apr. May.

60

70

80

50

Kindergarten Instructional Goals Establish an Instructional Goal for Alphabetic

Principle that is moving in the direction of achieving the middle of first grade goal.

No

nse

nse

Wo

rd F

luen

cy

Mid-kinder cutoff at risk

Mid-first cutoff low risk

1/13/05 Portland OR (c) 2005 55

Sept.

10

20

30

40


60

70

80

50

First Grade Instructional Goals Establish an Instructional Goal for Alphabetic

Principle that will change odds of being a reader

No

nse

nse

Wo

rd F

luen

cy

Mid-first cutoff at risk

Mid-first cutoff low risk

1/13/05 Portland OR (c) 2005 56

1/13/05 Portland OR (c) 2005 57

Oregon Reading First Review of Supplemental and Intervention Programs OR Reading First developed review criteria for

supplemental and intervention programs and reviewed 106 programs for the percent of criteria met.http://oregonreadingfirst.uoregon.edu/SIreport.php Phonemic Awareness

Early Reading Intervention 96%Road to the Code 80%Phonemic Awareness in Young Children 75%

Phonics or Alphabetic PrincipleReading Master Fast Cycle 96%Read Well 94%Voyager Passport 92%Early Reading Intervention 81%Funnix ??%

Fluency with Connected TextRead Naturally 92%Great Leaps 66%Headsprout 61%

1/13/05 Portland OR (c) 2005 58

Purpose of Diagnostic Assessment: Adequate Progress

Provide increased confidence of need for educational support.

Target essential component for intervention focus. Deficit on PA Intervention targeting PA Established PA, Deficit on AP Intervention

targeting AP Established PA and AP, Deficit on fluency with

connected text Intervention targeting reading connected text and fluency building.

Identify level of support and intensity of intervention e.g., strategic or intensive

Identify specific skill deficits or other instructionally relevant characteristics (e.g., program placement, behavior needs, RAN, language skills, background knowledge) to directly inform instruction.

1/13/05 Portland OR (c) 2005 59

“Because they are expensive and time-consuming to administer, diagnostic tests should not be given routinely to every struggling reader in a class or grade.” (Torgesen, 2004)

Use screening, progress monitoring, and outcome assessments, and specific program placement tests to obtain initial information to guide instruction.

Time-consuming diagnostic assessment should only be used when we do not have sufficient information to support a student to make adequate progress.

When we are providing appropriate instruction or intervention and a child is making adequate progress, additional diagnostic information is not indicated.

Role of Expensive, Time-Consuming, Diagnostic Assessment

1/13/05 Portland OR (c) 2005 60

1. Using screening, prior outcome assessment, and progress monitoring assessment to target essential component for intervention.

2. Implement research based intervention targeting the essential component (e.g., ERI for phonemic awareness)

3. Evaluate the adequacy of the intervention using progress monitoring assessment. If adequate progress maintain If lack of adequate progress increase intensity

A. Examine and increase integrity of implementationB. Examine and increase intervention timeC. Examine and decrease group size

If adequate progress maintain4. Only when a child has a serious, sustained, lack of adequate

progress with intensive intervention and increased intensity would additional, targeted, diagnostic assessment be indicated.

Determining need for Expensive, Time-Consuming, Diagnostic Assessment

1/13/05 Portland OR (c) 2005 61

4. Evaluating and Modifying Instructional Support

Key Decision for Progress Monitoring Assessment: Is the intervention effective in improving the child’s early

literacy skills?

How much instructional support is needed? Enough to get the child on trajectory for Benchmark

Goal.

When is increased support needed? Monitor child’s progress during intervention by graphing

and comparing their performance and progress to past performance and their aimline. Three consecutive assessments below the aimline indicates a need to increase instructional support.

1/13/05 Portland OR (c) 2005 62

Sept.

10

20

30

40


60

70

80

50

Evaluating Support: Modify Instruction?

Whoops! Time to make a change!

No

nse

nse

Wo

rd F

luen

cy

Aimline

InterventionChange

1/13/05 Portland OR (c) 2005 63

Sept.

10

20

30

40


60

70

80

50

Evaluating Support:Is Instructional Support Sufficient Now?

No

nse

nse

Wo

rd F

luen

cy

Aimline

InterventionChange

1/13/05 Portland OR (c) 2005 64

GPS On Where are we?

You are Here

Port

Where is the port?What course should I follow?How am I doing?

ActualCourse

DesiredCourse

1/13/05 Portland OR (c) 2005 65

Sept.

10

20

30

40


60

70

80

50

Evaluating Support:Is Instructional Support Sufficient Now?

No

nse

nse

Wo

rd F

luen

cy

Aimline

InterventionChange

1/13/05 Portland OR (c) 2005 66

Progress Monitoring Repeated, formative assessment to evaluate progress

toward important goals for the purpose of modifying instruction or intervention.

Frequency of Progress Monitoring 3 times per year for students at low risk (All Students)

Benchmark 1 per month for students with some risk

Strategic 1 per week for students at risk

Intensive

1/13/05 Portland OR (c) 2005 67

Research on Progress Monitoring Progress monitoring has been extensively researched

in Special Education For example:

Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199-208.

With Reading First, progress monitoring is not just for special education any more.

1/13/05 Portland OR (c) 2005 68

Effects of Progress Monitoring Fuchs and Fuchs (1986) found the average effect size

associated with progress monitoring was: +0.70 for monitoring progress +0.80 when graphing of progress was added +0.90 when decision rules were added

A student at the 50th percentile would be expected to move to the 82nd percentile (i.e., a score of 100 would move to a score of 114)

Perhaps more important, a student at the 6th percentile would be expected to move to the average range (25th percentile)(i.e., a score of 76 would move to a score of 90)

1/13/05 Portland OR (c) 2005 69

Progress Monitoring Tools Meaningful and important goals, waypoints, or

benchmarks representing reading health or wellness. Meaningful and Important Public and Measurable Ambitious

Brief, repeatable, formative assessment of progress toward benchmark goals that is sensitive to intervention. Brief and Efficient Repeatable - weekly or monthly Reliable and Valid indication of risk and growth

1/13/05 Portland OR (c) 2005 70

Is Progress is Related to Outcomes? The logic of the Evaluating and Modifying Support step

relies on evidence that amount of progress toward goals is related to important reading outcomes.

Given or controlling for initial skills, is slope of progress on NWF in the Fall of first grade related to first grade reading outcomes for at risk students? Evaluations of the relation between slope of

progress and outcomes must consider the student’s initial skills.

1/13/05 Portland OR (c) 2005 71

Progress GIVEN initial skills. Nora has a slope twice that of Nick, but substantially

lower reading outcome because her initial skills are so much lower.

0

10

20

30

40

50

60

70

80

0 4 8 12 16 20 24 28 32 36

Week

NW

F C

orr

ect

Let

ter

So

un

ds

Nick: slope = +0.70

Nora: slope = +1.50

Nick PredictedSpring DORF = 70

Nora PredictedSpring DORF = 26

Slope, by itself without considering initial skills is not enough to predict outcomes.

1/13/05 Portland OR (c) 2005 72

Similar Initial Skills – Slope is related to outcomes Nora and Nell have similar initial skills – Nell’s

higher slope predicts higher skills in middle of first grade and higher reading outcomes.

0

10

20

30

40

50

60

70

80

0 4 8 12 16 20 24 28 32 36

Week

NW

F C

orr

ect

Let

ter

So

un

ds

Nick: slope = +0.70

Nell: slope = +2.56

Nora: slope = +1.50

Nell PredictedSpring DORF = 36

Nora PredictedSpring DORF = 26

1/13/05 Portland OR (c) 2005 73

Does Slope Add to the Prediction of Reading Outcomes After Risk Level and Initial Skills? [All Students] Sequential model predicting first grade DORF reading

outcomes from (1) risk category, (2) initial NWF skill given risk, and (3) slope given risk and initial skill.

Source DF R2 change

NWF Risk Category 3 0.40

Initial NWF Skill Given Risk 1 0.08

Slope Given Risk, Initial Skill 1 0.11

Total 91714 1.00

Risk category, initial skills, and slope combined explain 59% of reading outcomes.

1/13/05 Portland OR (c) 2005 74

Variance Explained by Slope for Each Risk Category A separate analysis was conducted for each risk

category.

Percent of Risk Category Variance in

Reading Outcomes Explained

Group NWF Initial Skills NWF Slope Given

Initial Skills

At Risk 8% 26%

Some Risk 2% 21%

Low Risk 8% 21%

Hi AP

25% 11%

Rate of progress in alphabetic principle is especially important for students who are at risk for low reading outcomes.

But, is the variance explained by slope (given risk and initial skills) educationally important?

NWF 0 to 12

NWF 13 to 23

NWF 24 to 49

NWF 50 to 255

1/13/05 Portland OR (c) 2005 75

Sept.

10

20

30

40


60

70

80

50

Variability in Slope for At Risk Students About 68% of At Risk student’s trajectories are

between the low slope and the high slope.

Hi Slope

Lo Slope

No

nse

nse

Wo

rd F

luen

cy

Mid-year cutoff low risk

1/13/05 Portland OR (c) 2005 76

Slope of Progress for At Risk Students Goal Slope. The slope of progress necessary to achieve

the Alphabetic Principle Benchmark Goal of 50 on NWF in the middle of first grade and predict successful first grade reading outcomes. Adequate Progress.

High Slope. One standard deviation above the mean – the 86th percentile compared to other children at risk for reading difficulty. Indicates a very effective intervention is in place. Adequate Progress.

Typical Slope. Average or typical progress. Most students continue to be at risk. Not making adequate progress.

Low Slope. One standard deviation below the mean – the 14th percentile compared to other children at risk for reading difficulty. Indicates the lack of an effective intervention. Most at risk students continue to be at risk. Not making adequate progress.

1/13/05 Portland OR (c) 2005 77

Differences in Slope are Educationally Meaningful for At Risk Students Predicted reading outcomes are substantially

different. Goal slope predicts 40 end of first DORF.

0

5

10

15

20

25

30

35

40

45

50

0.52 0.77 1.03 1.28 1.54 1.79 2.05 2.30 2.56

MSlope - 1sd to MSlope + 1sd

Pre

dic

ted

En

d F

irst

DO

RF

Effective Intervention

Lack of Effective Intervention

Lo Slop

e

End of first reading difficulty

End of first reading goal

Hi Slope

1/13/05 Portland OR (c) 2005 78

Conclusions: Validity of DIBELS NWF Slope Initial risk status and initial skills on DIBELS

Nonsense Word Fluency are very important in predicting reading outcomes in first grade, explaining 48% of variance in outcomes.

An increasing pattern of scores through the first semester of first grade on DIBELS Nonsense Word Fluency appears to be a very important predictor of reading outcomes for students who are at risk, and indeed for each risk category.

We can be confident that increases in DIBELS Nonsense Word Fluency reflect improved performance on alphabetic principle skills that contribute to important end-of-year reading outcomes.

1/13/05 Portland OR (c) 2005 79

5. Reviewing OutcomesKey Decisions for Outcome/Accountability Assessment: Does the child have the early literacy skills predictive of

successful reading outcomes? Does the school have a schoolwide system of instruction and

support so their students achieve literacy outcomes?Data used to inform the decision: Evaluate individual student’s performance with respect to

benchmark goals that with the odds in favor of achieving subsequent literacy goals.

Compare school/district outcomes to goals and outcomes from previous year and to other schools in the district or region.

Evaluate the schoolwide system (core curriculum and instruction, supplemental support, and intervention) for each step to identify strengths and targets of opportunity for improvement.

1/13/05 Portland OR (c) 2005 80

Schoolwide System of Instruction and Support

Core Curriculum and Instruction

SupplementalSupport

Intervention

1/13/05 Portland OR (c) 2005 81

Schoolwide System of Instruction and Support -- Core Curriculum and Instruction --

Not just the reading curriculum selected but also the way it is delivered.

aka Primary Prevention or Benchmark Instruction Primary Goal: Meet the needs of 80% of students in

the school. If the schools has lots of children who need strategic or intensive support, the core curriculum and instruction will need to include many feature of strategic support and intensive intervention

Primary Step-Goal: Support all benchmark students to make adequate progress and achieve the benchmark goal.

Secondary Step-Goal: Support 50% of strategic students to achieve the benchmark goal.

1/13/05 Portland OR (c) 2005 82

Schoolwide System of Instruction and Support -- Supplemental Support --

Additional time, smaller group, more intensive, supplemental or intervention program, delivered with fidelity.

aka Secondary Prevention or Strategic Support Primary Goal: Meet the needs of 15% of students in

the school who will need more support than the core curriculum and instruction can provide.

Primary Step-Goal: Adequate progress to reduce risk of reading difficulty. Support strategic students to achieve the benchmark goal.

1/13/05 Portland OR (c) 2005 83

Schoolwide System of Instruction and Support -- Intervention --

Additional time, smaller group, more intensive, supplemental or intervention program, delivered with fidelity.

aka Tertiary Prevention or Intensive Intervention Primary Goal: Meet the needs of the 5% of students in

the school who will need very intensive intervention to achieve literacy goals.

Primary Step-Goal: Accelerate learning and progress to support intensive students to achieve the benchmark goal or reduce their risk of reading difficulty to strategic. If one step can get them to strategic, the next step can get them to benchmark.

1/13/05 Portland OR (c) 2005 84

Steps to Achieving Reading Outcomes





Accuracy &Fluency with

Connected Text

ReadingComprehension

AlphabeticPrinciple

PhonologicalAwareness

ISF PSF NWF ORF ORF ORF ORF & RTF



Fall Winter Spring

Vocabulary and Language Development

WUF WUF

Step 1Step 2 Step 3Step 4 Step 5Step 6 Step 7Step 8Step 3

1/13/05 Portland OR (c) 2005 85

Instructional Goals for Essential Components of Beginning Reading

Benchmark Goals to be On Grade Level:Middle K: Phonological Awareness with 25 - 35 on DIBELS

Initial Sound Fluency by mid kindergarten (and 18 on PSF)End K: Phonemic Awareness with 35 - 45 on DIBELS

Phoneme Segmentation Fluency by end of kindergarten (and 25 on NWF)

Middle 1st: Alphabetic principle 50 - 60 on DIBELS Nonsense Word Fluency by mid first grade with at least 15 words recoded (and 20 on DORF)

End 1st: Fluency with 40 - 50 on DIBELS Oral reading fluency by end of first grade (and RTF 25% or more).

End 2nd: Fluency with 90 + on DIBELS Oral reading fluency by end of second grade (and RTF 25% or more)

End 3rd: Fluency with 110 + on DIBELS Oral reading fluency by end of third grade (and RTF 25% or more)

1/13/05 Portland OR (c) 2005 86

Reviewing Outcomes: Effectiveness of Benchmark Instruction (Core Curriculum) For each step toward literacy outcomes, a school with

an effective core curriculum and instruction supports students who are on track (i.e., low risk or benchmark) to achieve each literacy goal.

For students with the odds in favor of achieving literacy goals, it is the job of the core to teach the essential components so that all students (100%) achieve the goals.

1/13/05 Portland OR (c) 2005 87

Evaluating EffectivenessI. Outcomes Criterion – Bottom line

95% of students achieve the early literacy goal.

II. Adequate Progress Criteria – are all students making adequate progress? Core Curriculum and Instruction: Benchmark

students make adequate progress and achieve goals Supplemental Support: Strategic students make

adequate progress and achieve goals Intensive Intervention: Intensive students make

adequate progress and achieve goals or at least reduce risk.

1/13/05 Portland OR (c) 2005 88

I. Outcomes Criterion Strength – The schoolwide instructional system is a

strength, including Core Curriculum and Instruction, Supplemental Support, and Intensive Intervention. Absolute Standard: 95% or more of students

schoolwide achieve the next literacy goal. If outcomes criterion is not met, evaluate the

effectiveness of each layer of the system using the Adequate Progress Criteria, including: Core Curriculum and Instruction, Supplemental Support, and Intensive Interventions.

1/13/05 Portland OR (c) 2005 89

II. Adequate Progress Criterion Benchmark Students are making adequate progress if they

achieve the next early literacy goal. Effective core curriculum and instruction should support

95% of benchmark students to achieve each literacy goal. Strategic Students are making adequate progress if they

achieve the next literacy goal. Effective supplemental support should get 80% of strategic

students to achieve each literacy goal. Intensive Students are making adequate progress if the

achieve the literacy goal or progress to lower risk. Effective interventions should help 80% of intensive

students to achieve the goal or achieve emerging or some risk status.

1/13/05 Portland OR (c) 2005 90

Absolute Standard and Relative Standard Absolute Standard – held constant from year to year,

represents an ambitious goal that all schools could attain. Strength: Adequate progress for 95% of Benchmark Adequate progress for 80% of Strategic Adequate progress for 80% of Intensive

Relative Standard – Based on most recently available schoolwide norms. Represents the current state of curriculum, supplemental support, intervention. Strength: Upper third compared to other schools Needs Support: Middle third compared to other schools Needs Substantial Support: Lower third compared to

other schools

1/13/05 Portland OR (c) 2005 91

II. Adequate Progress – Benchmark Students Core Curriculum and Instruction

Strength – Research-based effective reading core curriculum and delivery of that curriculum. Logic: The core curriculum and instruction should

support benchmark students to achieve literacy goals. Absolute Standard: 95% of benchmark students

achieve the next literacy goal. Relative Standard: Upper third of effectiveness of core

curriculum and instruction compared to other schools. Meet either the absolute standard or the relative

standard and the effectiveness of the core is a strength for the school.

1/13/05 Portland OR (c) 2005 92

II. Adequate Progress – Needs SupportCore Curriculum and Instruction

Needs Support – School (a) does not meet the Outcome Criterion, (b) does not meet the absolute standard for adequate progress and (c) the school is in the middle third of effectiveness compared to other schools. The school needs support in terms of professional

development, curriculum materials, integrity of delivery, or time investment to increase the effectiveness of the core.

1/13/05 Portland OR (c) 2005 93

II. Adequate Progress –Substantial SupportCore Curriculum and Instruction

Needs Substantial Support – School (a) does not meet the Outcome Criterion, (b) does not meet the absolute standard for adequate progress and (c) the school is in the lower third of effectiveness compared to other schools. Schoolwide priority for professional development,

curriculum materials, integrity of instruction, and time investment.

1/13/05 Portland OR (c) 2005 94

Focus on Support

What can we do systemically to support the effectiveness of the Schoolwide System of Instruction?

What would it take to help the school achieve literacy goals? Professional development on essential components

of early literacy? More powerful interventions or supplemental

materials? Coaching to improve fidelity of implementation? Additional resources to meet the needs of

challenging students (e.g., ELL, high mobility)? Administrative support to invest substantial time

and resources to change outcomes?

1/13/05 Portland OR (c) 2005 95

Evaluating Effectiveness Example Test District is a real school district that has been blinded –

all school names, district names, class names, and student names are fictitious.

Focus first on schoolwide evaluation of the core curriculum and instruction. Powerful and effective core enhances outcomes for all

students: Benchmark, Strategic, Intensive. Focus step by step. A school can have effective core

curriculum and instruction for one step but not another. First Semester of First Grade appears nationally to be

a target of opportunity to change reading outcomes.

1/13/05 Portland OR (c) 2005 96

Evaluating Effectiveness Worksheet First, clarify the primary instructional goal for the first

semester of first grade. Essential Component:

Phonics or Alphabetic Principle DIBELS Indicator:

Nonsense Word Fluency (NWF) Goal Skill Level:

50 letter sounds correct per minute with recoding Timeline:

by the middle of first grade.

1/13/05 Portland OR (c) 2005 97

First, Examine Schoolwide OutcomesI. Outcomes Criterion:

Schoolwide system of instruction and support in the first semester of first grade is a strength if 95% of students are Established on DIBELS NWF in the middle of first grade. Core curriculum and instruction is effective System of additional interventions is effective

For Example, schools on next slides have McKinley Washington

Established: 42% 82%Emerging: 43% 12%Deficit: 15% 7%

Neither school meets the Outcomes Criterion for Strength in their schoolwide system of instruction and support for the first semester of first grade.

Next step: Are students making adequate progress in the first semester of first grade?

1/13/05 Portland OR (c) 2005 98

Schoolwide distribution of scores (Histogram) for all students

1/13/05 Portland OR (c) 2005 99

Examining Outcomes for Alphabetic Principle Instruction--McKinley School--

15% Deficit

43% Emerging

42% Established

Middle 1st NWF Histogram Report

1/13/05 Portland OR (c) 2005 100

Examining Outcomes for Alphabetic Principle Instruction--Washington School--

7% Deficit

12% Emerging

82% Established

Middle 1st NWF Histogram Report

1/13/05 Portland OR (c) 2005 101

Examine progress of Benchmark Students – Are benchmark students reaching goal?

Effective core curriculum and instruction should support benchmark students to make adequate progress and achieve essential early literacy goals.

Use Effectiveness Report Focus on schoolwide summary Classroom report illustrates individual classrooms and

children For example,

Washington School on the next slides has 95% of Benchmark students reaching the middle of first grade goal.

McKinley School on the next slides has 67% of Benchmark students reaching the middle of first grade goal.

1/13/05 Portland OR (c) 2005 102

Summary of Effectiveness by School or District

1/13/05 Portland OR (c) 2005 103

1/13/05 Portland OR (c) 2005 104

1/13/05 Portland OR (c) 2005 105

1/13/05 Portland OR (c) 2005 106

Evaluating the Effectiveness of Core Curriculum and Instruction: 2001 – 2002 Schoolwide Norms McKinley School is not meeting the absolute standard for

effective core curriculum and instruction. How are their outcomes compared to other schools? 2001 – 2002 norms, Table 3, pages 8 & 9, 67% of

benchmark students achieving the middle of first grade goal is in the middle third compared to other schools.

Washington School is meeting the absolute standard for effective core curriculum and instruction. 95% of benchmark students achieve the middle of first grade goal.

Updated norms for 2003-2004 are in progress and should be available shortly.

1/13/05 Portland OR (c) 2005 107

Beginning First to Middle FirstMiddle of first grade outcomes for students with benchmark, strategic, and intensive instructional recommendations in the beginning of first grade

0

50

100

150

200

250

300

Conditional Percent Reaching NWF Goal

Nu

mb

er o

f S

cho

ols

Intensive

Strategic

Benchmark

Beginning First Instructional Recommendation

A typical (or middle) school had 59% to 75% of benchmark students achieve the middle of first grade goal of 50 or more with recoding on DIBELS NWF.

Schools with 58% or fewer of their benchmark students achieving the middle first grade NWF goal are in the lower third of effectiveness.

Schools with 76% or more of their benchmark students achieving the middle first grade NWF goal are in the upper third of effectiveness.

1/13/05 Portland OR (c) 2005 108

Compare to Decision Rules and Other Schools to evaluate effectiveness Effective core curriculum and instruction supports

95% of benchmark students to achieve the goal. Washington: Met - Strength McKinley: Not met.

Compared to other schools, McKinley School is in the Upper Third - Strength Middle Third - Support Lower Third – Substantial Support

1/13/05 Portland OR (c) 2005 109

Instructional Step Grade, Semester

Core Curriculum

and Instruction

Supplemental Support

Intensive Intervention

Kinder, 1st Sem: Phonemic Awareness

Kinder, 2nd Sem: Phonemic Awareness and Phonics

First, 1st Sem: Phonics and Fluency

First, 2nd Sem: Fluency and Comprehension

Second, 1st Sem: Fluency and Comp.

Second, 2nd Sem: Fluency and Comp.

Third, 1st Sem: Fluency and Comp.

Third, 2nd Sem: Fluency and Comp.

McKinley Elementary School

Support

1/13/05 Portland OR (c) 2005 110

Use Models of Effective Core Curriculum and Instruction Seek models of success in the district, state, or region. Within the district, Washington School is an exemplar

of effective core instruction in the first semester of first grade with students with similar skills at the beginning of first grade.

How are they structuring the school day? How are they assigning resources? What curriculum are they using? How can we support McKinley to accomplish the same

outcomes?

1/13/05 Portland OR (c) 2005 111

Classroom and Student Level Reports

Classroom level reports can identify strengths and weaknesses within a school, but caution is indicated. Sometimes students with additional needs or

challenges are grouped together in a class. Sometimes reading instructional groups are

organized across classes. Sometimes student mobility impacts one class

more than another. The most important level of interpretation and the

clearest information is the schoolwide report.

1/13/05 Portland OR (c) 2005 112

Summary of Effectiveness by Student and Classroom

1/13/05 Portland OR (c) 2005 113

1/13/05 Portland OR (c) 2005 114

Target of Opportunity

Identifying a classroom, schoolwide, or even district-wide are needing substantial support is a target of opportunity.

Needs Support or Needs Substantial Support means we have the knowledge, skills, curriculum, interventions to accomplish better outcomes for the instructional step and contribute to changing reading outcomes in third grade.

1/13/05 Portland OR (c) 2005 115

Themes Don’t loose track of the bottom line. Are we getting closer to

important and meaningful outcomes? Review Outcomes on -- and teach -- what is important:

Phonemic Awareness, Alphabetic Principle, Accuracy and Fluency with Connected Text

Alphabetic Principle is an important instructional goal and target of reviewing outcomes.

Use Effectiveness Reports to make decisions that support systems to change outcomes for children.

Evaluating Effectiveness should be efficient and purposeful. Start early! Trajectories of reading progress are very difficult

to change.

1/13/05 Portland OR (c) 2005 116

IDEL Fluidez en la segmentaciόn de fonemas

Indicador 3 Kindergarten Fluidez en la segmentación de fonemas IDELTM uva /u/ | /v/ /a/ salir /s/ /a/ | /l/ /i/ /r/ ___/8

lago /l/ /a/ | /g/ /o/ copa /c/ /o/ | /p/ /a/ ___/8

luz /l/ /u/ /s/ lindo /l/ /i/ /n/ | /d/ /o/ ___/8

beso /b/ /e/ | /s/ /o/ miedo /m/ /i/ | /e/ | /d/ /o/ ___/9

su /s/ /u/ tres /t/ /r/ /e/ /s/ ___/6

lleno /y/ /e/ | /n/ /o/ breve /b/ /r/ /e/ | /v/ /e/ ___/9

este /e/ /s/ | /t/ /e/ lloro /y/ /o/ | /r/ /o/ ___/8

antes /a/ /n/ | /t/ /e/ /s/ lima /l/ /i/ | /m/ /a/ ___/9

foca /f/ /o/ | /c/ /a/ seca /s/ /e/ | /c/ /a/ ___/8

pasa /p/ /a/ | /s/ /a/ sale /s/ /a/ | /l/ /e/ ___/8

goma /g/ /o/ | /m/ /a/ poner /p/ /o/ | /n/ /e/ /r/ ___/9

sello /s/ /e/ | /y/ /o/ amiga /a/ | /m/ /i/ | /g/ /a/ ___/9

Total _____

Tipos de errores:

© 2003 Dynamic Measurement Group, Inc. Página 21

“Voy a decir una palabra. Después de que la digo, quiero que me digas todos los sonidos que tiene la palabra. Por ejemplo, si yo digo, “oso,” tú dices /o/ /s/ /o/.” Vamos a probar. (pausa) Dime los sonidos en “mesa.”

1/13/05 Portland OR (c) 2005 117

Correction ProcedureRESPUESTA CORRECTA: Si el alumno dice /m/ /e/ /s/ /a/, usted dice,

RESPUESTA INCORRECTA:

Si el alumno dice cualquier otra respuesta, usted dice,

Muy bien. Los sonidos en la palabra “mesas” son /m/ /e/ /s/ /a/.

Los sonidos en la palabra “mesa” son /m/ /e/ /s/ /a/. Ahora te toca a ti. Dime los sonidos en “mesa”.

Bien. Aquí viene tu primera palabra.

1/13/05 Portland OR (c) 2005 118

Directions for Scoring1. Underline the sound segments in the word the

student produces that are correctly pronounced.

• Total Parts – Todas Las Partes (TLP): Students receive 1 point for each different, correct, part of the word – Same scoring rule as PSF.

• Syllable Parts – Sílabas (Síl): 1 point for each different, correct, syllable part, excluding one syllable words (no cuente monosílabos).

2. Put a slash ( / ) through segments pronounced incorrectly.

3. Leave segments that are omitted blank.4. If the student says the whole word, circle the entire

word and give 0 points.

Examiner says “abajo” student says “a...b...a...j...o”Examiner says “lista” student says “l...i...s...t...a”Examiner says “lana” student says “l...a...n...a”Examiner says “sala” student says “s...a...l...a”

Correct Phoneme Segmentation

Scoring Examples:Underline Correct Sound Segments

TLP Síl

abajo /a/ | /b/ /a/ | /j/ /o/ lista /l/ /i/ /s/ | /t/ /a/ __/10 ___/5

lana /l/ /a/ | /n/ /a/ sala /s/ /a/ | /l/ /a/ ___/8 ___/4

Todas Las Partes (TLP): ____ Sílabas (Síl): ____

Tipos de errores: (no cuente monosílabos)

18

10 1

1

8 0

Examiner says “abajo” student says “a...ba...jo”Examiner says “lista” student says “lis...ta”Examiner says “lana” student says “la...na”Examiner says “sala” student says “sa...la”

Correct Syllable SegmentationScoring Examples:

Underline Correct Sound Segments

TLP Síl





9

5 5

9

4 4

Examiner says “abajo” student says “a...b...a...jo”Examiner says “lista” student says “l...i...s...ta”Examiner says “lana” student says “l...a...na”Examiner says “sala” student says “s...a...la”

Mixed Phoneme & Syllable Segmentation

Underline Correct Sound Segments

TLP Síl





14

8 3

5

6 2

Practice Scoring Phonemes and Syllables

ella /e/ | /y/ /a/ bebé /b/ /e/ | /b/ /e/ ___/7 ___/4

grasa /g/ /r/ /a/ | /s/ /a/ poder /p/ /o/ | /d/ /e/ /r/ __/10 ___/4

pan /p/ /a/ /n/ unas /u/ | /n/ /a/ /s/ ___/7 ___/2

TLP Syl

Todas Las Partes (TLP): ____

Sílabas (Síl): ____ (no cuente monosílabos)

7

4 4

1

7 2

18

7

1/13/05 Portland OR (c) 2005 123

Pronunciation Guide Different countries or regions of a country use

different Spanish dialects. These pronunciation examples may be modified consistent with regional dialects and conventions.

For example, /s/ as in gustar is pronounced as a /j/ in certain regions of Central America and the Caribbean. Our examples are typical of the pronunciation in Mexico City.

1/13/05 Portland OR (c) 2005 124

Pronunciation GuideFonema Ejemplo de

Fonema Fonema Ejemplo de Fonema

/a/ hablar /g/ tango, gusta /e/ gente, ellos /j/ gente; juvenil; México /i/ hijo, iglesia /l/ loro, habla /o/ oso, ruido /m/ mano, vamos /u/ suyo, uno /n/ nadie, tren, una /ai/ aire, bailar /ñ/ año; muñeco /au/ jaula /p/ lápiz, piel /ei/ beisbol, seis /r/ ratón, comporta /eu/ eucalipto, Europa /rr/ chorro, perro

/oi/ oigo

/s/ rosa, cenar, zapato, siete

/b/ vista; combinación /t/ tapa, boleto /k/ con, pico, pequeño /x/ extraño

/ch/ chocolate, leche /y/ amarillo, ya /d/ dedo, morado /z/ zorro /f/ fino, café

1/13/05 Portland OR (c) 2005 125

Accommodations for FSF Setting/Examiner

Retest: Repeated assessment on different days with different probes.

Test in alternate setting, e.g., complete quiet, minimal distractions, enhanced lighting.

Test with familiar person, interpreter, specialist etc. present

Test by person with specialized training (e.g., SLP). Directions

Check child’s understanding (have the child repeat what to do).

Provide directions in child’s primary language. Repeat practice example, provide an additional example. Provide lead example (e.g., do it with me) in addition to

model.

Documents

Advanced DIBELS & IDEL: Scientifically Based Reading Research into Practice Roland H. Good III University of Oregon Reading First