Using the PMRN, ECI and EI Indices, and Other Information to Make Effective Decisions Elizabeth Crawford, MS, CCC-SLP Director of Interventions The Florida

Using the PMRN, ECI and EI Indices, and Other Information to Make Effective Decisions

Elizabeth Crawford, MS, CCC-SLPDirector of Interventions

The Florida Center for Reading Research

2

Agenda

Risk Levels and Instructional Levels of DIBELS

Use Data to: interpret ECI and EI tables calculate ECI and EI indices at the class level manage school level intervention resources monitor progress of specific groups of

students Concluding Thoughts and Resources

3

Risk Levels and Instructional Levels of

DIBELS

4

Risk Level Key

5

6

SY0607 Florida DIBELS Recommended Level of Instruction Decision Rules

http://www.fcrr.org/assessment/pdf/ril_decisionrules_0607.pdf

7

Various Uses of Data

8

Use Data to…

Interpret ECI and EI tables Calculate ECI and EI indices at

Class Level Manage school level intervention

resources Monitor progress of specific groups

of students

9

School Level Indices

10

ECI Definition Effectiveness of Core Instruction

(ECI) — this index shows the percentage of students who began the year reading “at grade level” and are continuing to meet grade level expectations at the mid-year or end of year assessment. This index tells you whether the core instruction is sufficiently powerful to produce a year’s growth in reading skill for a year’s worth of instruction for all students who begin the year at grade level.

11

ECI – “School”

91% of the 45 kindergarteners who started the year on grade level remained on grade level.91 x 45 = 41 children41 out of 45 children remained on grade level

.97 x 75 = 73 children73 out of 75 children remained on grade level, 2 children did not remain on grade level

In first grade, my school is doing equal to or better than 96% of the RF schools on keeping our 1st grade students on grade level.

12

EI Definition Effectiveness of Interventions (El)* —this

index shows the percentage of your students who began the year at some level of risk for reading difficulties (reading below grade level) but who have grown rapidly enough to advance to a lower level of risk by the mid year or end of year assessment (i.e. they move from intensive to strategic, or from strategic to grade level). This index, and the two explained below, provide information about how effectively intervention programs are working to accelerate the development of students who are lagging behind in reading growth.

13

EI – “School”

T81% of the 42 kindergartners who started the year at some level of risk, have made gains and are now at a lower level of risk. .81 x 42 = 34 students

In first grade, my school is doing equal to or better than 44% of the RF schools in moving children to strategic or grade level.

.23 x 30 = 7 students7 out of 30 3rd graders who started at some level of risk have moved to a lower level of risk. 23 students are still at some risk.

14

EI-S Definition

Effectiveness of Interventions for Strategic Students (EI-S) — this index shows the percentage of students that moved from the “strategic” risk level to grade level.

15

EI – S “School”

.56 x 27 kindergarten students who started the year at “strategic” moved to “grade level” = 15

27-15 = 12 are still “strategic”

In 1st grade, when compared to the other 583 RF schools in FL, We are doing equal to or better than 48% of them at moving first grade “strategic” students to “grade level”

In 3rd grade, 62% of the 16 children (10) who started at “strategic” are now at “grade level”

16

EI-I Definition

Effectiveness of Interventions for Intensive Students (EI-I) — this index shows the percentage of students that moved from the “intensive” risk category to the “strategic” risk level or to grade level.

17

EI – I “School”

93% of our 14 kindergarten students who started the year at “intensive” made progress to a lower risk level..93 x 14 = 13 students made growth, 1 is still “intensive”

In second grade, we are doing an equal or better job than 47% of the RF schools at moving our children out of the “intensive” category.

In all grades, K-3, of the 46 students who started at “intensive” 50% (or 23) of them made growth to “strategic” or “grade level.”

18

State Averages for ECI and EI Indices: Numbers to consider when

comparing data across years

Due to different level of difficulty with passages from assessment to assessment and year to year, it is sometimes difficult to compare indices across assessments and years.

19

Background

For all subtests of DIBELS, the same forms are used at each assessment from year to year, except Oral Reading Fluency (ORF).

In order to decrease familiarity with passages, a rotation system was used – passages rotated on a four year cycle.

20

Background

This rotation systems began in 2003 and was created based on the assumption that all DIBELS ORF passages were roughly the same level of difficulty.

Also, three passages are used with a median score to further to protect against difficulty.

21

ECI State Averages

Index End of Year 07-08 Middle of Year 07-08 End of Year 06-07

ECI Mean Mean Mean

ECI_K 91.4 ECI_K 84.0 ECI_K 90.1

ECI_1 71.3 ECI_1 74.3 ECI_1 73.0

ECI_2 74.5 ECI_2 82.9 ECI_2 74.8

ECI_3 82.9 ECI_3 97.4 ECI_3 87.3

ECI_ALL 78.0 ECI_ALL 82.7 ECI_ALL 79.5

Higher when compared to EOY 07-08

Lower when compared to EOY 07-08

22

EI State Averages




EI Mean Mean Mean

EI_K 72.5 EI_K 57.7 EI_K 69.8

EI_1 24.0 EI_1 22.1 EI_1 25.5

EI_2 10.6 EI_2 7.8 EI_2 11.6

EI_3 35.1 EI_3 59.9 EI_3 33.2

EI_ALL 42.4 EI_ALL 45.4 EI_ALL 39.3

23

EI_S State Averages




EI-S Mean Mean Mean

EIS_K73.2

EIS_K52.8

EIS_K70.7

EIS_123.4

EIS_120.7

EIS_125.9

EIS_211.0

EIS_29.7

EIS_213.4

EIS_327.8

EIS_364.8

EIS_331.3

EIS_ALL42.1

EIS_ALL44.5

EIS_ALL40.3

24

EI_I State Averages




EI-I Mean Mean Mean

EII_K70.3

EII_K68.4

EII_K67.3

EII_125.9

EII_126.8

EII_126.2

EII_210.5

EII_24.5

EII_29.2

EII_343.8

EII_355.0

EII_336.3

EII_ALL44.5

EII_ALL47.9

EII_ALL38.2

25

Points to Remember

We recognize that this variability in difficulty makes it difficult to compare classroom and school growth in ORF from one year to next so we are taking steps to correct this in the next set of early progress monitoring measures being developed for use in Florida.

26

Points to Remember If your school wants to know how you are doing in

improving your ECI and EI indices, the best way is to examine your percentile rank compared to other Reading First schools in the state.

If your percentile rank increased from last year to this year, then you are doing better than average in improving your students’ performance on these indices.

Perhaps more important than the year to year change in ECI and EI, though, is relative standing on these measures. If you have a very low percentile rank on any of these indices, that means that your instruction is relatively weak when compared to that provided in other Reading First schools.

27

• This school year, 08-09, PMRN will generate this School RLI report with your ECI and EI data. Percentile Ranks will only be generated once all Reading First School Data is entered.

• The tables on this report are very similar to the tables that were explained above.

28

29

Calculating the Class Level ECI, EI-I, and EI-S

How do you do it? AND…what does it all mean??

Refer to “DIBELS Data Analysis Conference” Worksheet Handout

30

31

Key ‘follow-up’ sections on worksheet

Items Discussed Roles and Responsibilities

Teacher will… Reading Coach will… Principal will…

32

Class Recommended Level of Instruction Report

This report provides a summary of the students’ overall progress. It can be used to get an overall sense of instructional levels in the class and to calculate the Effectiveness of Core Instruction (ECI) index and the three Effectiveness of Intervention (EI) indices.

33


34


This is the same type of report with “all” students included. Students 5, 6, 7, and 19 would not be used to calculate the ECI and EI indices because they were not in the school for all three assessment periods. This is why we select the full year only function for calculation.

35


You can use the pie charts to help you fill out the top part of the DIBELS worksheet

Initial Level = Low risk

Strategic Level = Moderate Risk

Intensive Level = High Risk

Assessment #3

% Initial = 44

% Strategic = 50

% Intensive = 6

36


To calculate ECI – Effectiveness of Core Instruction for the year, use only those students that were at Initial Instruction during the 1st assessment and remained in the class throughout the year (that is why we select full year only).

Divide the number of the students that finished at Initial Instruction for the 3rd assessment (5) by the number of students that were at Initial Instruction for the 1st assessment (9). The higher the percentage the better.

37


Effectiveness of Core Instruction

# stayed initial 5--------------------- = -------# started initial 9

5

--- = 56% ECI = 56% 9

Target Students:

Students 3, 6, 8, 9 (Initial to Strategic)

38


To calculate EI – Effectiveness of Intervention for the year, use only those students that were in the class for the first assessment and the last assessment (full year only).

Take the number of students who finished the year at ‘initial’ or ‘strategic’ (if they started at intensive) level by the students who started the year at some level of risk (intensive or strategic)

The higher the percentage the better.

39


Effectiveness of Intervention

# moved to initial or strategic from Intensive 2--------------------- = -------# started strategic & intensive 7

2

--- = 29% EI = 29% 7

Target Students:

Students 4, 7, and 10 (strategic to intensive during assessment 2)

40


Effectiveness of Intervention-Strategic

# moved to initial 1--------------------- = -------# began strategic 5

1

--- = 20% EI-S = 20% 5

Target Students: Students 4, 7, and 10 (strategic to intensive during assessment 2)

41


Effectiveness of Intervention - Intensive

# moved to strategic or initial 1--------------------- = -------# began intensive 2

1

--- = 50% EI-I = 50% 2

Target Students:

Student 5

42

What do all of these indices mean for my school and teachers? This particular teacher in 1st grade is struggling both with

maintaining children at grade level and reaching her struggling readers:

ECI = 56% EI = 29% EI-S = 20% EI-I = 50%

You may want to first determine if other first grade teachers have similar indices and if so, you want to examine your Core Reading Program and especially your Intervention Programs

You can look at the School Recommended Level of Instruction Report to see how different grade levels are doing by comparing the pie charts from one assessment period to the next

The goal is to see the green increase and the red decrease – to make the pie chart look more like an olive!!

43

Use Data to… monitor progress of specific groups of students

44

45

School Demographic Reports

Demographic data is available, by assessment period, at the school and district levels for individual measures and the Recommended Level of Instruction.

The data file allows all of the data to be downloaded so that Users can complete their own analyses.

46

47

48

School Demographic Reports

You can also display a bar graph for a specific demographic group from this report. The next slides shows a bar graph for kindergarten students at the third assessment by Ethnicity.

The total number of students in each category is in ( ) next to the group name.

For example Hispanic (17)

49

50

51

Use Data to… manage school level intervention resources

52

School Level Reports

“The Chart” Highlight:

School Status (“Home Base”) School Demographics

53

“The Chart”

Use the chart as a “quick start” guide to the PMRN for the School Level Reports

Refer to separate handout The PMRN User’s Guide is the

complete manual and should be read and referenced for specific questions and information

54

55

56

Using the School Status Report – “Home Base”

Make a list of school wide resources Who needs extra support? (High Risk,

Moderate Risk, Low Risk) Which Grades? Which Teachers?

Which skills need to be emphasized in professional development?

57

58

School Status Report – “Home Base”

59


60


61

Navigating from the School Status Report – “Home Base”

School Grade Summary – Click on the words ‘grade summary’ under the grade want to see

Class Status – From the extended view, click on the teacher’s name

School Progress – From Grade Summary, click on ‘show progress report’ at bottom

Class Progress – From the School Progress, use active header menu to select teacher/class of interest

School Comparison – From School Progress, click on ‘show school comparison’ at bottom

** Any time you click on the number below a specific measure you will get a progress report and depending on the level of the report you started from will determine which level of progress report you get

Concluding Thoughts and

Resources

63

Concluding Thoughts Use PMRN Reports to make school,

classroom, and student level decisions: Monitor progress Determine which students need more

intensive instruction Offer instructional strategies to the teacher Determine types of professional

development from which specific teachers could benefit

By asking some of the following questions…

64

Is the core reading program implemented with fidelity? Effectiveness?

Are the interventions being implemented with fidelity? Effectiveness?

Is there a need for more professional development for a particular area of reading?

Is there a need for more intensive coaching? Are materials allocated to maximize learning? Is personnel use maximized to focus on reading

instruction? Is the instructional day scheduled such that reading is

THE priority?

Important Considerations

65

Florida Center for Reading Research 227 North Bronough Street,

Suite 7250 Tallahassee, FL 32301(850) 644-9352 - Phone (850) 644-9085 (Fax)www.fcrr.org

PMRN Help Desk 850-644-0931 (Voice)1-866-471-5019 (Toll Free)850-645-1700 (Fax)[email protected]

User’s guidehttp://www.fcrr.org/pmrn/userguides.htm

66

Questions

67

Thank You!Liz Crawford, MS, CCC-SLPDirector of InterventionsFlorida Center for Reading [email protected]

After the Leadership Conference, this presentation can be found on www.fcrr.org under my name on the Presentations and Publications section.

Documents

Using the PMRN, ECI and EI Indices, and Other Information to Make Effective Decisions Elizabeth Crawford, MS, CCC-SLP Director of Interventions The Florida