Early Childhood Outcomes ECO Institute

Early Childhood Outcomes ECO Institute

Kathy Hebbeler, ECO at SRIRobin Rooney ECO at FPG

Prepared for the Office of Early Learning and School ReadinessOhio Department of Education

May 2010

What we hope you get out of our time together

• Understand the Summary Statements for child outcomes– The background, the meaning, the math

• Develop at least a beginning understanding of how to analyze child outcomes data

• Begin to understand how to use data for program improvement

• Be able to explain Summary Statements and other data to public and media

2

The Summary Statements

1. Of those children who entered the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program.

2. The percent of children who were functioning within age expectations in each Outcome by the time they turned 6 years of age or exited the program.

3

Why do you care?

• In 2011, data on the Summary Statements will

be released by local district for the first time.

• What does this mean for you?

4

Possible Implications

• You will be asked to explain what the summary statements mean to ____?

• You will be asked to explain/defend related data for your district.

• You will choose to examine your data to see what you can learn about the kinds of progress children are making in your district. – Program improvement

5Early Childhood Outcomes Center

Background on the Summary Statements

1992 Osborne and Graebler publish Reinventing Government

1993 GPRA (Government Performance and Results Act) passed

2003 2003 Program Assessment Review Tool (PART) reviews 130 federal programs including Part C and Part B preschool and gives both programs a score of 0 for accountability (“Results Not Demonstrated”)

2004 IDEA reauthorized. Shifts focus of monitoring to results.

6

Federal accountability and the focus on results:

IDEA 2004

7

SEC. 616. <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. (a) Federal and State Monitoring.-…..…..(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- (A) improving educational results and functional outcomes for all children with disabilities;

Child Outcomes for Special Education

2003 OSEP funds the Early Childhood Outcome (ECO) Center

2005 OSEP announces indicator for the three child outcomes

2008 States first report data on the 5 progress categories

2010 States set targets for the Summary Statements

2011 Summary statement data released by district


OSEP Reporting Requirements: Child Outcomes

– Positive social emotional skills (including positive social relationships)

– Acquisition and use of knowledge and skills (including early language/ communication and early literacy)

– Use of appropriate behaviors to meet their needs

9

What States Report:OSEP Reporting Categories

Percentage of children who:

a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer

to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged

peers but did not reach itd. Improved functioning to reach a level comparable to

same-aged peerse. Maintained functioning at a level comparable to same-

aged peers

103 outcomes x 5 “measures” = 15 numbers

What is the connection?

COSF ratings OSEP categories

Summary Statements


Early Childhood Outcomes Center 12

COSF Ratings

Ratings Review

The concepts in the scale can be summarized with a number or with a

statement.

Day 1, Handout 3



0 6 12 18 24 30 36 42 48 54 60 660

10

20

30

40

50

60

70

Developmental Trajectories

Functioning like same aged peersImproved functioning to that of same aged peersMoved closer to function-ing like same aged peersImproved functioning, no change in trajectoryDid not improve function-ing

Age in Months

Gro

wth

in O

utco

me

Day 1, Handout 4

Key Concepts Related to Progress Categories

• Progress categories require 2 data points for each child,

• are based on growth trajectories,

• compare a child to him or herself over time, and also compare each child to age expectations


What States Report:OSEP Reporting Categories

Percentage of children who:

a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer

to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged

peers but did not reach itd. Improved functioning to reach a level comparable to

same-aged peerse. Maintained functioning at a level comparable to same-

aged peers

163 outcomes x 5 “measures” = 15 numbers

Origin of the Summary Statements

• States reported on the OSEP Progress Categories for a few years

• States knew they would be asked to set targets

• Using the progress categories would require setting 15 targets…


Origin of the Summary Statements

• ECO prepared papers with options• Convened stakeholders• Extensive discussion about pros and cons

of various summary statementsSee Options and ECO Recommendations

for Summary Statements for Target Setting on the ECO web site

http://www.fpg.unc.edu/~eco/assets/pdfs/summary_of_target_setting-2.pdf


Summary Statement 1

1. Of those children who entered the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they turned 6 years of age or exited the program.


Other Ways to Think about Summary Statement 1

• How many children changed growth trajectories during their time in the program?

• Percent of the children who entered the program below age expectations made greater than expected gains, made substantial increases in their rates of growth, i.e. changed their growth trajectories

21Early Childhood Outcombegines Center

Formula for SS 1

(c+d)/(a+b+c+d)

22 22

Summary Statement 2

2. The percent of children who were functioning within age expectations in each Outcome by the time they turned 6 years of age or exited the program.


24

Other Ways to Think about Summary Statement 2

•How many children were functioning like same aged peers when they left the program?

•Percent of the children who were functioning at age expectations in this outcome area when they exited the program, including those who:• started out behind and caught up and• entered and exited at age level

25

Formula for SS 2

(d+e)/(a+b+c+d+e)

26

Activity

Use the Ohio data on Handout 8. 1. Calculate SS 1 for Outcome 1.2. Calculate SS 2 for Outcome 3.

Use the National data on Handout 8.3. Calculate SS 1 for Outcome 1.4. Calculate SS 2 for Outcome 3.

Early Childhood Outcomes Center

27

Other Summary Statements

Percent of children who made progress

What is the formula?

28

Other Summary Statements

Of those children who entered the program below age expectations in Outcome __, the percent who made progress

What is the formula?

Summary Statements 1 and 2 for Local Districts

• Summary statement data for local districts will be released in 2010.

• Be prepared to explain the numbers.• Step 1. Understand where the numbers

came from.• Step 2. Understand what the numbers

mean.


Who do you want to be?

30

Day 1, Handout 9

Documents

Early Childhood Outcomes ECO Institute