Child Outcomes for Early Intervention and Preschool Special Education: The National Picture

Presented at OSEP Project Directors Meeting

Washington, DCJuly, 2012

Child Outcomes for Early Intervention and Preschool Special

Education: The National Picture

Lynne KahnECO at FPG

Kathy HebbelerECO at SRI

Overview

• National numbers and the quality of the state data

• The findings for 2010-11 • Findings for the last 3 years• State variation and data quality

2Early Childhood Outcomes Center

3Early Childhood Outcomes Center

OSEP Reporting Requirements: Child Outcomes

• Positive social emotional skills (including positive social relationships)

• Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])

• Use of appropriate behaviors to meet their needs

If you have not seen it….

4Early Childhood Outcomes Center

www.the-eco-center.org

State Approaches to Outcomes Data, 2010-11

Approach Part C (56 states/jurisdictions)

Preschool(59 states/jurisdictions)

COS* 7 pt. scale 43/56 (77%) 36/59 (61%)

One tool statewide 7/56 (13%) 9/59 (15%)

Publishers’ online analysis

3/56 (5%) 6/59 (10%)

Other 5/56 (9%) 8/59 (14%)

*Child Outcomes Summary Rating

• States are in various stages with regard to building their child outcomes measurement systems

• Some phrased in• Some changed approaches

6Early Childhood Outcomes Center

Child Outcomes: What we know

3 Methods Methods for Calculating National Estimates

1. States with the highest quality data weighted by child count to represent all states*

2. All states weighted by child count3. All states averaged (each state weighted

as 1)

7Early Childhood Outcomes Center

*The data we will be presenting for the national picture

Identifying the States with Highly Questionable Data

Criteria for eliminating states:• Not reporting data on enough children• Odd patterns in the data• Review of method reported suggested the

data were of questionable quality

8Early Childhood Outcomes Center

Continued increase in the number of states included

9Early Childhood Outcomes Center

08-09 09-10 10-11

Pt. C 19 29 39

Part B Preschool 15 33 36

Number of States that Met Criteria for Inclusion in the National Analysis

• And now the data….

10Early Childhood Outcomes Center

And now the data….

11Early Childhood Outcomes Center

OSEP Reporting Categories

Percentage of children who: a. Did not improve functioningb. Improved functioning, but not sufficient to move nearer

to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged

peers but did not reach itd. Improved functioning to reach a level comparable to

same-aged peerse. Maintained functioning at a level comparable to same-

aged peers

3 outcomes x 5 “measures” = 15 numbers

Illustration of 5 Possible Paths

0

10

20

30

40

50

60

70

1 6 11 16 21 26 31 36 41 46 51 56Age in Months

Sco

re

Maintained functioning comparable to age peersAchieved functioning comparable to age peersMoved nearer functioning comparable to age peersMade progress; no change in trajectoryDid not make progress

13*Based on 39 states with highest quality data

14*Based on 36 States with the highest quality data.

The Summary Statements

1. Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program.

2. The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program.

15Early Childhood Outcomes Center

16*Based on 39 states with highest quality data

17*Based on 36 States with the highest quality data.

Part C: Trends over Time

18Early Childhood Outcomes Center

Part C: Trends over Time

19Early Childhood Outcomes Center

Part B Preschool: Trends over Time

20Early Childhood Outcomes Center

21Early Childhood Outcomes Center

Part B Preschool: Trends over Time

Looking for Patterns

22Early Childhood Outcomes Center

Early Childhood Outcomes Center 23

24Early Childhood Outcomes Center

%

Early Childhood Outcomes Center 25

Building the national capacity to report and use child outcomes data is a

long term endeavor.

26Early Childhood Outcomes Center

Part C: States continue to report on more children

Total Number of Children States Included in the Child Outcomes Data

Number of children reported

Number of States and Jurisdictions

2007-08(N=56)

2008-09(N=56)

2009-10(N=54)

2010-11(N=54)

99 or less 13 6 5 4

100-499 25 16 9 10

500-999 6 13 10 7

1,000-1,999 9 11 10 13

2,000+ 3 10 20 22

27Early Childhood Outcomes Center

28Early Childhood Outcomes Center

Early Childhood Outcomes Center 29

Total Number of Children Included in Progress Data

Number of children reported

Number of States and Jurisdictions

FFY 2007(N=58)

FFY 2008(N=58)

FFY 2009(N=58)

FFY 2010 (N=59)

10 or fewer 1 1 1 1

10-99 11 7 6 7

100–499 14 6 4 2

500-999 10 12 10 8

1000–1999 8 7 9 5

2000–2999 5 10 11 12

3000–4999 5 11 9 13

5000-8999 3 3 7 9

9000+ 1 1 1 2

Part B Preschool: States continue to report on more children

Early Childhood Outcomes Center 30

31Early Childhood Outcomes Center

%

Early Childhood Outcomes Center 32

Conclusions

• The data show good outcomes for many children who attend Part C and Part B, Preschool.

• The data quality continues to get better.• The data will be a powerful tool for

program improvement when– The data are consistently good at the local level– States and local programs consistently use it.

33Early Childhood Outcomes Center

For more information….

www.the-eco-center.org

34Early Childhood Outcomes Center

35Early Childhood Outcomes Center

Identifying States with Quality Data

36Early Childhood Outcomes Center

Calculating Missing Data for Part C

Proxy for missing data =Number with data for C3/

Exiting Data (618)

37Early Childhood Outcomes Center

• Do not expect this number to be 100%

• ..but we don’t expect it to be 10% either

Percentage of Exiters Reported by Part C Continues to Increase

38Early Childhood Outcomes Center

Calculating Missing Data for 619

Proxy for missing data =Number with data for B7/

Child count

39Early Childhood Outcomes Center

• Do not expect this number to be 100%

• ..but we don’t expect it to be 10% either

Percentage of Child Count Reported by Part B Preschool Continues to Increase

40Early Childhood Outcomes Center

08-09 09-10 10-1102468

1012141618

Pt. B Preschool: % of Child Count Reported in Child Outcomes Data

<10%

10-20%

21-30%

31-40%

41-50%

>50%Num

ber o

f Sta

tes

Many states reporting a low percentage for “a” but still have a few outliers.

Part CPart B

Preschool<2% 32 302 to <5% 11 115 to <7% 5 3

7 to <13% 0 5

28 to 49% 3 0

41Early Childhood Outcomes Center

Number of States by % Reported in “a,” 2010-11

Early Childhood Outcomes Center 42

Reason Part C state was excludedNumber

excludedPercent(n=51)

State is sampling 3 6No outcomes data reported 0 0Reported outcomes data on less than 28% of reported exiters

3 6

Had at least one outcome with category a greater than 10% or category e greater than 65%

4 8

Reported outcomes data on less than 28% of reported exitersANDHad at least one outcome with category a greater than 10% or category e greater than 65%

1 2

Questionable data quality based on review of SPP/APR and knowledge gained through TA

1 2

States included in the analysis 39 76

Early Childhood Outcomes Center 43

Reason Part B state was excludedNumber

excludedPercent (n=51)

State is sampling 4 8No outcomes data reported 1 2Reported outcomes data on less than 12% of child count 2 4

Had at least one outcome with category a greater than 10% or category e greater than 65% 4 8

Reported outcomes data on less than 12% of child countANDHad at least one outcome with category a greater than 10% or category e greater than 65%

0 0

Questionable data quality based on review of SPP/APR and knowledge gained through TA 2 4

No child count data available 1 2Reported Summary statements but no progress categories 1 2

States included in the analysis 36 71

Producing National Estimates Using the States with the Highest Quality Data

1. Divide the states into those with large, medium, and small child counts.

2. Identify the states that appear to have the best data.

3. Weight by child count and calculate the overall OSEP percentages using the “best data” states in each category.

4. Use the OSEP percents and total child count for each category to get the national estimates.

44Early Childhood Outcomes Center