OSEP Initiatives on Early Childhood Outcomes

Presented at ECEA-SCASS Meeting

Savannah, GeorgiaOctober, 2010

OSEP Initiatives on Early Childhood Outcomes

Kathy Hebbeler

Early Childhood Outcomes Center

SRI International

Topics

• What state are required to report

• State approaches

• Most recent data

• Child Outcomes Measurement Framework

2Early Childhood Outcomes Center

Reporting Requirement for

Early Intervention and

Preschool Special Education

3Early Childhood Outcomes Center

Early Childhood Outcomes Center

Why does the federal government want data on child outcomes?

• Government Performance and Results

Act (GPRA)

• Program Assessment Rating Tool (PART)

• Individuals with Disabilities Education

Act (IDEA)

Requires goals and indicators be established for

IDEA

Indicators and data collection for school age

population included data on outcomes

Previously, for early childhood data had been

collected on:

• Number of children served (Part C)

• Settings (both Part C and 619

Government Performance and Results Act (GPRA) passed in 1993

Early Childhood Outcomes Center

130 programs examined in 2002; 50%

programs had no performance data

Programs looking at inputs, not results

Part C and Section 619

No long-term child outcome goals or data

Department of Education needs to develop

a strategy to collect annual performance

data in a timely manner

OSEP: PART evaluation results (2002)

SEC. 616. <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.…..

(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- (A) improving educational results and functional outcomes for all children with disabilities;

IDEA 2004

8

OSEP Reporting Requirements: Child Outcomes

– Positive social emotional skills (including positive social relationships)

– Acquisition and use of knowledge and skills (including early language/ communication and early literacy)

– Use of appropriate behaviors to meet their needs

What States Report:OSEP Reporting Categories

Percentage of children who:

a. Did not improve functioningb. Improved functioning, but not sufficient to move

nearer to functioning comparable to same-aged peers

c. Improved functioning to a level nearer to same-aged peers but did not reach it

d. Improved functioning to reach a level comparable to same-aged peers

e. Maintained functioning at a level comparable to same-aged peers

3 outcomes x 5 “measures” = 15 numbers

Early Childhood Outcomes Center 10

Reporting details

• Progress for all children who exited between July 1, 2009 and June 30, 2010

• Stayed in the program at least 6 months

• Data will be reported to OSEP in February 2011

• Data reported for the first time for children who exited in 07-08 year.

11Early Childhood Outcomes Center

The Summary Statements

1. Of those children who entered the program below

age expectations in each outcome, the percent who

substantially increased their rate of growth by the

time they turned 3 [6] years of age or exited the

program.

2. The percent of children who were functioning within

age expectations in each outcome by the time they

turned 3 [6] years of age or exited the program.

12

Formula for SS 1

(c+d)/(a+b+c+d)

1313

Formula for SS 2

(d+e)/(a+b+c+d+e)

14

State approaches

15Early Childhood Outcomes Center

16Early Childhood Outcomes Center

17Early Childhood Outcomes Center

State Approaches to Measuring Child Outcomes

Approach

Part C

(56 states/jur)

Preschool

(59 states/jur)

One tool statewide 7/56 (13%) 9/59 (15%)

Publishers’ online analysis

3/56 (5%) 6/59 (10%)

COSF 7 pt. scale 41/56 (73%) 38/59 (64%)

Other 5/56 (9%) 7/59 (10%)

Child Outcomes Rating Form (COSF)

• 7-point rating scale with defined criteria for each point

• Criteria describe child’s functioning relative to same aged peers

• Child’s team uses multiple sources of information to assign rating

• Rating assigned at program entry and program exit

19Early Childhood Outcomes Center

The State Data

for ‘08-’09

20Early Childhood Outcomes Center

N=66,000

N=113,700

Early Childhood Outcomes Center 27

A Framework and Self Assessment

for Building a Child Outcomes

Measurement System

Purpose of the Framework

• Provide a common language for ECO and other TA providers to use in discussing COMSs with states.

• Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS.

• Serve as the organizing structure for the self assessment

28Early Childhood Outcomes Center

Framework and Self-Assessment

• FRAMEWORK– Set of components and quality indicators– Provides the structure for the self-assessment

• SELF-ASSESSMENT – Scale that provides criteria for levels of

implementation within each quality indicator– Rating assigned based on level of

implementation within each indicator 29

Early Childhood Outcomes Center

Process for Framework Development

• Built off what we had learned from ECO work with states and previous ECO conceptual framework

• Literature review• Repeated discussion and

review internally and with 7 Partner States

30

Framework Partner States

State Part C 619

California X

Colorado X X

Delaware X X

Maine X X

Minnesota X X

New York X

Ohio X X

31

Early Childhood Outcomes Center32

Early Childhood Outcomes Center 33

The state has effective procedures for collecting,

storing, and transmitting data to the state.

Early Childhood Outcomes Center 34

State coordinates child outcomes measurement and data use across EC

systems.

Quality Indicator

• Provides additional detail as to what constitutes quality implementation of the component.

• 18 quality indicators across the 7 components

35Early Childhood Outcomes Center

Quality Indicators for Data Collection and Transmission

2. Data collection procedures are carried out efficiently and effectively.

3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment.

4. State's method for entering, transmitting, and storing data is effective and efficient.

36Early Childhood Outcomes Center

Structure of Self Assessment

• Components (7) = Major areas of framework

– Quality Indicators (18 total) = Statements of basic requirements of a quality COMS

• Elements (number varies with each indicator) = Define what constitutes high quality on the Quality Indicator.

37Early Childhood Outcomes Center

38Early Childhood Outcomes Center

39Early Childhood Outcomes Center

40

41Early Childhood Outcomes Center

42

The Scale for the Quality Indicators

43Early Childhood Outcomes Center

Implementation of Elements

Quality Indicator

ScoreAll elements are fully implemented 7

Nearly all elements are fully implemented and the rest are in process 6Most of the elements are fully implemented and the rest are in process. 5

At least one element is fully implemented and the rest are in process 4

All of the elements are in process 3

Some of the elements are in process 2

None of the elements are yet in process 1

Early Childhood Outcomes Center 44

Recommended State Approach for Using the Self Assessment

1. Complete the entire self assessment.

2. Identify the component(s) and quality indicators to address first.

3. Develop action plan to improve the related elements.

4. Implement improvement activities.

5. Re-assess status and identify “next step” priorities at regular intervals

45Early Childhood Outcomes Center

Is and Isn’t

• Is a comprehensive resource to alert states to all the pieces that need to be in place to have a well functioning COMS

• Is not a cookbook or roadmap with each step in the process spelled out.– Way too many decisions!

46Early Childhood Outcomes Center

Where would a state start?

47Early Childhood Outcomes Center

Packaging

• Entire tool will exist online– Live link from each element to a “back up”

section– Profile will be filled automatically based on the

QI pages• Will develop a version with live links to the back

up and profile that will operate off line.• Version that can be printed off as a manual.

48Early Childhood Outcomes Center

Next steps for ECO

• Populate the COMS framework with resources related to the components

• Revise the self assessment based on feedback

• Develop a framework for a Family Outcomes Measurement System

49Early Childhood Outcomes Center

System for Producing Good Child and Family Outcomes

Good Federal

policies and programs

Good State policies

and programs

High quality services and supports for children 0-5

and their families

Good outcome

s for children

and families

Good Local policies

and programs

Information infrastructure

Evidence Based Practice

Adequate funding

Strong Leadership

Prof’l Development•Preservice•Inservice

Early Childhood Outcomes Center 51

Early Childhood Information Infrastructure:

Data Needed for Program Improvement

WHO SERVICES

COSTPERSONNEL

OUTCOMES

For more information

• For updates to the framework and the self-assessment and resources to support the quality indicators:

www.the-eco-center.org

52Early Childhood Outcomes Center