16
Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags Donna Noyes, New York Part C Program Lauren Barton, ECO at SRI International Cornelia Taylor, ECO at SRI International

Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Embed Size (px)

Citation preview

Page 1: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010

Looking for Patterns in Child Outcomes Data:

How to Examine Data for Red Flags

Donna Noyes, New York Part C Program

Lauren Barton, ECO at SRI International

Cornelia Taylor, ECO at SRI International

Page 2: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Session Overview

• Why do pattern checking?

• What are we looking for?

• Example: New York Part C Data

• Activity: Getting started looking for patterns

2Early Childhood Outcomes Center

Page 3: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

What is Pattern Checking?

• A process of slicing and displaying your child outcomes data in different ways

• Reveals relationships between variables

• Does this data look similar to or different from

– Data for various subgroups

– Overall data observed in other states

– The relationship or pattern you would expect to see

3Early Childhood Outcomes Center

Page 4: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Searching for Red Flags

Searching for patterns - sometimes • Warning signs… • Look more closely, can this be

right?– Missing data?– Data quality?– Data analysis?– Program itself?

Early Childhood Outcomes Center 4

Page 5: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Early Childhood Outcomes Center 5

Available on the ECO website www.the-eco-center.org, ECO resources,

Quality Assurance, Data Quality (called the Pattern Checking Table)

Page 6: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Why Undertake Pattern Checking?

• Makes a case that your data are valid (or not yet)• Uncovers information about the quality of data –

confidence and/or concern• Spotlight on areas for further investigation • Provides clues about where to target resources

– Quality improvement– Program improvement

6Early Childhood Outcomes Center

Page 7: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Outcomes Measurement Approach

• Pattern checking is a process

• It works with any outcomes measurement approach

• Ways you split the data may change

• Still looking for data to show the same relationships

• What you do with the red flags you find may differ

Early Childhood Outcomes Center 7

Page 8: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Caveat!!

• Remember this is only weighing the pig

• Weighing the pig does not make it fatter

• Need to take what you learn from the analysis and do something with it

8Early Childhood Outcomes Center

Page 9: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Using Data

Evidence

Inference

Action

The numbers, facts

What they mean from thinking about the facts in context

Implications, what to do

Early Childhood Outcomes Center 9

Page 10: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

State Example: New York Part C Program

10Early Childhood Outcomes Center

Page 11: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Pattern Checking Activity

11Early Childhood Outcomes Center

Page 12: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Example: Distribution of Progress Categories in Two States

Early Childhood Outcomes Center 12

Page 13: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Distribution of COSF Ratings at Entry in Two States

Early Childhood Outcomes Center 13

Page 14: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Progress Categories by Gender

14Early Childhood Outcomes Center

A B C D E

Males (n=312)

1.6% 28.4% 29% 23% 18%

Females(n=299)

1.0% 14% 26.5% 31% 27.5%

Page 15: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Crosstab of Entry and Exit Ratings on Outcome 1 (could use scores)

15Early Childhood Outcomes Center

Exit Outcom

e 1

Entry Outcome 1

1 2 3 4 5 6 7

1 7 1 1 0 0 0 2

2 20 16 4 1 2 0 0

3 33 36 26 5 8 1 0

4 16 28 39 1 4 12 4

5 22 62 72 40 24 9 8

6 34 63 106 64 93 26 11

7 20 39 58 44 44 31 14

Page 16: Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010 Looking for Patterns in Child Outcomes Data: How to Examine Data for Red Flags

Early Childhood Outcomes Center 16