Background of the Early Communication Indicator for Infants and Toddlers Dale Walker, Judith Carta, and Charles Greenwood

Background of the Early Background of the Early Communication Indicator for Communication Indicator for

Infants and ToddlersInfants and Toddlers

Dale Walker, Judith Carta, and Dale Walker, Judith Carta, and Charles GreenwoodCharles Greenwood

This WorkThis Work Early Childhood Research Institute Early Childhood Research Institute

on Measuring Growth and on Measuring Growth and DevelopmentDevelopment

Other partners:Other partners:– University of MinnesotaUniversity of Minnesota– University of OregonUniversity of Oregon

Developing Developing indicatorsindicators toward a set toward a set of nationally validated child of nationally validated child outcomes from birth to age 8 years outcomes from birth to age 8 years to produce a comprehensive to produce a comprehensive performance measurement system performance measurement system

University of KansasUniversity of KansasECI Development Project Staff ECI Development Project Staff

for Children birth to 3 yearsfor Children birth to 3 years::

Judith Carta, Charles Greenwood & Dale Judith Carta, Charles Greenwood & Dale WalkerWalker

With: Jane Atwater, Gayle Luze,With: Jane Atwater, Gayle Luze,

Deborah Linebarger, Carol Leitschuh,Deborah Linebarger, Carol Leitschuh,

Ken Parsley, Annessa Staab and Gabe Cline Ken Parsley, Annessa Staab and Gabe Cline

Objectives of Training:Objectives of Training:

Understand the background of the ECI and Understand the background of the ECI and this type of assessmentthis type of assessment

Learn how to administer and score the ECI Learn how to administer and score the ECI and how to become a certified administratorand how to become a certified administrator

Understand how to use ECI data for Understand how to use ECI data for monitoring individual progress and monitoring individual progress and evaluating programsevaluating programs

Learn about the website and how to enter Learn about the website and how to enter datadata

Learn how to train othersLearn how to train others

Current Focus on Current Focus on Outcomes-Based Program Outcomes-Based Program EvaluationEvaluation

More emphasis needed on child More emphasis needed on child outcomesoutcomes

Programs (and individual staff Programs (and individual staff members) need to know when members) need to know when they are making a difference in they are making a difference in moving children toward moving children toward outcomesoutcomes

Most available Most available approaches to program approaches to program evaluation:evaluation: Not linked to individual child Not linked to individual child

progress on outcomesprogress on outcomes Child data not used in making Child data not used in making

program decisionsprogram decisions– Not frequent enough Not frequent enough – Not accessible enoughNot accessible enough

Indicators of child Indicators of child progress are needed to progress are needed to determine if children are determine if children are growing in their growing in their development development

What’s an Indicator?What’s an Indicator?What is an Individual What is an Individual Growth and Development Growth and Development Indicator (IGDI)?Indicator (IGDI)?

What’s an indicator?What’s an indicator?

Way of monitoring progress toward Way of monitoring progress toward outcomes: knowing whether you outcomes: knowing whether you are on-trackare on-track

Tells you when a change is neededTells you when a change is needed If you make changes, helps you If you make changes, helps you

“see” the results“see” the results

Helpful features of the Helpful features of the thermometer indicator:thermometer indicator:

Quick measureQuick measure Gives instant informationGives instant information Tells a lot about an important Tells a lot about an important

general outcome: Health Statusgeneral outcome: Health Status Can repeat it frequentlyCan repeat it frequently Measures intervention resultsMeasures intervention results

PediatriciaPediatrician’s n’s

Standard Standard Growth Growth Charts: Charts:

Height and Height and Weight Weight

IndicatorsIndicators

Features of growth chartsFeatures of growth charts

Easy to doEasy to do Relatively inexpensiveRelatively inexpensive RepeatableRepeatable Can indicate potential problem and Can indicate potential problem and

effectiveness of an interventioneffectiveness of an intervention

General Outcome General Outcome MeasurementMeasurement

Not a new concept in educationNot a new concept in education– Recognized in special education Recognized in special education

practice and researchpractice and research– Reading Rate (CBM)Reading Rate (CBM)– DIBELS (Dynamic Indicators of Basic DIBELS (Dynamic Indicators of Basic

Early Literacy Skills)Early Literacy Skills)

Outcomes and Indicators to Outcomes and Indicators to Assess Progress Assess Progress

OutcomesOutcomes are general, socially are general, socially desired abilities/capabilitiesdesired abilities/capabilities IndicatorsIndicators are sensitive markers of are sensitive markers of progress progress

–Way of seeing if children (and programs) Way of seeing if children (and programs) are moving toward outcomes.are moving toward outcomes.–Share features of other effective Share features of other effective indicators:indicators:–Quick, easily repeatable, provide quick Quick, easily repeatable, provide quick feedbackfeedback–Show change easilyShow change easily

Expressive Expressive CommunicationCommunicationGeneral OutcomeGeneral Outcome

One of 15 general child outcomes One of 15 general child outcomes validated by a national sample of validated by a national sample of parents (n=351) and professionals parents (n=351) and professionals (n=672)(n=672)

Expressive communication was most Expressive communication was most highly rated outcomehighly rated outcome

Others include, social, movement, Others include, social, movement, problem-solving and adaptive behaviorsproblem-solving and adaptive behaviors

Expressive Meaning Expressive Meaning OutcomeOutcome

““The child uses gestures, The child uses gestures, sounds, words, or sentences sounds, words, or sentences to convey wants and needs or to convey wants and needs or to express meaning to to express meaning to others.”others.”

General ECI General ECI Administration: How it’s Administration: How it’s DoneDone

Administration ProceduresAdministration Procedures– Toy-play setting with familiar adultToy-play setting with familiar adult– 6 minute assessment sessions6 minute assessment sessions– Adult’s role is to facilitate play and Adult’s role is to facilitate play and

follow child’s leadfollow child’s lead– Set-up/clean-up/put awaySet-up/clean-up/put away

Alternate Toy FormsAlternate Toy Forms Observational Observational

Recording ProceduresRecording Procedures frequency of frequency of communicative behavior (key skill communicative behavior (key skill elements) tallied to get rate of total elements) tallied to get rate of total communicationcommunication

©2003 Juniper Gardens Children’s Project

Toy Form A: House

Toy Form B: Barn

Vocalizations Multiple WordsSingle Words

Total Communication

Gestures

Key Skill Elements

What does the ECI Measure?

Recording Recording ExampleExample

Expressive Communication ECI CODING SHEET

Gestures Vocalizations Single Word Utterances

Multi-Word Utterances

Total

:00

G

I

V

II

W

I

M

IIIIII

IIIII IIIII

1:00

G V W M

2:00

G V W M

3:00

G V W M

4:00

G V W M

5:00

G V W M

Child ID 502 Wave:_______8_____ Test Date:_________________

Assessor:_Annessa____

Coder: _Gabriel_________ Circle one: Barn / House

Location: _____Testing Room_________ Reliability Y N Primary Coder Name: ________________

How we know it works: Using How we know it works: Using the ECI for individual and the ECI for individual and program monitoringprogram monitoring

How we know it worksHow we know it works Does it grow?Does it grow? Is it reliable?Is it reliable? Is it valid?Is it valid? Is it predictive?Is it predictive? Individual progress monitoringIndividual progress monitoring Program evaluationProgram evaluation


Age Cohort (0-12)

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9

Ra

te p

er

Min

ute

Gestures Vocalizations Words MulWord

How Did Key Skill Elements Grow How Did Key Skill Elements Grow over 9 Months for Children in Their over 9 Months for Children in Their 1st Year?1st Year?


Age Cohort (13-24)

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9

Rat

e pe

r M

inut

e


How Did Key Skill Elements How Did Key Skill Elements Grow for Children in their 2Grow for Children in their 2ndnd

Year of Life?Year of Life?


Age Cohort (25-36)

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8 9

Monthly Occasions

Rat

e pe

r M

inut

e


How Did Key Skill Elements Grow For How Did Key Skill Elements Grow For Children in Their 3Children in Their 3rdrd Year of Life? Year of Life?


Did Total Communication Did Total Communication Grow and Differ by Age Grow and Differ by Age

Cohort?Cohort?

0

5

10

15

20

Month

1

Month

2

Month

3

Month

4

Month

5

Month

6

Month

7

Rat

e pe

r M

inut

e

0-1 years1-2 years2-3 years


Did we detect differences Did we detect differences for children with for children with disabilities?disabilities?

0

5

10

15

20

Month

1

Month

2

Month

3

Month

4

Month

5

Month

6

Month

7

Rat

e pe

r M

inut

e

DisabilityNo Disability


Does this approach work?Does this approach work?

Is it sensitive to growth?Is it sensitive to growth?Scores show change over time as children age and Scores show change over time as children age and develop develop 0 – 12 mos: M = 4.8; 13 – 24 mos. M = 8.3; 0 – 12 mos: M = 4.8; 13 – 24 mos. M = 8.3; 25 – 36 mos. M = 11.7 25 – 36 mos. M = 11.7

Is it reliable? Is it reliable? Trained observers agree on what communication Trained observers agree on what communication they record. (90% overall)they record. (90% overall)

Is it valid?Is it valid?ECI indicators correlate with standardized and ECI indicators correlate with standardized and parent report measures. (PLS; CCM)parent report measures. (PLS; CCM)

Is it predictive?Is it predictive?ECI indicators correlate with preschool literacy and ECI indicators correlate with preschool literacy and communication indicators.communication indicators.


How the ECI is used for How the ECI is used for Progress Monitoring: Progress Monitoring: Levels of Growth Levels of Growth Assessment Assessment

Bench Marking (quarterly – every 3 months)Bench Marking (quarterly – every 3 months)– All children assessedAll children assessed– Who is making or not making expected progress?Who is making or not making expected progress?– Who needs additional support?Who needs additional support?– How is the program doing?How is the program doing?

Strategic Monitoring (monthly)Strategic Monitoring (monthly)– Is more support accelerating growth? Is more support accelerating growth? – Is a change in intervention supports needed?Is a change in intervention supports needed?

Intensive Care Monitoring (weekly)Intensive Care Monitoring (weekly)– Is growth accelerating?Is growth accelerating?– Is a change in intervention supports needed?Is a change in intervention supports needed?


IGDIs ESAs

IGDI Decision Making IGDI Decision Making ModelModel

Monitor

Identify/Validate Need

For Intervention

Generate Intervention

Strategies

ImplementIntervention

EvaluateIntervention

Effectiveness

Monitor

Families

Outcomes©2003 Juniper Gardens Children’s Project

Framing Intervention Framing Intervention Solutions: Ruling out, Solutions: Ruling out, Ruling inRuling in

History with child?History with child? Medical issues?Medical issues? Family/caregiver concerns?Family/caregiver concerns? What child knows and does?What child knows and does? Settings, activities, routines, Settings, activities, routines,

curriculum?curriculum? Challenges to implementing Challenges to implementing

intervention?intervention?©2003 Juniper Gardens Children’s Project

IdentificationIdentification

0.0

5.0

10.0

15.0

20.0

25.0

30.0

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39

Age at Test

Wei

ghte

d C

omm

unic

atio

n R

ate

per

Min

ute

Child 1 Norm -1SD +1SD

Intervention


1 5 10 15 20 25 30 35 40 46

Weeks

0

10

20

30

40

50

60

Rat

e P

er M

inut

es

Communications per Minute Expected Growth

Expected Growth

The ECI Tells Us When to Intervene and The ECI Tells Us When to Intervene and Whether or Not Intervention Is Working Whether or Not Intervention Is Working or Program is Effectiveor Program is Effective

Baseline Intervention Modified Intervention


How Approach is Used to How Approach is Used to Inform Decision-Making at Inform Decision-Making at Individual LevelsIndividual Levels

Home visitors use individual child data Home visitors use individual child data to monitor child progressto monitor child progress

Use child data to evaluate Use child data to evaluate effectiveness of interventions and effectiveness of interventions and programprogram

Use child data to plan with parent(s) Use child data to plan with parent(s)


Effects of Individual Effects of Individual InterventionsInterventions

0

5

10

15

20

25

30

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

ECI Testings

To

tal C

om

mu

nic

ati

on

Rate

Identification Weekly Baseline Intervention(Child 207)Monthly


How Approach is Used to How Approach is Used to Inform Decision-Making at Inform Decision-Making at Program and State Levels Program and State Levels

ECI data available to State Head Start/Early ECI data available to State Head Start/Early Head Start Administrators, intervention staff Head Start Administrators, intervention staff and familiesand families

Program administrative staff, Early Head Program administrative staff, Early Head Start Coordinator and SRS use ECI for :Start Coordinator and SRS use ECI for :– Program EvaluationProgram Evaluation– Program PlanningProgram Planning– Quality ImprovementQuality Improvement– Inform Public and Funding Sources about EHS Inform Public and Funding Sources about EHS

ProgressProgress©2003 Juniper Gardens Children’s Project

Monitoring ProgramsMonitoring Programs Program data aggregated up from individual Program data aggregated up from individual

datadata Assess all or some children in a programAssess all or some children in a program Measure every 3 months: (6, 9, 12, etc.)Measure every 3 months: (6, 9, 12, etc.) Each child has a growth trajectory (rate of Each child has a growth trajectory (rate of

growth and mean level)growth and mean level) Calculate the average (or program) trajectory Calculate the average (or program) trajectory

(rate of growth and mean level)(rate of growth and mean level) Children below expected rate of growth - assess Children below expected rate of growth - assess

monthly and provide (ramp up) intervention.monthly and provide (ramp up) intervention. Continue monthly progress monitoring and Continue monthly progress monitoring and

continue intervention decision makingcontinue intervention decision making


Growth in Expressive Communication

-5

0

5

10

15

20

253 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39

Age at Measurement

Exp

ress

ive

Com

mun

icat

ion

Rat

e

Norm

P1

P2

P3

P1 = Growing at .31 responses per monthP2 = Growing at .51 responses per monthP3 = Growing at .60 responses per month


Program ProgressProgram Progress Overall Child Rate of GrowthOverall Child Rate of Growth

– Ave. Slope of Communications for individual Ave. Slope of Communications for individual children and groups of children in programschildren and groups of children in programs

– Mean Communications at different periods of Mean Communications at different periods of development for individuals and groups of children development for individuals and groups of children

Child Growth Rate Child Growth Rate – By GenderBy Gender– By IFSP StatusBy IFSP Status– By Language StatusBy Language Status

Staff OperationsStaff Operations– Staff Qualifications SummaryStaff Qualifications Summary– Interobserver Agreement SummaryInterobserver Agreement Summary

Better assessment can lead Better assessment can lead to improvements in to improvements in interventionintervention

Can give practitioners a focus for Can give practitioners a focus for their effortstheir efforts

Can help interventionists “see” Can help interventionists “see” when they’re making a differencewhen they’re making a difference

Can help practitioners know more Can help practitioners know more quickly when a change is necessaryquickly when a change is necessary

Can help directors understand when Can help directors understand when programs need improvementsprograms need improvements

SourcesSources Greenwood, C. R., Luze, G. J., & Carta, J. J. (2002). Assessment of intervention Greenwood, C. R., Luze, G. J., & Carta, J. J. (2002). Assessment of intervention

results with infants and toddlers IV. In A. Thomas, & J. Grimes (Eds.), results with infants and toddlers IV. In A. Thomas, & J. Grimes (Eds.), Best Best Practices in School PsychologyPractices in School Psychology (Vol. 2, pp. 1219-1230). Washington DC: National (Vol. 2, pp. 1219-1230). Washington DC: National Association of School Psychology.Association of School Psychology.

Greenwood, C. R., Luze, G. J., Cline, G., Kuntz, S., & Leitschuh, C. (2002). Greenwood, C. R., Luze, G. J., Cline, G., Kuntz, S., & Leitschuh, C. (2002). Developing a general outcome measure of growth in movement for infants and Developing a general outcome measure of growth in movement for infants and toddlers. toddlers. Topics in Early Childhood Special EducationTopics in Early Childhood Special Education..

Luze, G. J., Linebarger, D. L., Greenwood, C. R., Carta, J. J., Walker, D., Leitschuh, Luze, G. J., Linebarger, D. L., Greenwood, C. R., Carta, J. J., Walker, D., Leitschuh, C., & Atwater, J. B. (2001). Developing a general outcome measure of growth in C., & Atwater, J. B. (2001). Developing a general outcome measure of growth in expressive communication of infants and toddlers. expressive communication of infants and toddlers. School Psychology Review, School Psychology Review, 30(330(3), 383-406.), 383-406.

McConnell, S. R. (2000). Assessment in early intervention and early childhood McConnell, S. R. (2000). Assessment in early intervention and early childhood special education: Building on the past to project into the future. special education: Building on the past to project into the future. Topics in Early Topics in Early Childhood Special Education, 20Childhood Special Education, 20, 43-48., 43-48.

McConnell, S. R., Priest, J. S., Davis, S. D., & McEvoy, M. A. (2002). Best practices McConnell, S. R., Priest, J. S., Davis, S. D., & McEvoy, M. A. (2002). Best practices in measuring growth and development for preschool children. In A. Thomas, & J. in measuring growth and development for preschool children. In A. Thomas, & J. Grimes (Eds.), Grimes (Eds.), Best practices in school psychology IV.Best practices in school psychology IV. (Vol. 2, pp. 1219-1230). (Vol. 2, pp. 1219-1230). Washington DC: National Association of School Psychology.Washington DC: National Association of School Psychology.

Priest, J. S., McConnell, S. R., Walker, D., Carta, J. J., Kaminski, R. A., McEvoy, M. Priest, J. S., McConnell, S. R., Walker, D., Carta, J. J., Kaminski, R. A., McEvoy, M. A., Good, R. H., III, Greenwood, C. R., & Shinn, M. R. (2001). General growth A., Good, R. H., III, Greenwood, C. R., & Shinn, M. R. (2001). General growth outcomes for children: Developing a foundation for continuous progress outcomes for children: Developing a foundation for continuous progress measurement. measurement. Journal of Early Intervention, 24 (3), Journal of Early Intervention, 24 (3), 163-180163-180..

Carta, J., Greenwood, C. R., Walker, D., Kaminski, R., Good, R., McConnell, S. R., & McEvoy, M. (2002). Individual Growth and Development Indicators (IGDIs): Individual Growth and Development Indicators (IGDIs): Assessment that guides intervention for young children. In M. Ostrosky & E. Horn Assessment that guides intervention for young children. In M. Ostrosky & E. Horn (Eds.). (Eds.). Assessment: Gathering meaningful information. The Young Exceptional Assessment: Gathering meaningful information. The Young Exceptional Children Monograph SeriesChildren Monograph Series #4. Longmont, CO: Sopris West. #4. Longmont, CO: Sopris West.

Documents

Background of the Early Communication Indicator for Infants and Toddlers Dale Walker, Judith Carta, and Charles Greenwood