Embed Size (px)
Citation preview
This article was downloaded by: [University of Nebraska, Lincoln]On: 18 October 2014, At: 06:27Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK
Early Education andDevelopmentPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/heed20
The Evidence Base for MentalHealth Consultation in EarlyChildhood Settings: ResearchSynthesis Addressing Staff andProgram OutcomesEileen M. Brennan a , Jennifer R. Bradley b , MaryDallas Allen c & Deborah F. Perry da School of Social Work, Portland State Universityb J. Bradley & Associatesc School of Social Work, University of Alaska ,Anchoraged Bloomberg School of Public Health, Johns HopkinsUniversityPublished online: 12 Dec 2008.
To cite this article: Eileen M. Brennan , Jennifer R. Bradley , Mary Dallas Allen &Deborah F. Perry (2008) The Evidence Base for Mental Health Consultation in EarlyChildhood Settings: Research Synthesis Addressing Staff and Program Outcomes, EarlyEducation and Development, 19:6, 982-1022, DOI: 10.1080/10409280801975834
To link to this article: http://dx.doi.org/10.1080/10409280801975834
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,
or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.
This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone isexpressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATIONBRENNAN, BRADLEY, ALLEN, PERRY
The Evidence Base for Mental HealthConsultation in Early Childhood
Settings: Research Synthesis AddressingStaff and Program Outcomes
Eileen M. BrennanSchool of Social Work, Portland State University
Jennifer R. BradleyJ. Bradley & Associates
Mary Dallas AllenSchool of Social Work, University of Alaska, Anchorage
Deborah F. PerryBloomberg School of Public Health, Johns Hopkins University
Research Findings: One strategy to support early childhood providers’ work with chil-dren exhibiting challenging behavior is offering mental health consultation services inorder to build staff skills and confidence and reduce staff stress and turnover. Throughsystematic search procedures, 26 recent studies were identified that addressed the effec-tiveness of early childhood mental health consultation with respect to staff- and pro-gram-level outcomes. Across the reviewed studies, there is some evidence that earlychildhood mental health consultation helped increase staff self-efficacy/confidence andcompetence in dealing with troubling or difficult behaviors of young children in theircare. In several studies, staff receiving consultation had improved sensitivity and lowerjob-related stress. In addition, consultation generally helped improve overall quality ofearly care and education settings and was linked to reduced staff turnover. Practice or
EARLY EDUCATION AND DEVELOPMENT, 19(6), 982–1022Copyright © 2008 Taylor & Francis Group, LLCISSN: 1040-9289 print / 1556-6935 onlineDOI: 10.1080/10409280801975834
Correspondence regarding this article should be addressed to Eileen M. Brennan, Research andTraining Center on Family Support and Children’s Mental Health, Portland State University, P.O. Box751, Portland, OR 97207-0751. E-mail: [email protected]
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
Policy: Because the majority of the studies were not in peer-reviewed publications, thisresearch synthesis underscores the importance of increasing the rigor of future studies toprovide policymakers and practitioners with better evidence on consultation effective-ness. Priorities for future research include articulating the theory of change underlyingconsultation activities; developing additional measurement instruments to explore keycomponents of mental health consultation; and examining the effects of consultant quali-fications, consultation intensity, and specific activities on outcomes.
The majority of U.S. children younger than the age of 5 receive regular child carefrom nonrelatives or center-based programs (Capizzano & Adams, 2003). Theseearly childhood settings are crucial environments for nurturing young children’ssocial and emotional development (Denham, 2006; Lombardi, 2003). Innovativeways to improve the quality of early care and education are necessary to effec-tively use the time children spend in these settings (Roach, Riley, Adams, &Edie, 2005) and to promote their healthy social and emotional development. Re-search from the longitudinal study of child care funded by the National Instituteof Child Health and Human Development reported that longer hours in care wereassociated with increases in problem behavior for young children (NICHD EarlyChild Care Research Network, 2006). The increased time that young childrenspend in out-of-home early care and education settings and the increase in the to-tal number of children in care suggest that early care and education providersneed skills and resources to handle these challenging behaviors (Cohen &Kaufmann, 2000).
A variety of strategies have been used to help frontline staff manage and reducebehavior problems in young children in their care and promote healthy social andemotional development, as well as deal with the stressful nature of their work. Onesuch approach teams a mental health consultant with the child care provider in an on-going problem-solving and capacity-building relationship (Cohen & Kaufmann,2000; Donohue, Falk, & Provet, 2000; Johnston & Brinamen, 2006). Early child-hood mental health consultation “aims to build the capacity (improve the ability) ofstaff, families, programs, and systems to prevent, identify, treat, and reduce the im-pact of mental health problems among children from birth to age 6 and their fami-lies” (Cohen & Kaufmann, 2000, p. 4). It involves a collaborative relationship be-tween a professional consultant with mental health expertise and one or moreindividuals with other areas of expertise. By its very nature, this intervention is a ser-vice provided to the child care provider, not a therapeutic service delivered to thechild or family directly (Hepburn et al., 2007). These consultation services can haveas their eventual target a particular problem that is manifesting in children and/orfamilies or focus on an entire program or classroom. In the former, child- and fam-ily-focused consultation, the mental health consultant works through the child careprovider and with a child and/or family to address the specific behaviors of concernin an individual child or family. The latter, program-focused consultation, is in-
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 983
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
tended both to improve the overall quality of the classroom environment as well asto provide strategies to build staff capacity to address problematic behaviors orsystem problems that may be affecting one or more of the children, families, orstaff. In this article, we have chosen to focus our analyses on the proximal targetsof change in staff- and program-level outcomes; a companion paper (Perry, Allen,Brennan, & Bradley, 2008) reports on changes in child-level outcomes, as we be-lieve those changes are often mediated through gains in staff knowledge, skills,and behaviors.
When teachers lack the skills they need to cope with increasing numbers of chil-dren with behavior problems, the result may be to expel these young children. Astudy examining national preschool expulsion rates from 3,898 randomly selectedclassrooms across the United States found that pre-kindergarten expulsion rateswere higher than K–12 expulsion rates in 37 out 40 (92.5%) states that fundpre-kindergarten (Gilliam, 2005). Gilliam and Shahar (2006) found that the rate ofexpulsion among preschoolers in Massachusetts was more than 34 times the rate ofexpulsion during the K–12 years. In this study, 39.3% of teachers reported thatthey had expelled at least one child and 14.7% had suspended at least one child inthe past year. Expulsion was significantly related to group size and program struc-ture. In addition, teacher job stress and depressive symptoms were positively re-lated to expulsion, whereas teacher sense of satisfaction was negatively related toexpulsion. It is interesting that teachers who had access to a mental health consul-tant reported significantly lower rates of expelling children from state-fundedpre-kindergarten programs in the national study. The lowest expulsion rates wereassociated with onsite access to mental health professionals, and itinerant accesswas also related to fewer expulsions. Programs without access to mental healthconsultation services had the highest expulsion rates at 8.56 children per 1,000,whereas programs with regular access to a mental health consultant expelled 6.29children per 1,000 (Gilliam, 2005).
One way in which mental health consultation might affect expulsion rates is byincreasing the capacity of staff to cope with the stress of working with children whoare exhibiting difficult behaviors. Reducing stress can lead to better quality care andreduce the risk of turnover in staff (Helburn, 1995). Given that the quality of chil-dren’s relationships with child care providers has an impact on children’s sense ofself and of the world (Johnston & Brinamen, 2005), identifying effective strategiesthat enhance staff capacity is critical to increasing overall quality of care and pro-moting a stable caregiving environment that optimizes child development. Taken to-gether, the growing demand for child care services outside of the home and the highrates of children with behavior problems (Gross, Sambrook, & Fogg, 1999; Kuper-smidt, Bryant, & Willoughby, 2000; Qi & Kaiser, 2003; Raver & Knitzer, 2002;West, Denton, & Reaney, 2001) underscore the necessity of developing innovativeapproaches that reduce staff stress and turnover while building increased capacity torespond to children’s needs in a sensitive manner.
984 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
STAFF AND PROGRAM OUTCOMES AS A FOCUS OF STUDY
In deciding to focus on the staff- and program-level outcomes of mental health consul-tation, we explore the following theory of change: Consultation services are provideddirectly to early care and education providers in the context of a trusting relationship ofmutual respect. This permits changes in teachers’attitudes, skills, and behaviors, lead-ing to improvements in the emotional climate of the classroom, advances in classroommanagement practices, and a more intentional focus on socioemotional development.It is through these changes that enhanced child-level outcomes may result.
A growing number of states and communities have adopted mental health con-sultation as a strategy to build capacity in early childhood settings and address theconcerns of policymakers (Brennan, Bradley, Gettman, & Ama, 2006). Short-termfunding of initiatives and lack of support for systematic evaluation have preventedan extensive examination of the effectiveness of this promising intervention strat-egy. A variety of program evaluations and research studies have attempted to deter-mine the effectiveness of different early childhood mental health consultationmodels; however, few of these studies have been published to date. Given the pushfor states and communities to adopt proven strategies, there is a need for a criticalreview of the extent to which current data can support expanding the availability ofthis approach. In order to address this gap in the level of credible evidence, we un-dertook a research synthesis to ascertain the following:
1. How effective is mental health consultation in building staff capacity tomanage problem behaviors and to promote social and emotional develop-ment in young children?
2. What effects on the early childhood program are seen when a mental healthconsultant spends time working with teachers, children, and families?
The purpose of this article is to answer these research questions by reviewing find-ings from studies examining the effects of the wide array of mental health consul-tation models that have been implemented in early childhood settings and evaluat-ing the level of evidence for their effectiveness. In an effort to concentrate on theeffects of mental health consultation that are specific to early childhood care andeducation settings, this research review intentionally excludes studies from thewell-developed research base on mental health consultation in elementary, middle,and high schools (Berkowitz, 2001; Medway, 1979; Medway & Updyke, 1985;Reddy, 2000; Zins, 2007). Therefore, this article reviews findings specifically ad-dressing the effectiveness of mental health consultation in promoting improvedstaff- and program-level outcomes in early childhood settings. We chose staff andprogram outcomes as our focus because these are the primary targets of pro-gram-focused consultation, and they are also posited to be important mediators ofchange in children’s problem behaviors (Cohen & Kaufmann, 2000; Johnston &
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 985
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
Brinamen, 2006). We also expect that researchers and program evaluators who areattempting to determine the effectiveness of these approaches will find this reviewhelpful for identifying tools and strategies to use in assessing the effect that consul-tation may have on staff and program outcomes.
METHOD
The research review team conducted a comprehensive search of published andunpublished research on early childhood mental health consultation in twophases using established research synthesis procedures (Cook et al., 1992; Coo-per & Lindsay, 1998). We included unpublished articles in this review becausesuch scholarship as evaluation reports, dissertations, and monographs can in-volve high-quality science, although the authors may not have submitted them toa journal for publication (Cook et al., 1992). Additionally, we hoped to temperthe bias that can result from research syntheses focused solely on peer-reviewedarticles, given the tendency not to publish null findings (Greenwald, 1975;Sadish, 1992).
In the first phase of our search, we searched electronic databases—includingthe Educational Resources Information Center (ERIC), Dissertation Abstracts In-ternational, PsycINFO, Social Work Abstracts, and Social Services Abstracts—using combinations of key words, such as mental health, consultation, earlychildhood education, child care, and Head Start. Additional online early child-hood databases were searched, such as the Child Care and Early Education Re-search Connections Web site, the National Early Childhood Transition LiteratureDatabase, the National Child Care Information Library, and the Annotated Bibli-ography of Head Start Research. Finally, national organization, government, anduniversity websites—such as Zero to Three, the American Educational ResearchAssociation, the Head Start Bureau, the Research and Training Center on FamilySupport and Children’s Mental Health at Portland State University, and the Re-search and Training Center on Children’s Mental Health of the University of SouthFlorida—were searched for conference proceedings, bulletins, and reports onearly childhood mental health consultation.
In the second phase of the literature search, the research review team contacted ex-perts in the field of mental health consultation in order to uncover unpublished internaland external evaluations, program reports, and papers on mental health consultationprogram outcomes. These researchers contributed papers and reports and sharedknowledge of the most current investigations when the review team presented the pre-liminary findings at a national conference on early childhood mental health consulta-tion (Brennan, Bradley, Allen, Perry, & Tsega, 2005). Because of the scarcity of pub-lished research on early childhood mental health consultation, unpublished reports andresearch findings of limited circulation were included in the review.
986 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
Research Definition of Mental Health Consultation
For the purposes of this review, we have defined mental health consultation as aservice offered by providers with formal preparation and experience in children’smental health who collaborate with administrators, staff, and family members ofchildren from birth through 6 years of age participating in group care and early ed-ucation settings in order to build staff capacity to promote children’s social andemotional development and to address challenging behaviors. We use the termmental health for several reasons: First, we seek to focus attention on the specificskills, knowledge, and training that the consultant is bringing to the relationship;second, this term is familiar to early care and education providers, particularlyHead Start staff; and lastly, we want to distinguish this approach from one that isprimarily aimed at modifying behavior of individual children (such as Positive Be-havioral Support).
Early childhood mental health consultation activities can be focused on differentaspects of the early care and education environment: child- or family-level variables,or staff and program-level variables. Staff-level consultation may involve such activ-ities as providing training and support for staff regarding children’s social or emo-tional challenges, or providing classroom-based coaching and crisis intervention.Examples of program-level consultation include addressing staff wellness and com-munication issues, advising directors on program issues, and suggesting policychanges. In all cases, the consultation services are offered to the early care and edu-cation provider in the context of a trusting, supportive, ongoing relationship of mu-tual respect (Cohen & Kaufmann, 2000; Johnston & Brinamen, 2006).
In order to be included in this review, each study had to address an early child-hood mental health consultation program that provided services at at least two ofthe four mental health consultation levels, given the emphasis on consultation as acollaborative enterprise between the mental health consultant, the early childhoodstaff, and the family. By including only the studies of programs that provided men-tal health consultation services at two or more levels, we were able to identify andexclude the studies of those programs that provided mental health services to chil-dren and families without developing a collaborative relationship with the earlychildhood staff and family members. Collaboration between consultant, staff, andfamily members is an essential component of early childhood mental health con-sultation. Although this article is focused on staff and program outcomes, consul-tant efforts at the child and family levels can also build staff capacity by providingspecific and detailed information on children and families and putting supports inplace that can increase staff competencies and improve classroom environments.
Study Selection and Methods of Analysis
The research team established criteria to select the studies included in the review ofearly childhood mental health consultation. The first inclusion criterion was that
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 987
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
the research had to be empirical, using either quantitative, qualitative, or mixedmethods of research. Second, the research was required to examine mental healthconsultation according to our research definition. Therefore, research on healthconsultation and general early intervention services was excluded. Third, the liter-ature review targeted early childhood mental health consultation, so it only in-cluded studies of mental health consultation with early childhood care and educa-tion programs serving children birth to 6 years old. Studies of consultation toelementary schools were excluded from the review, with the exception of earlycare and education programs providing after-school and vacation services forschool age children up to 6 years of age. Fourth, only research that addressed staffand program outcomes was included in this review. Finally, in order to be consid-ered for inclusion, the studies must have been completed between 1985 and 2008.
A qualitative data analysis approach was used to conduct the research synthesis.The research team was unable to perform a statistically based meta-analysis of theresearch studies found, because few studies had the necessary sample size andcomparison groups to calculate an effect size of the intervention. Instead the studyused established systematic review procedures (Cooper & Lindsay, 1998; Mullen& Ramírez, 2006) in order to produce a written synthesis of the results found in theresearch and evaluation reports.
Each study identified in the search was read by one of the review team membersand screened for suitability at a meeting of three investigators. For those articlesthat met the eligibility criteria, an extensive matrix was constructed through a con-tent analysis that identified data elements in each of the studies. These data ele-ments included distinguishing characteristics, consultation features, consultantqualifications, funding sources, methodological factors, measures used, and majorstudy outcomes. After at least two team members read each study and reviewed thematrix, all studies were assigned to a classification type, and tables were con-structed to assist in answering the major research questions of the review. Three re-search team members reviewed the tables for accuracy and consistency. Finally,findings were synthesized according to categories that were responsive to the ma-jor research questions, and subcategories were constructed within the researchquestion categories.
Summary of Studies Included
Fifty-one studies were initially identified through the literature search. Twenty-five investigations did not meet the inclusion criteria: 5 studies were excluded fornot meeting the age criteria, 8 studies were excluded for not specifically examiningmental health consultation, and 12 studies were excluded because they addressedonly child or other outcomes, not staff and program outcomes. The 26 studiesmeeting our criteria were classified into four groups according to the type of de-sign they employed and are summarized in Table 1. The first group consisted of
988 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
989
TAB
LE1
Stu
dies
ofM
enta
lHea
lthC
onsu
ltatio
nin
Ear
lyC
hild
hood
Set
tings
:Con
sulta
tion
Type
s,Le
vels
and
Inte
nsity
,Stu
dyS
ampl
ean
dD
esig
n,an
dS
taff
and
Pro
gram
Out
com
es
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
es
Type
1—E
valu
atio
nsus
ing
rand
omiz
edco
ntro
lled
tria
ldes
igns
Gill
iam
(200
7)T
heE
arly
Chi
ldho
odC
onsu
ltatio
nPa
rtne
rshi
p,ba
sed
inth
est
ate
ofC
onne
ctic
ut,h
ad10
mas
ter’
s-le
vel
trai
ned
MH
Cs
prov
idin
g8
wee
ksof
child
-an
dst
aff-
base
dse
rvic
es,
incl
udin
g4–
6hr
incl
assr
oom
spe
rw
eek.
Dat
aw
ere
colle
cted
inch
ildca
rece
nter
san
dH
ead
Star
tpro
gram
sfr
om2
coho
rts
from
Janu
ary
2005
toJu
ne20
05an
dfr
omSe
ptem
ber
2005
toM
arch
2006
.EC
MH
Cle
vels
:C,F
,S,P
.
Ran
dom
ized
cont
rolle
dde
sign
with
43tr
eatm
enta
nd42
wai
t-lis
tcon
trol
clas
sroo
ms.
Teac
hers
wer
epr
edom
inan
tlyfe
mal
e(9
6%)
and
Eur
o-A
mer
ican
(79%
);th
eyco
mpl
eted
the
SBP,
CC
WJS
I,C
ES-
D,a
ndPM
scal
esas
wel
las
child
beha
vior
mea
sure
s.E
CE
RS-
Ran
dC
ISra
tings
byob
serv
ers
blin
dto
trea
tmen
tsta
tus.
Alth
ough
trea
tmen
tand
cont
rol
clas
sroo
ms
show
edsi
gnif
ican
teff
ects
for
MH
Cin
terv
entio
non
child
beha
vior
s,th
ere
wer
eno
sign
ific
ant
diff
eren
ces
betw
een
trea
tmen
tand
cont
rols
taff
onth
eSB
P,C
CW
JSI,
CE
S-D
,or
PMsc
ales
.EC
ER
S-R
and
CIS
ratin
gsal
sodi
dno
tdif
fer
betw
een
clas
sroo
mgr
oups
.
Rav
eret
al.(
inpr
ess)
,pe
erre
view
edT
heC
hica
goSc
hool
Rea
dine
ssPr
ojec
tha
dsi
te-b
ased
MSW
cons
ulta
nts
deliv
era
man
ualiz
edco
nsul
ting
mod
elfo
cuse
don
trai
ning
and
supp
ortiv
ecl
assr
oom
coac
hing
.Par
ticip
atin
gH
ead
Star
tcla
ssro
oms
in7
low
-inc
ome
neig
hbor
hood
sw
ere
offe
red
30hr
ofte
ache
rtr
aini
ngan
dre
ceiv
edM
=13
2hr
ofin
divi
dual
-an
dst
aff-
leve
lco
nsul
tatio
nse
rvic
es.C
ontr
ols
rece
ived
addi
tiona
lcla
ssro
omai
de.C
hild
ren:
67%
Afr
ican
Am
eric
an,2
6%H
ispa
nic.
EC
MH
Cle
vels
:C,S
.
Of
94pa
rtic
ipat
ing
teac
hers
,65
(69%
)fi
lled
outs
urve
yson
dem
ogra
phic
char
acte
rist
ics.
The
staf
fm
irro
red
the
dive
rsity
ofch
ildre
n,w
ith70
%A
fric
anA
mer
ican
,20%
Lat
ina,
and
10%
Eur
o-A
mer
ican
.Clu
ster
edra
ndom
ized
cont
rolle
dde
sign
sort
ed35
clas
sroo
ms
into
18in
terv
entio
nan
d17
wai
t-lis
tco
ntro
lcoh
orts
;cla
ssro
omqu
ality
rate
dus
ing
the
EC
ER
S-R
and
clas
sroo
mcl
imat
era
ted
onth
eC
LA
SSby
obse
rver
sbl
ind
totr
eatm
ents
tatu
s.
Eve
naf
ter
base
line
cova
riat
esw
ere
cont
rolle
d,tr
eatm
entc
lass
room
ssh
owed
asi
gnif
ican
tlygr
eate
rin
crea
sein
posi
tive
clas
sroo
mcl
imat
e(d
=.8
9),
decr
ease
inne
gativ
ecl
imat
e(d
=.6
4),
and
incr
ease
inte
ache
rse
nsiti
vity
(d=
.53)
over
cont
rols
.Tea
cher
sre
ceiv
ing
MH
Cal
soha
dsi
gnif
ican
tim
prov
emen
tin
thei
rm
anag
emen
tof
disr
uptiv
ebe
havi
orco
mpa
red
with
cont
rols
(d=
.52)
.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
990
Type
2—E
valu
atio
nsus
ing
non-
rand
omly
assi
gned
com
pari
son
grou
psfo
rch
ildan
d/or
staf
fou
tcom
es
Alk
onet
al.(
2003
),pe
erre
view
edE
valu
atio
nof
serv
ices
give
nth
roug
hE
CM
HI
inSa
nFr
anci
sco
met
ropo
litan
area
in19
97–1
999.
Supp
ortiv
eco
nsul
ting
rela
tions
hips
betw
een
MH
staf
ffr
om4
agen
cies
and
staf
fat
25ur
ban
child
care
cent
ers
serv
ing
child
ren
betw
een
2an
d5
year
sof
age.
Inte
nsity
vari
ed,w
ithco
nsul
tant
sse
rvin
gbe
twee
n5
and
20ce
nter
s.E
CM
HC
leve
ls:C
,F,S
,P.
TO
Ste
ache
rra
tings
and
EC
ER
Ssc
ores
com
pare
dat
two
times
1ye
arap
art.
GA
San
dC
AS
(dev
elop
edfo
rst
udy)
give
nto
cent
erdi
rect
ors
and
teac
hers
atT
2.St
aff
inte
rvie
ws,
focu
sgr
oups
.n=
25di
rect
ors,
n=
83st
aff
(T1)
;n=
25di
rect
ors,
n=
54st
aff
(T2)
.Tea
cher
set
hnic
ally
dive
rse:
39%
Asi
anA
mer
ican
,24%
Afr
ican
Am
eric
an,
19%
Eur
o-A
mer
ican
,12%
Lat
ino.
14ce
nter
sne
wly
rece
ivin
gM
HC
com
pare
dw
ith9
cent
ers
rece
ivin
gM
HC
for
atle
ast1
year
.
Posi
tive
incr
ease
inT
OS
scor
eson
self
-eff
icac
yfr
omT
1to
T2
(t=
–4.6
,p<
.05)
,pos
itive
ratin
gson
GA
Sat
T2.
Freq
uenc
yof
MH
Cw
asre
late
dto
low
erst
aff
turn
over
(r=
–.43
,p<
.05)
and
chan
ges
ince
nter
qual
ity(r
=.4
4,p
<.0
5).C
hang
esin
cent
erqu
ality
pred
icte
dby
dura
tion
ofM
HC
(yea
rs)
and
mea
nG
AS
(R2
=.5
8,p
<.0
5).
Posi
tive
chan
ges
ince
nter
qual
ityal
sore
late
dto
high
erte
ache
rco
mpe
tenc
yat
T2
(t=
2.4,
p<
.05)
.Qua
litat
ive
anal
ysis
show
edgr
eate
rem
path
yfo
ran
dre
cogn
ition
ofm
eani
ngof
child
ren’
sdi
ffic
ultb
ehav
ior
atT
2;im
prov
emen
tin
cent
erco
mm
unic
atio
n,te
amw
ork.
Ble
ecke
r&
Sher
woo
d(2
003)
Con
sulta
tion
prov
ided
aspa
rtof
onsi
teco
mpr
ehen
sive
MH
serv
ices
by8
MH
agen
cies
to75
cent
ers
and
100
fam
ilych
ildca
repr
ovid
ers
inSa
nFr
anci
sco
duri
ng20
01–2
002
thro
ugh
Hig
hQ
ualit
yC
hild
Car
eIn
itiat
ive.
Indi
vidu
al-
and
prog
ram
-lev
elco
nsul
tatio
n,ac
tiviti
esva
ryin
gac
ross
site
s.A
ppro
xim
atel
y5,
000
ethn
ical
lydi
vers
ech
ildre
nag
edbi
rth
thro
ugh
5ye
ars
atsi
tes.
EC
MH
Cle
vels
:C,F
,S,P
.
CC
OS
com
plet
edby
teac
hers
atba
selin
ein
fall
and
follo
w-u
pin
spri
ng.
EC
ER
S-R
adm
inis
tere
dto
rand
omly
sele
cted
sam
ple
ofce
nter
s(n
=24
)at
follo
w-u
pin
spri
ng.N
=50
head
teac
hers
atsa
mpl
edce
nter
sfi
lled
out
SBP.
Targ
etch
ildre
n(n
=21
4)co
mpa
red
with
no-t
reat
men
tchi
ldre
n(n
=89
)at
cent
ers.
No
sign
ific
antd
iffe
renc
esin
CC
OS
betw
een
base
line
and
follo
w-u
p;st
aff
with
long
erte
nure
had
high
erse
lf-c
ompe
tenc
yra
tings
.90%
ofce
nter
sin
sam
ple
had
EC
ER
S-R
tota
lra
tings
ofgo
odor
exce
llent
.On
SBP,
32%
focu
sed
onso
cioe
mot
iona
lde
velo
pmen
t,9%
focu
sed
onac
adem
icre
adin
ess,
50%
wer
e“m
ixed
mod
el.”
SBP
scor
esw
ere
notr
elat
edto
EC
ER
S-R
orC
CO
Site
ms.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
es
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
991
Ble
ecke
r&
Sher
woo
d(2
004)
San
Fran
cisc
oH
igh
Qua
lity
Chi
ldC
are
Initi
ativ
eus
edco
nsul
tant
sfr
om8
MH
agen
cies
tose
rve
atle
ast7
5ce
nter
-bas
edpr
ogra
ms
and
100
fam
ilych
ildca
repr
ovid
ers
in20
02–2
003.
App
roxi
mat
ely
5,00
0cu
ltura
llydi
vers
ech
ildre
nat
site
sse
rved
.Con
sulta
nts
spen
t8–1
0hr
atce
nter
s,pe
rfor
min
gbo
thch
ild-
and
prog
ram
-foc
used
cons
ulta
tion,
and
refe
rral
s.E
CM
HC
leve
ls:C
,F,S
,P.
TO
Sdi
stri
bute
dto
allt
each
ers
atsi
tes
serv
ed.N
=34
2te
ache
rsu
rvey
sre
turn
edfr
om78
cent
ers
atba
selin
e(f
all2
002)
,N=
323
from
77ce
nter
sat
follo
w-u
p(s
prin
g,20
03).
CIS
ratin
gsof
34ra
ndom
lyse
lect
edce
nter
s:n
=34
infa
ll,n
=30
insp
ring
.Con
sulta
nts
repo
rted
type
ofro
les
and
inte
rven
tions
used
.Chi
ldou
tcom
esco
mpa
red
with
cent
erch
ildre
nno
trec
eivi
ngM
Hse
rvic
es.
Teac
hers
rate
dse
lves
asle
asts
kille
dw
orki
ngw
ithch
ildre
nha
ving
MH
prob
lem
sor
sign
ific
antd
isab
ilitie
s.Sk
illra
tings
did
notc
hang
eov
ertim
e,an
dw
ere
notr
elat
edto
cons
ulta
ntco
ntac
t.M
HC
repo
rted
help
fuli
nal
lar
eas;
help
fuln
ess
sign
ific
antly
corr
elat
edw
ithM
HC
cont
act.
CIS
ratin
gsha
dsi
gnif
ican
tinc
reas
esin
teac
her
sens
itivi
tyfr
ompr
e-to
post
test
(t=
–2.2
6,p
>.0
5),d
etac
hmen
t,pe
rmis
sive
ness
,and
hars
hnes
sde
crea
sed.
MH
Cin
terv
entio
nsva
ried
byce
nter
type
.B
leec
ker
etal
.(20
05)
In20
03–2
004,
San
Fran
cisc
oH
igh
Qua
lity
Chi
ldC
are
Initi
ativ
eus
edco
nsul
tant
sin
11M
Hag
enci
esto
serv
e10
7ch
ildca
rece
nter
sw
ith96
7st
aff,
150
fam
ilych
ildca
repr
ovid
ers,
and
11ho
mel
ess
shel
ters
.Ato
talo
f4,
737
cultu
rally
dive
rse
child
ren
rece
ived
both
dire
ctan
din
dire
ctse
rvic
es,
incl
udin
gch
ildan
dpr
ogra
mco
nsul
tatio
nan
dre
ferr
als.
EC
MH
Cle
vels
:C,F
,S,P
.
TO
Sdi
stri
bute
din
spri
ngto
site
s;n
=39
9te
ache
rsfr
om95
site
sre
turn
edsu
rvey
s.n
=29
4E
nglis
h,n
=40
Span
ish,
and
n=
65C
hine
sesu
rvey
sre
turn
ed.
Sem
istr
uctu
red
qual
itativ
ein
terv
iew
sco
nduc
ted
with
18ce
nter
dire
ctor
san
d3
fam
ilych
ildca
repr
ovid
ers.
Chi
ldou
tcom
esco
mpa
red
with
cent
erch
ildre
nno
trec
eivi
ngse
rvic
es.
86%
ofst
aff
rate
dM
HC
ashe
lpfu
lon
all
5T
OS
item
s,in
clud
ing
assi
stan
cew
ithch
ild,s
taff
MH
issu
es.H
elpf
ulne
ssra
tings
corr
elat
edw
ithqu
ality
ofst
aff–
pare
ntre
latio
nshi
p(r
=.1
1,p
<.0
5),c
aree
rsa
tisfa
ctio
n(r
=–.
19,p
<.0
01),
and
plan
sto
cont
inue
inch
ildca
reca
reer
s(r
=–.
19,p
<.0
01).
Qua
litat
ive
data
supp
orte
dth
eus
eful
ness
ofM
HC
child
-an
dfa
mily
-foc
used
cons
ulta
tion,
asw
ella
sst
aff
supp
orts
,cri
sis
inte
rven
tion,
refe
rral
san
dad
voca
cy,s
ugge
sted
prog
ram
impr
ovem
ents
,and
brid
ging
cultu
ralg
aps.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
992
Bow
man
Ass
ocia
tes
&K
agan
(200
3)E
valu
atio
nof
serv
ices
give
nth
roug
hE
arly
Chi
ldho
odM
enta
lHea
lthC
onsu
ltatio
nPr
ogra
min
2002
.In
volv
ed40
child
care
cent
ers
in4
coun
ties
inSa
nFr
anci
sco
Bay
area
serv
ing
ethn
ical
lydi
vers
ein
fant
sth
roug
hpr
esch
oole
rs.C
onsu
lting
atpr
ogra
man
dch
ildle
vels
(wor
ked
with
fam
ilies
,chi
ldre
n,st
aff,
man
ager
s).
EC
MH
Cle
vels
:C,F
,S,P
.
TO
S,G
AS,
and
CE
Fco
mpl
eted
atye
aren
dby
n=
135
ethn
ical
lydi
vers
ete
ache
rs:3
4%L
atin
o,30
%E
uro-
Am
eric
an,1
8%A
sian
Am
eric
an,
14%
Afr
ican
Am
eric
an.E
CE
RS-
Ran
dC
ISra
tings
thro
ugh
obse
rvat
ion
atn
=20
cent
ers.
Com
pari
son
n=
21ce
nter
sw
ithM
HC
oflo
nger
dura
tion
(23
mon
ths
orm
ore)
,with
n=
19ce
nter
sof
shor
ter
dura
tion
(les
sth
an23
mon
ths)
.
Impr
ovem
ents
inte
ache
rpr
e/po
stra
tings
ofco
mpe
tenc
yat
year
end
com
pare
dw
ithre
tros
pect
ive
com
pete
ncy
prio
rto
cons
ulta
tion
for
TO
S(p
=.0
29)
and
GA
S(p
<.0
5).G
ood
qual
ityin
stud
yce
nter
s(E
CE
RS-
RM
=4.
96),
with
nodi
ffer
ence
sbe
twee
nce
nter
sw
ithsh
ort
and
long
MH
Cdu
ratio
nin
EC
ER
S-R
orC
ISsc
ores
.Hig
her
cent
erqu
ality
asso
ciat
edw
ithgr
eate
rte
ache
rse
nsiti
vity
(r=
.75,
p<
.05)
.Dir
ecto
rs,
teac
hers
gave
high
cons
ulta
ntef
fect
iven
ess
ratin
gs.
Hen
niga
net
al.(
2004
)T
FKIn
itiat
ive
inC
entr
alM
assa
chus
etts
used
MH
cons
ulta
nts
aspa
rt-t
ime
staf
fw
hode
liver
edin
divi
dual
inte
rven
tions
,cl
assr
oom
staf
fas
sist
ance
,and
staf
fan
dpa
rent
trai
ning
.Mea
nex
tent
ofin
divi
dual
serv
ice
was
22hr
deliv
ered
over
4–6
mon
ths.
Chi
ldre
nw
ere
ethn
ical
lyan
dlin
guis
tical
lydi
vers
e;ab
out3
4.7%
had
emot
iona
lor
beha
vior
alpr
oble
ms.
EC
MH
Cle
vels
:C
,F,S
.
InY
ear
1,6
pres
choo
lpilo
tsite
sre
ceiv
edM
HC
;in
Yea
r2,
2ne
win
terv
entio
nsi
tes
wer
ead
ded;
com
pari
son
betw
een
pilo
tand
new
site
s.40
teac
hers
,mos
tlyE
uro-
Am
eric
an,c
ompl
eted
surv
eys
rega
rdin
gkn
owle
dge
ofE
Cbe
havi
oral
issu
esan
dpe
rson
albu
rnou
tusi
ngth
eM
BI.
Focu
sgr
oups
held
with
teac
hers
and
inte
rvie
ws
with
MH
cons
ulta
nts.
Cla
ssro
oms
with
TFK
inte
rven
tion
had
decr
ease
sin
child
susp
ensi
onan
dex
puls
ion
rate
s.Pi
lots
ites
with
2ye
ars
ofM
HC
had
sign
ific
antly
low
erle
vels
ofbe
havi
orpr
oble
ms
inth
eir
clas
sroo
ms
inY
ear
2th
anin
Yea
r1
orin
new
inte
rven
tion
site
s.M
HC
foun
dte
ache
rm
otiv
atio
nch
alle
ngin
g,w
ithso
me
havi
nglit
tlein
vest
men
tin
wor
king
with
diff
icul
tchi
ldre
n.Te
ache
rsre
port
edim
prov
edab
ility
tow
ork
with
child
ren
and
fam
ilies
,but
lack
oftim
eto
deal
with
MH
Can
dch
ildbe
havi
orpr
oble
mat
ic.P
erso
nal
burn
outh
ighe
rin
new
than
pilo
tsite
s.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
es
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
993
Leh
man
etal
.(20
05)
SAM
HSA
-fun
ded
prev
entio
npr
ogra
min
Port
land
,OR
,tar
getin
gch
ildre
n,pa
rent
s,an
dce
nter
staf
f.O
nsite
cons
ulta
tion
(10–
20hr
per
wee
k)at
4ch
ildca
rece
nter
sse
rvin
gch
ildre
n2–
6ye
ars,
and
use
ofIn
cred
ible
Yea
rspa
rent
trai
ning
seri
esin
inte
rven
tion
grou
p.E
CM
HC
leve
ls:C
,F,S
,P.
Inte
rven
tion
grou
p(n
=49
pare
nts;
42ch
ildre
n)re
ceiv
edco
nsul
tatio
npl
us12
-wee
kIn
cred
ible
Yea
rspa
rent
trai
ning
seri
es.C
ompa
riso
ngr
oup
rece
ived
site
-bas
edco
nsul
tatio
non
ly(n
=38
pare
nts;
38ch
ildre
n).S
elf-
repo
rtm
easu
res
ofte
ache
rst
ress
and
com
pete
nce
atba
selin
e(n
=10
),6
mon
ths
(n=
26),
18m
onth
s(n
=19
),an
d30
mon
ths
(n=
13).
Ann
ual
dire
ctor
inte
rvie
ws.
Con
sulti
ngac
tiviti
esre
port
.
Mos
tfre
quen
tcon
sulti
ngac
tiviti
esw
ere
clas
sroo
mob
serv
atio
nan
ddi
rect
child
-lev
elin
terv
entio
n.Im
prov
emen
tin
mea
nab
ility
ofte
ache
rsto
iden
tify
conc
erni
ngbe
havi
ors,
mak
ere
ferr
als,
and
man
age
chal
leng
ing
beha
vior
s;t
test
sdi
dno
trea
chsi
gnif
ican
ce.O
vera
llte
ache
rst
ress
leve
lsfl
uctu
ated
.
Olm
os&
Gri
mm
er(2
004)
PEA
RL
cons
ulta
tion,
prev
entio
n,an
din
terv
entio
npr
ogra
min
Den
ver,
CO
.C
onsu
ltatio
nw
ithte
ache
rs3
hr/w
eek.
202
child
ren
serv
edin
8ch
ildca
re/H
ead
Star
tset
tings
.Chi
ldre
n39
%H
ispa
nic,
52%
pove
rty
leve
l,61
%m
ale.
EC
MH
Cle
vels
:C,F
,S.
N=
174
trea
tmen
tchi
ldre
ntr
acke
dfo
r3
serv
ice
year
s;co
mpa
riso
nw
ith15
8ch
ildre
nno
tser
ved
due
tolo
wer
DE
CA
scor
es;u
sed
regr
essi
on-d
isco
ntin
uity
anal
ysis
.Chi
ldre
nm
easu
red
atbe
ginn
ing
and
end
ofsc
hool
year
usin
gth
eC
CA
Ran
dD
EC
A.M
ultip
lete
ache
rsan
dad
min
istr
ator
sin
terv
iew
edan
dpa
rtic
ipat
edin
focu
sgr
oups
.
Inin
terv
iew
san
dfo
cus
grou
ps,s
taff
invo
lved
with
PEA
RL
stat
edth
eyfe
ltm
ore
com
pete
ntad
dres
sing
the
soci
alan
dem
otio
naln
eeds
ofch
ildre
n,re
port
edlo
wer
leve
lsof
stre
ss.T
each
ers
stat
edth
eyfe
lta
“sen
seof
relie
f,”fe
ltm
ore
posi
tivel
yab
outc
hild
ren,
and
held
high
erex
pect
atio
nsof
child
ren’
spo
tent
ial.
The
yal
sore
port
edle
arni
ngsk
ills/
stra
tegi
esth
atin
crea
sed
thei
ref
fect
iven
ess
and
havi
nggr
eate
ref
fica
cy.B
urno
utan
dtu
rnov
erw
ere
redu
ced
inse
tting
sse
rved
.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
994
Shel
ton
etal
.(20
02)
Proj
ectM
aste
rypr
ovid
edin
divi
dual
ized
beha
vior
plan
sfo
rN
orth
Car
olin
aH
ead
Star
tpre
scho
oler
sag
ed3
or4
who
wer
eid
entif
ied
asha
ving
beha
vior
aldi
ffic
ultie
s.O
nsite
MH
Cw
asal
sopr
ovid
edas
part
ofa
10-w
eek
syst
emof
care
inte
rven
tion
that
feat
ured
teac
her
trai
ning
,cla
ssro
om-b
ased
and
pare
ntbe
havi
orm
anag
emen
t,fa
mily
supp
ort,
and
coor
dina
ted
serv
ices
.Fam
ilies
wer
elo
win
com
e,an
d82
%w
ere
Afr
ican
Am
eric
an.E
CM
HC
leve
ls:C
,F,S
.
Of
41ch
ildre
nin
the
stud
y,n
=28
child
ren
wer
eas
sign
edto
inte
rven
tion
and
n=
13to
the
asse
ssm
ent-
only
grou
p.A
llch
ildre
nex
ceed
edth
e93
rdpe
rcen
tile
for
gend
eran
dag
eei
ther
for
inat
tent
ion,
hype
ract
ivity
,or
aggr
essi
vebe
havi
or.S
taff
com
plet
edT
SQin
pre-
and
post
test
surv
ey.C
lass
room
sw
ere
asse
ssed
usin
gth
eE
CE
RS-
R.
Pre-
/pos
t-m
easu
res
ofch
ildre
n’s
stre
ngth
san
dne
eds
and
pare
ntin
gan
dfa
mily
supp
orts
wer
eco
mpa
red
for
inte
rven
tion
and
cont
rols
.
Att
hepo
st-m
easu
re,t
each
ers
inin
terv
entio
ncl
assr
oom
sre
port
edm
ore
conf
iden
ce(p
<.0
05),
wer
em
ore
likel
yto
prom
ote
pare
ntin
volv
emen
t(p
<.0
1),w
ere
mor
elik
ely
toof
fer
advi
ceto
fam
ilies
onpa
rent
ing
skill
s(p
<.0
04),
and
wer
em
ore
likel
yto
use
met
hods
topr
omot
ebe
havi
oral
com
pete
nce
onth
epa
rtof
child
ren
(e.g
.,cl
ass
ince
ntiv
es,
rew
ards
for
acce
ptab
lebe
havi
or,
com
men
ton
posi
tive
beha
vior
;p<
.01)
.
Type
3—E
valu
atio
nsw
ithou
tcom
pari
son
grou
ps
Ear
lyH
ead
Star
tN
atio
nalR
esou
rce
Cen
ter
(200
4)
Infa
ntM
enta
lHea
lthIn
itiat
ive
(Pat
hway
sto
Prev
entio
n)in
24E
arly
Hea
dSt
art/M
igra
ntan
dSe
ason
alH
ead
Star
t.In
fant
MH
cons
ulta
ntas
sign
edto
wor
kw
ithpr
ogra
mte
amof
coor
dina
tor,
2st
aff,
pare
nt,a
ndlo
calM
Hco
nsul
tant
.G
oalt
oim
prov
epr
ogra
mca
paci
tyin
infa
ntM
H.E
CM
HC
leve
ls:S
,P.
Self
-rep
ortm
ails
urve
yof
24te
amle
ader
s,48
team
mem
bers
,24
pare
nts,
24lo
calM
Hco
nsul
tant
san
d21
infa
ntM
Hco
nsul
tant
s.A
dmin
iste
red
at6
and
12m
onth
saf
ter
impl
emen
tatio
nas
part
ofev
alua
tion
ofth
epr
ogra
m.T
1,n
=10
6;T
2,n
=82
.
Eva
luat
ion
iden
tifie
dne
edfo
rm
ore
time
for
team
build
ing;
acco
mpl
ishe
dpr
ogra
mgo
als
wer
em
ainl
yin
area
ofkn
owle
dge
deve
lopm
ent.
Cha
lleng
esid
entif
ied
incl
uded
sust
aina
bilit
y;ge
ogra
phic
aldi
stan
ceof
infa
ntM
Hco
nsul
tatio
n;la
ckof
clar
ityab
out
role
s/re
spon
sibi
litie
s,in
clud
ing
loca
lM
Hco
nsul
tant
and
infa
ntM
Hco
nsul
tatio
n;in
adeq
uate
know
ledg
eof
the
infa
ntM
Hco
nsul
tant
abou
tthe
prog
ram
and
clie
nts
serv
ed;l
imite
din
volv
emen
tof
pare
nts
inth
epr
ogra
m.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
e
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
995
Fiel
det
al.(
2004
)C
arin
gfo
rK
ids
Initi
ativ
e.SA
MH
SA-f
unde
dm
odel
ofM
Hco
nsul
tatio
nto
prov
ide
prev
entio
nan
dea
rly
inte
rven
tion
inco
mm
unity
child
care
cent
ers
inur
ban
Det
roit,
MI.
Prog
ram
prov
ided
1,01
9se
rvic
eho
urs
to47
fam
ilies
in9
pilo
tsite
sse
rvin
get
hnic
ally
dive
rse
child
ren
aged
0–5.
Prog
ram
goal
sin
clud
edre
gula
rco
nsul
tatio
nan
dca
paci
tybu
ildin
gof
staf
fan
dpa
rent
sus
ing
rela
tions
hip-
base
dco
nsul
tatio
n,tr
aini
ng,r
efle
ctiv
esu
perv
isio
n,an
dSt
eps
toSe
lf-D
eter
min
atio
ncu
rric
ulum
.EC
MH
Cle
vels
:C,F
,S.
Mul
timet
hod
prog
ram
eval
uatio
nus
ing
inte
rvie
ws,
obse
rvat
ions
,sur
veys
at3
ofth
e9
site
s(n
=64
staf
f).S
ervi
ceda
tafr
omal
lsite
s.St
aff
98%
fem
ale,
47%
Afr
ican
Am
eric
an,4
4%C
auca
sian
,9%
othe
r.10
%of
staf
fha
da
bach
elor
’sde
gree
,7%
had
CD
A,3
0%ha
dso
me
colle
gecr
edit.
No
sign
ific
antc
hang
ein
pret
est(
n=
64)
and
post
test
scor
es(n
=39
)on
TO
S-R
.G
AS
com
plet
edby
7di
rect
ors
indi
cate
dpo
sitiv
evi
ewof
cons
ulta
tion.
Con
sulta
ntsu
rvey
(n=
8)in
dica
ted
that
they
felt
thei
rse
rvic
esw
ere
valu
ed.
Car
egiv
ersu
rvey
(n=
32)
indi
cate
din
crea
sed
staf
faw
aren
ess
ofth
eir
impo
rtan
ceto
child
ren
inth
eir
care
.No
sign
ific
antc
hang
ein
pre-
and
post
test
scor
eson
adap
ted
SDC
S.Q
ualit
ativ
ere
sults
:im
port
ance
ofre
latio
nshi
psbe
twee
nco
nsul
tant
,dir
ecto
r,an
dst
aff
and
ofsu
ppor
ting
staf
fw
ell-
bein
g.
Fiel
d&
Mac
krai
n(2
004)
Mic
higa
nC
hild
Car
eE
xpul
sion
prev
entio
npr
ojec
thad
child
ren
from
138
child
care
setti
ngs
with
8,27
6ch
ildre
nag
edbi
rth
thro
ugh
5ye
ars
refe
rred
to6
serv
ice
site
s.To
tals
of2,
484
prov
ider
and
954
fam
ilyco
nsul
tatio
nsw
ere
give
nov
era
1-ye
arpe
riod
,inc
ludi
ngdi
rect
serv
ices
to22
4fa
mili
es,p
hone
cons
ulta
tions
with
staf
f,re
ferr
als,
prov
isio
nof
reso
urce
s,an
dtr
aini
ngs.
EC
MH
Cle
vels
:C,F
,S.
Car
epr
ovid
ers
com
plet
edch
ildbe
havi
orre
port
sat
entr
yan
dco
mpl
etio
nof
serv
ices
.Sta
ffan
dfa
mily
mem
bers
wer
esu
rvey
edre
gard
ing
prog
ram
outc
omes
and
satis
fact
ion.
N=
34ca
regi
vers
resp
onde
dto
surv
ey.
Onl
y8
of21
3ch
ildre
nw
hoha
ddi
rect
inte
rven
tions
wer
eex
pelle
d(4
%ex
puls
ion
rate
).M
ean
over
all
satis
fact
ion
with
cons
ulta
tion
serv
ices
was
3.9
on4-
poin
tsca
leha
ving
16ite
ms.
94%
agre
edth
atth
eyle
arne
dne
wst
rate
gies
tohe
lpch
ildre
nw
ithch
alle
nges
and
that
they
had
abe
tter
unde
rsta
ndin
gof
the
child
’sbe
havi
or.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
996
Gou
ld(2
003)
Con
sulta
tion
aspa
rtof
Con
solid
ated
Chi
ldC
are
Pilo
tMen
talH
ealth
min
i-gr
ants
(ave
ragi
ng$1
0,00
0)to
12ch
ildca
repr
ogra
ms
inC
olor
ado.
Goa
lto
addr
ess
emot
iona
land
beha
vior
alpr
oble
ms
inyo
ung
child
ren.
Impl
emen
ted
wid
era
nge
ofin
itiat
ives
.73
5te
ache
rsin
the
part
icip
atin
gpr
ogra
ms
rece
ived
cons
ulta
tion
and/
ortr
aini
ng.P
rogr
ams
serv
edto
talo
f68
2ch
ildre
ndi
rect
ly,5
,789
child
ren
indi
rect
ly;5
31fa
mili
espa
rtic
ipat
ed.
EC
MH
Cle
vels
:C,F
,S,P
.
Mul
tisite
inte
rven
tion
eval
uatio
n(2
001–
2002
,Yea
r2)
ofm
ini-
gran
tre
cipi
ents
.Inc
lude
dqu
antit
ativ
epr
ogra
min
form
atio
nda
ta,g
oal
achi
evem
ent(
self
-rep
orte
dby
prog
ram
),an
dqu
alita
tive
data
(les
sons
lear
ned,
unm
etne
eds)
.Dat
are
port
ing
vari
edac
ross
prog
ram
s.
Som
eev
iden
ceof
redu
ced
staf
ftu
rnov
er.
Eva
luat
ion
repo
rtno
ted
high
dem
and
for
earl
yin
terv
entio
nM
Hse
rvic
esan
dfo
rst
aff
with
know
ledg
eof
both
EC
and
men
talw
elln
ess.
Nee
dfo
rst
aff
trai
ning
inas
sess
men
tand
indo
cum
entin
gou
tcom
es.S
taff
turn
over
incr
ease
sne
edfo
rre
peat
edtr
aini
ng.
Staf
fco
ncer
nsin
clud
edla
ckof
fund
ing,
impa
ctof
dise
nrol
lmen
ton
child
,sta
ffbu
rnou
tand
poor
empl
oym
ent
cond
ition
s,po
orqu
ality
,bar
rier
sto
serv
ices
for
som
efa
mili
es(e
.g.,
noin
sura
nce,
notc
ultu
rally
appr
opri
ate)
.H
utch
ison
(198
6)C
onsu
ltatio
nas
part
of14
-mon
thde
mon
stra
tion
proj
ect(
fund
edby
U.S
.D
HH
S)in
64H
ead
Star
tcla
ssro
oms
in23
site
s(s
ervi
ng96
0fa
mili
esin
Los
Ang
eles
,CA
).64
%m
onol
ingu
alSp
anis
h.N
=19
9ch
ildre
nre
ferr
edfo
rM
Hse
rvic
es.M
inim
umof
4.5
hrof
obse
rvat
ion
and
cons
ulta
tion
for
each
refe
rral
.Ser
vice
spr
ovid
edby
exte
rnal
agen
cies
for
staf
f(c
onsu
ltatio
nan
dtr
aini
ng),
pare
nts
(edu
catio
nan
din
volv
emen
t),a
ndch
ildre
n(M
Htr
eatm
ent)
.Sta
ffre
ceiv
edto
talo
f1,
156
hrof
cons
ulta
tion
and
trai
ning
.E
CM
HC
leve
ls:C
,F,S
.
Nee
dsas
sess
men
tand
prog
ram
eval
uatio
n.M
ainl
yde
scri
ptiv
eda
ta.
Fam
ilyne
eds
asse
ssm
ents
urve
yT
1(n
=84
5)an
dT
2(n
=22
0;se
lect
edsa
mpl
e).C
lass
room
obse
rvat
ions
ofch
ildre
nan
dst
aff.
Cas
ere
view
.Tea
cher
perf
orm
ance
and
clas
sroo
men
viro
nmen
trat
ings
byM
Hst
aff
(n=
64).
Dat
apr
ovid
edba
sis
for
cons
ulta
tion
and
trai
ning
.Eva
luat
ion
ofpr
ogra
mby
staf
fan
dby
pare
nts.
89%
ofte
ache
rs(n
=75
)ra
ted
cons
ulta
tion/
trai
ning
proc
ess
ashe
lpfu
l(5
8%re
spon
sera
te).
Serv
ices
mos
tva
lued
byte
ache
rsw
ere
supp
orta
ndtr
aini
ngto
deal
with
clas
sroo
mpr
oble
ms.
Rep
orte
dpr
oble
ms
incl
uded
inad
equa
tetim
ean
dco
mm
unic
atio
n,an
dla
ckof
hom
evi
sits
.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
e
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
997
Lan
gkam
p(2
003)
Con
sulta
tion
aspa
rtof
EC
MH
Iin
pres
choo
land
hom
ese
tting
sin
38co
untie
sin
Ohi
o.A
ppro
ache
san
din
tens
ityto
cons
ulta
tion
diff
ered
acro
ss39
prog
ram
s(r
ange
=le
ssth
anon
cea
mon
thto
twic
ea
wee
k).N
=77
1id
entif
ied
child
ren
in9-
mon
thpe
riod
;65
%m
ale;
n=
598
≥36
mon
ths,
n=
135
≤35
mon
ths;
50%
repo
rted
asha
ving
exte
rnal
izin
gpr
oble
ms,
14%
inte
rnal
izin
g.D
irec
tcon
sulta
tion
prov
ided
for
n=
544
child
ren,
n=
172
rece
ived
dire
cttr
eatm
ent,
n=
263
wer
ere
ferr
edou
tfor
addi
tiona
lser
vice
s.H
ome
visi
tsw
ithn
=34
fam
ilies
.E
CM
HC
leve
ls:C
,F,S
,P.
Mul
timet
hod
prog
ram
eval
uatio
n.Pr
etes
tev
alua
tions
of37
1ch
ildre
n,su
rvey
of37
MH
cons
ulta
nts,
tele
phon
esu
rvey
of39
dire
ctor
s.Pa
rent
and
child
asse
ssm
entf
or34
fam
ilies
atT
1an
dT
2(4
–6m
onth
sla
ter)
.Inc
lude
dm
aile
dsu
rvey
(22-
item
MB
I)to
com
pare
teac
her
outc
omes
(job
satis
fact
ion
and
burn
out)
atm
atch
edsi
tes
with
and
with
outE
CM
HI
fund
ing
usin
gca
seco
ntro
ldes
ign.
Tele
phon
esu
rvey
foun
dth
at67
%of
dire
ctor
sre
port
edth
atco
nsul
tatio
nse
rvic
esin
clud
edpr
even
tion
and
inte
rven
tion/
refe
rral
.Maj
ority
ofdi
rect
ors
(69%
–92%
)re
port
edpo
sitiv
eim
pact
ofco
nsul
tatio
non
prog
ram
(e.g
.,im
prov
edqu
ality
,low
erst
aff
stre
ss).
Onl
y28
%of
dire
ctor
sre
port
edth
atco
nsul
tatio
nha
dim
pact
onst
aff
turn
over
.Sta
ffbu
rnou
tsur
vey
foun
dno
sign
ific
antd
iffe
renc
esin
MB
Isc
ores
ofst
aff
atsi
tes
with
orw
ithou
tMH
cons
ulta
tion
(res
pons
era
teof
74.6
%fo
r1,
420
surv
eys
mai
led)
.
Paw
l&Jo
hnst
on(1
991)
Con
sulta
tion
to37
child
care
cent
ers
inSa
nFr
anci
sco,
CA
,are
ase
rvin
get
hnic
ally
dive
rse
child
ren
from
infa
nts
to5-
year
-old
s.Pr
ogra
man
dca
seco
nsul
tatio
n,re
ferr
alse
rvic
es,d
irec
tw
ork
with
fam
ilies
and
prov
ider
s,in
-ser
vice
trai
ning
s.2,
780
hrof
dire
ctse
rvic
esin
prio
rye
ar.E
CM
HC
leve
ls:
C,F
,S,P
.
N=
221
care
give
rs;4
1%E
uro-
Am
eric
an,
25%
Asi
anA
mer
ican
;20%
Afr
ican
Am
eric
an;1
2%L
atin
o;2%
othe
r.C
hild
care
qual
ityra
ted.
Prog
ram
cons
ulta
tion
goal
sat
tain
men
tmea
sure
don
5-po
int
scal
eat
3,6,
9,12
mon
ths.
Pre-
and
post
-mea
sure
sof
care
give
rin
tera
ctio
nw
ithch
ildan
dpa
rent
s.
Of
prog
ram
goal
s63
%w
ere
orga
niza
tiona
ldif
ficu
lties
,29%
wer
epr
ogra
mm
atic
issu
es;b
oth
show
edga
ins
at3,
6,an
d9
mon
ths,
butg
reat
est
gain
sat
12m
onth
s.C
ase
cons
ulta
tion
goal
sw
ere
35%
indi
vidu
alch
ildre
n,17
%pr
ovid
er–p
aren
trel
atio
ns,3
0%di
rect
wor
kw
ithpa
rent
s;ch
ildga
ins
grea
test
betw
een
3an
d6
mon
ths,
othe
rm
eans
also
incr
ease
dw
ithtim
e.Si
gnif
ican
tim
prov
emen
tin
care
give
r–ch
ild(t
=10
.9,p
<.0
01)
and
care
give
r–pa
rent
inte
ract
ion
(t=
1.8,
p<
.10)
over
time.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
998
Perr
y,D
unne
,eta
l.(2
008)
,pee
rre
view
ed4-
year
(BE
ST)
initi
ativ
epr
ovid
ing
child
-foc
used
cons
ulta
tion
targ
etin
gch
ildre
n(n
=19
2)at
risk
for
expu
lsio
nfr
omlic
ense
dan
dre
gula
ted
child
care
setti
ngs
insu
burb
anM
aryl
and.
Con
sulta
tion
prov
ided
by2
full-
time
mas
ter’
s-le
velp
rofe
ssio
nals
with
EC
MH
expe
rtis
e;5–
10on
site
obse
rvat
ions
last
ing
upto
2hr
,tra
ckin
gfr
eque
ncy
ofta
rget
beha
vior
sin
10-m
inin
terv
als;
deve
lopm
ento
fin
divi
dual
ized
plan
;and
cons
ulta
tion
with
child
care
prov
ider
and
fam
ily.
EC
MH
Cle
vels
:C,F
,S.
Prog
ram
eval
uatio
n.Pr
etes
t/pos
ttest
desi
gnto
mea
sure
child
outc
omes
(sta
ndar
dize
dag
e-ap
prop
riat
ech
ildbe
havi
orm
easu
res
com
plet
edby
child
care
prov
ider
s).6
0%of
prog
ram
refe
rral
sex
hibi
ted
aggr
essi
vebe
havi
ors.
78%
wer
ebo
ys.5
7%of
child
ren
lived
with
both
pare
nts.
26%
lived
with
mot
her
only
.77%
Cau
casi
an,
15%
Afr
ican
Am
eric
an,2
%H
ispa
nic.
Ran
dom
sam
ple
of(n
=20
)ch
ildca
repr
ovid
ers
inte
rvie
wed
byte
leph
one
mid
way
thro
ugh
4-ye
arpr
ojec
t.E
xpul
sion
rate
str
acke
d.
Of
the
child
care
prov
ider
sin
terv
iew
ed,
the
maj
ority
(80%
)fo
und
the
prog
ram
help
fuli
nbu
ildin
gth
eir
skill
san
dm
odel
ing
appr
opri
ate
resp
onse
s.A
min
ority
ofth
epr
ovid
ers
(mai
nly
fam
ilych
ildca
reho
me
prov
ider
s)re
port
edne
gativ
ere
spon
seto
cons
ulta
tion
beca
use
ofpe
rcei
ved
clas
hbe
twee
nph
iloso
phy/
appr
oach
and
sugg
estio
nsof
beha
vior
alsp
ecia
lists
.
Saff
ord
etal
.(20
01)
Con
sulta
tion
aspa
rtof
Cuy
ahog
aC
ount
y,O
H,E
arly
Chi
ldho
odIn
itiat
ive
esta
blis
hed
in19
97(D
ayC
are
Plus
Serv
ices
)se
rvin
gch
ildre
nof
0–6
year
san
dfa
mili
esin
32ch
ildca
rece
nter
s.Pr
ogra
min
clud
edcr
isis
inte
rven
tion
and
prev
entio
nac
tiviti
es.E
CM
HC
leve
ls:C
,F,S
,P.
Mul
timet
hod
prog
ram
eval
uatio
nin
clud
edon
site
obse
rvat
ions
usin
gE
CE
RS
asan
obse
rvat
iona
ltoo
land
“war
ning
sign
chec
klis
t,”pa
rtic
ipan
tobs
erva
tion
ofm
eetin
gs,d
ocum
enta
naly
sis,
inte
rvie
ws
and
focu
sgr
oups
with
care
give
rs,c
onsu
ltant
san
dfa
mili
es.
Inte
rvie
ws
with
n=
32ce
nter
dire
ctor
s.M
embe
rch
ecki
ngus
edto
valid
ate
qual
itativ
ein
terp
reta
tions
.
Con
sulta
ntac
tiviti
esin
clud
edch
ild-l
evel
inte
rven
tion,
refe
rral
san
dst
aff
trai
ning
,re
ferr
als.
Qua
litat
ive
data
onch
alle
nges
expe
rien
ced
ince
nter
s.Id
entif
ied
need
for
spec
ific
qual
ityim
prov
emen
ts(p
rogr
amst
ruct
ure,
envi
ronm
ent)
.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
e
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
999
Tym
inks
i(20
01)
1999
–200
1ev
alua
tion
ofth
eSa
nFr
anci
sco
Hig
hQ
ualit
yC
hild
Car
eM
enta
lHea
lthC
onsu
ltatio
nIn
itiat
ive
in44
child
care
cent
ers
inSa
nFr
anci
sco,
CA
,ser
ving
fam
ilies
ofdi
vers
era
ces/
ethn
iciti
es.6
6%of
child
ren
wer
em
ale.
57%
Eng
lish
spea
king
.Ave
rage
leng
thof
cons
ulta
ntac
tivity
was
34m
onth
s.V
arie
dm
odel
sof
cons
ultin
g.E
CM
HC
leve
ls:C
,F,S
,P.
Prog
ram
eval
uatio
nus
ing
pret
est/p
ostte
stde
sign
.Pre
test
self
-rep
ortm
easu
reof
cent
erph
iloso
phy
(SB
P),f
ield
obse
rvat
ions
tora
tepr
ogra
mqu
ality
over
2-ye
arpe
riod
(EC
ER
S),p
re-
and
post
test
staf
fjo
bsa
tisfa
ctio
nse
lf-r
epor
tm
easu
re(d
esig
ned
for
stud
y),
cons
ulta
ntra
tings
ofch
ildou
tcom
es(V
inel
and
soci
aliz
atio
ndo
mai
n).
Sam
ple
size
vari
edac
ross
mea
sure
s.A
ppro
xim
atel
yn
=95
staf
f,n
=97
child
ren.
Cen
ter
Philo
soph
y(S
BP)
of46
clas
sroo
ms
incl
uded
11%
acad
emic
,39
%ch
ildin
itiat
ed(s
ocio
emot
iona
lem
phas
is),
and
50%
mix
ed.N
osi
gnif
ican
tdif
fere
nces
betw
een
T1
and
T2
(9m
onth
sla
ter)
onjo
bsa
tisfa
ctio
nm
easu
re(d
evel
oped
for
stud
y)bu
tpo
ssib
lece
iling
effe
ct.P
rete
stE
CE
RS-
Rsc
ore
=5.
68.P
ostte
stE
CE
RS-
Rsc
ore
=5.
35.E
CE
RS
data
colle
cted
over
2-ye
arpe
riod
due
tolo
gist
ical
prob
lem
s.
Type
4—D
escr
iptiv
eor
corr
elat
iona
lstu
dies
Bre
nnan
etal
.(20
03)
Chi
ld-,
fam
ily-,
and
prog
ram
-lev
elco
nsul
tatio
nof
fere
dby
staf
fan
dco
ntra
cted
MH
cons
ulta
nts
at9
child
care
cent
ers
loca
ted
in6
stat
es.C
ente
rsse
lect
edby
nom
inat
ion
for
succ
essf
ulin
clus
ion
ofch
ildre
nw
ithbe
havi
oral
chal
leng
es.M
HC
vari
edin
inte
nsity
:10
%to
full
time.
Serv
ed32
–1,5
00ch
ildre
n.E
CM
HC
leve
ls:C
,F,S
,P.
Cen
ters
vari
edw
idel
yin
degr
eeof
ethn
icdi
vers
ity,a
ges
ofch
ildre
nse
rved
,re
gion
,and
loca
tion
type
(urb
an,
subu
rban
,or
rura
l).I
nten
sive
inte
rvie
ws
cond
ucte
dw
ithn
=9
dire
ctor
s,n
=40
staf
f,n
=39
fam
ilym
embe
rs;n
=25
one-
hour
child
–sta
ffob
serv
atio
nson
site
attr
ansi
tion
times
atth
ece
nter
s.M
embe
rch
ecki
ngus
ed.
Staf
fan
dad
min
istr
ator
sre
port
edth
atco
nsul
tant
spr
ovid
edst
aff
supp
ort,
incr
easi
ngth
eir
conf
iden
cein
hand
ling
child
beha
vior
,and
assi
sted
with
staf
fre
latio
nshi
ps.S
taff
–con
sulta
ntre
latio
nshi
pbu
ilton
mut
ualt
rust
.C
onsu
ltant
ssu
pplie
dpa
rtof
the
trai
ning
and
tech
nica
lass
ista
nce
for
staf
f,he
lpin
gst
aff
tom
ore
effe
ctiv
ely
care
for
child
ren
inan
incl
usiv
ese
tting
,an
dso
met
imes
inte
rven
eddi
rect
lyin
the
clas
sroo
m.
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
1000
Elia
s(2
004)
Stud
yof
head
teac
hers
inco
mm
unity
-bas
edpr
esch
oolp
rogr
ams
inC
onne
ctic
utw
ith80
%E
uro-
Am
eric
an,2
0%H
ispa
nic
popu
latio
n.N
osp
ecif
icde
fini
tion
ofM
HC
was
used
;76%
ofre
spon
dent
sre
port
edth
atM
HC
was
avai
labl
eon
anon
-cal
l/as-
need
edba
sis.
EC
MH
Cle
vels
:C,F
,S.
Cro
ss-s
ectio
nals
elf-
repo
rtsu
rvey
cond
ucte
dw
ithn
=42
head
teac
hers
inpr
esch
oolp
rogr
ams
Sam
ple
sele
cted
thro
ugh
mul
tista
gecl
uste
rsa
mpl
ing.
Pres
choo
lMen
talH
ealth
Con
sulta
tion
Que
stio
nnai
reha
dite
ms
onav
aila
bilit
yof
MH
Cse
rvic
es,b
ehav
ior
prob
lem
sne
edin
gco
nsul
tatio
n,at
titud
esto
war
dM
HC
.
46%
ofte
ache
rssa
idm
ore
stud
ents
need
edse
rvic
esth
anre
ceiv
edth
em;
only
41%
agre
edth
atM
Hco
nsul
tatio
nse
rvic
esw
ere
adeq
uate
toad
dres
sco
ncer
ns.F
orse
rvic
esre
ceiv
ed,9
8%ag
reed
MH
Che
lped
inun
ders
tand
ing
and
deve
lopi
ngso
lutio
nsto
beha
vior
prob
lem
s,93
%sa
idth
atM
HC
prom
oted
wel
l-be
ing
for
staf
fan
dch
ildre
n.86
%ag
reed
MH
cons
ulta
nts
help
edte
ache
rsto
talk
topa
rent
s79
%st
ated
that
MH
Che
lped
prom
ote
cultu
rald
iver
sity
.G
reen
,Eve
rhar
t,et
al.
(200
4)N
atio
nals
urve
yof
staf
ffr
omH
ead
Star
tpr
ogra
ms;
child
-an
dpr
ogra
m-f
ocus
edM
Hco
nsul
tatio
nan
dM
Hse
rvic
esof
fere
dw
ithin
prog
ram
.32%
ofpr
ogra
ms
wer
epr
edom
inan
tlyE
uro-
Am
eric
an.3
6%se
rved
mor
eth
an40
%A
fric
anA
mer
ican
fam
ilies
.22%
serv
edm
ore
than
40%
His
pani
cfa
mili
es.E
CM
HC
leve
ls:C
,F,S
,P.
Sam
ple
draw
nfr
om79
part
icip
atin
gH
ead
Star
tpro
gram
s;n
=80
2st
aff
mem
bers
from
rand
omsa
mpl
eof
staf
flis
ts;n
=15
4fa
mily
mem
bers
,3fr
omea
chof
62pr
ogra
ms.
Cro
ss-s
ectio
nals
elf-
repo
rtm
aile
dsu
rvey
used
;com
pari
son
ofpr
ogra
ms
with
high
vs.l
owin
tegr
atio
nof
MH
cons
ulta
nts
into
prog
ram
stru
ctur
ean
dcl
assr
oom
s.
67%
ofst
aff
and
85%
ofad
min
istr
ator
sra
ted
the
MH
Cas
supp
ortin
gst
aff
todo
thei
rjo
bsbe
tter.
Dir
ectr
elat
ions
hip
betw
een
prog
ram
-foc
used
cons
ulta
tion
and
staf
fw
elln
ess,
and
inte
grat
ion
ofM
Hco
nsul
tant
and
team
and
staf
fw
elln
ess.
Prog
ram
sw
ithhi
ghle
vels
ofM
HC
inte
grat
ion
had
sign
ific
antly
low
ertim
esbe
twee
nre
ferr
alan
das
sess
men
t.
TAB
LE1
(Con
tinue
d)
Stud
yA
utho
r(s)
and
Dat
eE
CP
rogr
aman
dM
HC
Des
crip
tion
Sam
ple
and
Des
ign
Staf
fand
Pro
gram
Out
com
e
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
1001
Gre
enet
al.(
2006
),pe
erre
view
edSu
bsam
ple
from
natio
nals
urve
yof
Hea
dSt
artp
rogr
ams
(Gre
en,E
verh
art,
etal
.,20
04)
incl
uded
134
adm
inis
trat
ors,
447
staf
f,an
d74
MH
Cs.
Con
sulta
nts
pred
omin
antly
fem
ale
(72%
),W
hite
(74%
);se
rved
M=
231
child
ren.
EC
MH
Cle
vels
:C,F
,S,P
.
Freq
uenc
yof
prog
ram
-an
din
divi
dual
-lev
elco
nsul
tatio
nac
tiviti
esre
port
ed,a
ndqu
ality
ofst
aff
rela
tions
hips
mea
sure
d.H
LM
anal
ysis
exam
ined
cons
ulta
nts
with
inpr
ogra
ms
and
look
edat
outc
ome
mea
sure
sof
help
fuln
ess
and
staf
fw
elln
ess.
Prog
ram
-lev
elac
tiviti
esm
ore
freq
uent
than
indi
vidu
al-l
evel
cons
ulta
tion.
Mor
efr
eque
ntco
nsul
tatio
nw
aslin
ked
tost
aff
perc
eptio
nof
grea
ter
MH
Che
lpfu
lnes
sto
child
ren
and
ofgr
eate
rst
aff
wel
lnes
s.T
heef
fect
offr
eque
ncy
was
med
iate
dby
the
qual
ityof
staf
f–co
nsul
tant
rela
tions
hips
.G
reen
,Sim
pson
,eta
l.(2
004)
,pee
rre
view
edQ
ualit
ativ
est
udy
ofM
HC
atth
ein
divi
dual
,fam
ily,c
lass
room
,and
prog
ram
leve
lsin
3Pa
cifi
cN
orth
wes
tH
ead
Star
ts(u
rban
,sub
urba
n,an
dru
ral)
.Con
sulta
tion
inte
nsity
vari
edfr
om7%
to32
0%of
afu
ll-tim
epo
sitio
nse
rvin
gbe
twee
n25
0–55
0ch
ildre
n.E
CM
HC
leve
ls:C
,F,S
,P.
In-p
erso
nse
mis
truc
ture
din
terv
iew
sof
n=
59st
aff
in5
site
sw
ithin
3pr
ogra
ms.
N=
16pr
ogra
mad
min
istr
ator
s,n
=8
MH
cons
ulta
nts,
n=
8fa
mily
serv
ice
advo
cate
s,n
=13
teac
hers
,n=
14te
ache
r’s
assi
stan
ts.S
taff
wer
e88
%E
uro-
Am
eric
an,8
%H
ispa
nic,
and
4%ot
her.
Goo
dre
latio
nshi
psw
ithM
HC
sre
late
dto
mor
ein
tegr
ated
appr
oach
esto
MH
.St
aff
repo
rtin
gM
Hco
nsul
tant
sto
bem
ore
invo
lved
also
wer
em
ore
likel
yto
repo
rta
clea
ran
dco
nsis
tent
prog
ram
philo
soph
y.St
aff
inpr
ogra
ms
with
high
MH
Cin
volv
emen
tals
ore
port
edgr
eate
rM
Hpr
ogra
mef
fect
iven
ess
and
had
MH
appr
oach
cons
iste
ntw
ithac
cept
edpr
actic
es.
Not
e:E
C=
earl
ych
ildho
od;M
HC
=m
enta
lhea
lthco
nsul
tatio
n/co
nsul
tant
;EC
MH
C=
earl
ych
ildho
odm
enta
lhea
lthco
nsul
tatio
n;C
=ch
ild;F
=fa
mily
;S=
staf
f;P
=pr
ogra
m;S
BP
=Su
rvey
ofB
elie
fsan
dPr
actic
es;C
CW
JSI=
Chi
ldC
are
Wor
kerJ
obSt
ress
Inve
ntor
y;C
ES-
D=
Cen
terf
orE
pide
mio
logi
cSt
udie
s–D
epre
ssio
nsc
ale;
PM=
Pare
ntal
Mod
erni
tySc
ale
ofC
hild
rear
ing
and
Edu
catio
nB
elie
fs;E
CE
RS-
R=
Ear
lyC
hild
hood
Env
iron
men
tRat
ing
Scal
e–R
evis
ed;C
IS=
Car
egiv
erIn
tera
ctio
nSc
ale;
MSW
=m
aste
r’s
inso
cial
wor
k;C
LA
SS=
Cla
ssro
omA
sses
smen
tSco
ring
Syst
em;M
H=
men
talh
ealth
;TO
S=
Teac
her
Opi
nion
Surv
ey;G
AS
=G
oalA
ttain
-m
entS
calin
g;C
AS
=C
onsu
ltant
Act
ivity
Surv
ey;T
2=
Tim
e2;
T1
=T
ime
1;C
CO
S=
Chi
ldC
are
Opi
nion
Surv
ey;C
EF
=C
onsu
lting
Eva
luat
ion
Form
;TFK
=To
geth
erfo
rK
ids;
MB
I=
Mas
lach
Bur
nout
Inve
ntor
y;SA
MH
SA=
Subs
tanc
eA
buse
and
Men
talH
ealth
Serv
ices
Adm
inis
trat
ion;
PEA
RL
=Pa
rent
/Pro
vide
rE
ffec
tiven
ess
inE
arly
Lea
rnin
gE
nvir
onm
ents
;DE
CA
=D
ever
eux
Ear
lyC
hild
hood
Ass
essm
ent;
CC
AR
=C
olor
ado
Clie
ntA
sses
smen
tRec
ord;
TSQ
=Te
ache
rStr
ateg
yQ
uest
ionn
aire
;C
DA
=C
hild
Dev
elop
men
tAss
ocia
tecr
eden
tial;
TO
S-R
=Te
ache
rO
pini
onSu
rvey
–Rev
ised
;SD
CS
=Se
lf-D
eter
min
atio
nC
areg
iver
sSc
ale;
DH
HS
=D
epar
tmen
tof
Hea
lthan
dH
uman
Serv
ices
;E
CM
HI
=E
arly
Chi
ldho
odM
enta
lH
ealth
Initi
ativ
e;B
EST
=B
ehav
iora
lE
mot
iona
lSu
ppor
tan
dT
rain
ing;
HL
M=
hier
arch
ical
linea
rm
odel
ing.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
two studies using randomized control designs. The second group was composed ofnine investigations, each of which included an intervention and a comparisongroup; usually children receiving treatment were compared to those in a no-treat-ment condition. The third set of 10 studies were evaluations using quasi-experi-mental or even pre-experimental pretest–posttest designs without comparisongroups. Finally, the fourth group of 5 studies were descriptive or correlational indesign.
Sample sizes in the investigations ranged from 20 to 802 staff members, with amedian staff sample of 64. Programs generally served urban children from 2through 5 years of age who were ethnically and linguistically diverse. However,the diversity of staff members varied among programs, and several programs re-ported challenges the staff experienced in serving families having cultural back-grounds different from their own. A high proportion of the children in the earlychildhood programs were identified as being from low-income families.
The studies included in this review utilized a variety of different standardizedmeasurement instruments to collect quantitative program and staff data. These aresummarized in Table 2, which provides instrument names, acronyms, authors,psychometric characteristics, and a list of studies using each of the measures. Themajority of studies also collected qualitative data from interviews or focus groups,and some employed instruments specifically designed to capture program or staffinformation.
RESULTS
Type and Quality of Evidence
All of the investigations included in this review and reported in Table 1 had clearlystated research questions and gave specific information regarding staff participantsand mental health consultation (MHC) involved in the related programs. Addi-tionally, most of the quantitative studies used one or more of the established mea-sures reported in Table 2. However, only the two Type 1 studies used randomizedcontrolled trials (RCT) designs (Gilliam, 2007; Raver et al., in press). In contrastType 2 (evaluations with non-randomly assigned comparison groups) and Type 3(evaluations without comparison groups) studies used designs that limited internalvalidity, and therefore conclusions reached regarding staff and program outcomesof MHC must be viewed with caution. Type 4 investigations (correlational or de-scriptive studies) used established survey methods or met qualitative standards(Denzin & Lincoln, 2000) that increased the trustworthiness of their conclusionsregarding MHC, including the use of peer reviewers and member checking. Eachof the four types of studies had at least one published paper which had been sub-jected to peer review (Type 1: Raver et al., in press; Type 2: Alkon, Ramler, &
1002 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
1003
TAB
LE2
Inde
xof
Sta
ff-B
ased
Mea
sure
sU
sed
inR
evie
wed
Men
talH
ealth
Con
sulta
tion
Stu
dies
Acr
onym
Ful
lNam
ean
dTy
peof
Mea
sure
Aut
hor
and
Dat
e;P
sych
omet
rics
Stud
y
CC
AR
Col
orad
oC
lient
Ass
essm
entR
ecor
d:C
linic
alas
sess
men
tfor
trea
tmen
tpl
anni
ngan
dst
atew
ide
asse
ssm
ent
Elli
set
al.(
1991
).In
terr
ater
relia
bilit
y:90
%of
rate
rs’s
core
sw
ere
with
in2
poin
tsof
the
dom
ain
mea
n.L
ower
relia
bilit
yfo
rne
wer
item
s(C
olor
ado
Div
isio
nof
Men
talH
ealth
,200
6).
Olm
os&
Gri
mm
er(2
004)
CC
OS
Chi
ldC
are
Opi
nion
Surv
ey:T
each
ers
repo
rtth
eir
opin
ion
ofth
ech
ildca
reen
viro
nmen
t
Tym
insk
i(20
01).
30ite
ms
divi
ded
into
6su
bsca
les.
No
relia
bilit
y/va
lidity
prov
ided
.
Ble
ecke
r&
Sher
woo
d(2
003)
CC
WJS
IC
hild
Car
eW
orke
rJo
bSt
ress
Inve
ntor
y:C
hild
care
wor
ker
self
-rep
ort
Cur
bow
etal
.(20
00).
17-i
tem
mea
sure
with
3jo
bst
ress
subs
cale
s.Su
bsca
lere
liabi
lity
rang
edfr
omá
s=
.77–
.89.
Con
stru
ctva
lidity
was
dem
onst
rate
dfo
rsu
bsca
les
(Cur
bow
etal
.,20
00).
Gill
iam
(200
7)
CE
FC
onsu
lting
Eva
luat
ion
Form
:Se
lf-r
epor
tcom
plet
edby
the
MH
CE
rchu
leta
l.(1
992)
.12-
item
,7-p
oint
ratin
gsc
ale.
Bow
man
Ass
ocia
tes
&K
agan
(200
3)
CE
S-D
Cen
ter
for
Epi
dem
iolo
gic
Stud
ies–
Dep
ress
ion
scal
e:Te
ache
rse
lf-r
epor
t
Rad
loff
(197
7).I
nter
nalc
onsi
sten
cyfr
omαs
=.8
5–.9
0;te
st–r
está
=.5
4;V
alid
ityes
tabl
ishe
d(R
adlo
ff,1
977)
.
Gill
iam
(200
7)
CIS
Car
egiv
erIn
tera
ctio
nSc
ale:
Chi
ldca
repr
ovid
erra
ted
byan
outs
ide
obse
rver
Arn
ett(
1989
).In
terr
ater
relia
bilit
y.7
5–.9
7be
twee
nce
rtif
ied
obse
rver
and
trai
nees
(Jae
ger
&Fu
nk,2
001)
.C
oncu
rren
tval
idity
.43–
.67
betw
een
CIS
and
EC
ER
S(L
ayze
ret
al.,
1993
).
Ble
ecke
r&
Sher
woo
d(2
004)
;Bow
man
Ass
ocia
tes
&K
agan
(200
3);G
illia
m(2
007)
CL
ASS
Cla
ssro
omA
sses
smen
tSco
ring
Syst
em:
Chi
ldca
repr
ovid
erra
ted
bya
trai
ned
outs
ide
obse
rver
La
Paro
etal
.(20
04).
Inte
rrat
erre
liabi
lity
of87
%;e
stab
lishe
dpr
edic
tive,
crite
rion
,and
cons
truc
tva
lidity
(Ham
reet
al.,
n.d.
).
Rav
eret
al.(
inpr
ess)
(con
tinue
d)
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
1004
DE
CA
Dev
ereu
xE
arly
Chi
ldho
odA
sses
smen
t:Se
lf-r
epor
tque
stio
nnai
reco
mpl
eted
byca
regi
ver
LeB
uffe
&N
aglie
ri(1
999b
).R
elia
bilit
yfo
reac
hsc
ale,
ás
=.8
0–.9
2.C
rite
rion
valid
ity:c
orre
ctly
clas
sifie
d69
%of
181
child
ren
(LeB
uffe
&N
aglie
ri,1
999a
).
Olm
os&
Gri
mm
er(2
004)
EC
ER
SE
arly
Chi
ldho
odE
nvir
onm
entR
atin
gSc
ale:
Chi
ldca
repr
ogra
ms
rate
dby
atr
aine
dou
tsid
eob
serv
er
Har
ms
&C
liffo
rd(1
980)
.90%
inte
rrat
erre
liabi
lity
(Alk
onet
al.,
2003
).
Alk
onet
al.(
2003
);Sa
ffor
det
al.
(200
1)
EC
ER
S-R
Ear
lyC
hild
hood
Env
iron
men
tRat
ing
Scal
e–R
evis
ed:C
hild
care
prog
ram
sra
ted
bya
trai
ned
outs
ide
obse
rver
Har
ms
etal
.(20
02).
Tota
lsca
lein
tern
alco
nsis
tenc
yα
=.8
6(C
assi
dyet
al.,
2005
).
Ble
ecke
r&
Sher
woo
d(2
003)
;Bow
man
Ass
ocia
tes
&K
agan
(200
3);G
illia
m(2
007)
;She
lton
etal
.(20
02);
Tym
insk
i(20
01)
GA
SG
oalA
ttain
men
tSca
ling:
Chi
ldca
repr
ovid
er’s
and
teac
her’
sse
lf-r
epor
tof
com
pete
ncie
son
iden
tifie
dm
enta
lhe
alth
activ
itygo
als
Kir
esuk
etal
.(19
94).
Goo
din
ter-
judg
ere
liabi
lity
(coe
ffic
ient
s.5
1–.9
5;C
ytry
nbau
met
al.,
1979
).C
onte
ntva
lidity
isty
pica
llyhi
gh(S
hefl
eret
al.,
2001
).
Alk
onet
al.(
2003
);B
owm
anA
ssoc
iate
s&
Kag
an(2
003)
;Fie
ldet
al.(
2004
)
MB
IM
asla
chB
urno
utIn
vent
ory:
Teac
her
self
-rep
ort
Mas
lach
etal
.(19
96).
22ite
ms
that
asse
ss3
area
sof
burn
out.
Inte
rnal
relia
bilit
y(α
s=
.60–
.82)
and
esta
blis
hed
disc
rim
inan
tval
idity
(Byr
ne,1
991)
.
Hen
niga
net
al.(
2004
);L
angk
amp
(200
3)
MH
SSM
enta
lHea
lthSe
rvic
esSu
rvey
:Se
lf-r
epor
tsur
vey
com
plet
edby
earl
ych
ildho
odca
regi
vers
and
MH
Cs
Gre
enet
al.(
2006
).R
elia
bilit
ypr
ovid
edfo
rthe
follo
win
gsc
ales
:Rel
atio
nshi
pbe
twee
nst
affa
ndM
HC
(α=
.84)
,ef
fect
iven
ess
inhe
lpin
gch
ildou
tcom
es(α
>.8
4),s
taff
wel
lnes
s(α
=.8
6).
Gre
en,E
verh
art,
etal
.(20
04);
Gre
enet
al.(
2006
)
TAB
LE2
(Con
tinue
d)
Acr
onym
Ful
lNam
ean
dTy
peof
Mea
sure
Aut
hor
and
Dat
e;P
sych
omet
rics
Stud
y
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
1005
PMPa
rent
alM
oder
nity
Scal
e:Se
lf-r
epor
tof
am
oder
nor
trad
ition
alat
titud
eto
war
dch
ildre
arin
g
Scha
efer
&E
dger
ton
(198
5).F
ound
tobe
high
lyre
liabl
eba
sed
onte
sts
ofin
tern
alre
liabi
lity
and
test
–ret
est
relia
bilit
y.
Gill
iam
(200
7)
PMH
CQ
Pres
choo
lMen
talH
ealth
Con
sulta
tion
Que
stio
nnai
re:T
each
erse
lf-re
port
surv
eyre
gard
ing
perc
eptio
nsof
cons
ulta
tion
and
child
ren’
sm
enta
lhea
lth
Elia
s(2
004)
.Rel
iabi
lity
and
valid
ityno
tpr
ovid
ed.
Elia
s(2
004)
SBP
Surv
eyof
Bel
iefs
and
Prac
tices
:Te
ache
rse
lf-r
epor
tabo
utce
nter
appr
oach
toch
ildca
re
Mar
con
(199
9).R
elia
bilit
yα
=.9
5fo
rfu
llsc
ale
(Mar
con,
1999
).B
leec
ker
&Sh
erw
ood
(200
3);G
illia
m(2
007)
;Tym
insk
i(20
01)
SDC
SSe
lf-D
eter
min
atio
nC
areg
iver
sSc
ale:
Car
egiv
erse
lf-r
epor
tof
perc
eptio
nsof
self
-det
erm
inat
ion
Hof
fman
etal
.(20
01).
Scal
ew
asad
apte
dfr
omSe
lf-D
eter
min
atio
nE
duca
tor’
sSc
ale
(Hof
fman
etal
.,20
01).
Rel
iabi
lity
and
valid
ityno
tpr
ovid
ed.
Fiel
det
al.(
2004
)
TO
STe
ache
rO
pini
onSu
rvey
:Chi
ldca
repr
ovid
ers’
/pre
scho
olte
ache
rs’
self
-rep
orta
bout
thei
rat
titud
esan
dbe
liefs
Gel
ler
&Ly
nch
(199
9).R
elia
bilit
yfo
r13
item
scal
eα
=.6
6.A
lkon
etal
.(20
03);
Ble
ecke
r&
Sher
woo
d(2
004)
;Ble
ecke
ret
al.
(200
5);B
owm
anA
ssoc
iate
s&
Kag
an(2
003)
TO
S-R
Teac
her
Opi
nion
Surv
ey–R
evis
ed:
Chi
ldca
repr
ovid
ers’
/pre
scho
olte
ache
rs’
self
-rep
orta
bout
thei
rat
titud
esan
dbe
liefs
Gel
ler
&Ly
nch
(200
0).T
OS
was
revi
sed
byFi
eld
etal
.(20
04)
tore
flec
tchi
ldca
reco
mm
unity
,inf
ants
,an
dto
ddle
rs.R
elia
bilit
yan
dva
lidity
for
TO
S-R
notp
rovi
ded.
Fiel
det
al.(
2004
)
VS
Vin
elan
dA
dapt
ive
Beh
avio
rSc
ale:
Teac
her
orpa
rent
repo
rtof
child
ren’
spe
rson
alan
dso
cial
skill
s
Spar
row
etal
.(19
85).
Rel
iabi
lity
for
297-
item
surv
eyfo
rm:s
plit-
half
(αs
=.8
3–.9
0),t
est–
rete
st(α
s=
.81–
.86)
,in
terr
ater
(αs
=.6
2–.7
8).
Tym
inks
i(20
01)
Not
e:M
HC
=m
enta
lhea
lthco
nsul
tant
.
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
MacLennan, 2003; Type 3: Perry, Dunne, McFadden, & Campbell, 2008; Type 4:Green, Everhart, Gordon, & Gettman, 2006; Green, Simpson, Everhart, Vale, &Gettman, 2004).
Findings from RCT and Peer-Reviewed Studies
What can we learn about staff and program outcomes from the six studies that wereeither based on RCT designs or were peer reviewed or both?
Some of the most convincing evidence comes from Raver et al. (2008), who ex-amine the outcomes for staff and programs of the Chicago School Readiness Pro-ject using a RCT design. The study was based on the premise that consultationwould strengthen the capacity of teachers to be sensitive to children’s needs, tomanage difficult child behavior, and to create a positive (rather than negative)classroom climate, and that the changed interactions of staff with children wouldbe linked to improved child outcomes. A manualized consultation intervention(Madison-Boyd et al., 2006) was implemented in 18 randomly selected classroomsand contrasted with 17 control classrooms. Teachers in the selected classroomswere invited to 30 hours of training in the fall based on the Incredible Years teachertraining module (Webster-Stratton, Reid, & Hammond, 2004). A mental healthconsultant with a master’s degree in social work was also placed in each treatmentcondition classroom one morning a week to consult with and coach the teachers,delivering an average of 82 hr of consultation to each classroom. Consultantscoached teachers on the use of classroom management strategies learned in train-ing for the first 20 weeks starting in the fall, focused on stress reduction in the win-ter, and worked with teachers on the needs of individual children in the spring.Teachers in control classrooms were assisted by teacher’s aides for a time equiva-lent to the time spent by classroom consultants. Observers blind to the treatment orcontrol status of the classrooms collected data using the Classroom AssessmentScoring System (La Paro, Pianta, & Stuhlman, 2004) and the revised Early Child-hood Environment Rating Scale (ECERS-R; Harms, Clifford, & Cryer, 2002).
Using Hierarchical Linear Modeling, Raver et al. (2008) found that, control-ling for baseline levels on the four CLASS scales and for baseline ECERS-R rat-ings of classroom quality, the intervention had significant classroom-level ef-fects, boosting the positive climate of the classroom (d = .89), decreasing thenegative climate (d = .64), increasing teachers’ sensitivity (d = .53), and improv-ing their classroom management (d = .52). Raver et al. stated that the findingswere encouraging, given that their long-term goal was to link the classroom andteacher improvements to gains in children’s emotional, behavioral, and academicdevelopment.
In the other RCT study, Gilliam (2007) evaluated the effectiveness over time ofthe statewide Early Childhood Consultation Partnership (ECCP) in Connecticut.Again, the intervention aimed at improving teacher–child interactions as assessed
1006 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
by condition-blind observers who rated positive interactions and levels of teacherdetachment, permissiveness, and punitiveness, this time using the Caregiver Inter-action Scale (CIS; Arnett, 1989). Gilliam (2007) also measured whether the con-sultation would (a) improve the classroom environment as measured by theECERS-R; (b) positively affect the teachers’ beliefs and feelings about child-rearing as gauged by the Parental Modernity Scale of Childrearing and EducationBeliefs (Schaefer & Edgarton, 1985) and the Pre-K Survey of Beliefs and Practices(Marcon, 1999); and (c) decrease stress and depression as evidenced by lowerscores on the Child Care Worker Job Stress Inventory (CCWJSI; Curbow, Spratt,Ungaretti, McDonnell; & Breckler, 2000) and on the Center for EpidemiologicalStudies Depression Scale (CES-D; Radloff, 1977). Gilliam (2007) postulated thatthe classroom-level and teacher-level changes would be linked to improvements inteacher ratings of child behavior.
As in the Raver et al. (2008) study, ECCP also used consultants with master’sdegrees in a human services field but provided services in early childhood settingsover a more compressed period lasting 8 weeks, for 4 to 6 hr per week, with a fol-low-up visit at Week 12. Consultation was based on a manual that was menu drivenand provided help to individual teachers on classroom-level or child-level con-cerns. One-time 90-minute training sessions were held at each participating site ona topic of the teachers’ choosing. Gilliam (2007) randomly assigned early child-hood sites that were willing to participate in the study to treatment or wait-list con-trol groups. Two cohorts of participating sites yielded 43 treatment and 42 controlclassrooms.
In contrast to the Raver et al. (2008) results, Gilliam (2007) found no significantdifferences between treatment and control teachers in observations of teach-er–child interactions. Additionally, after pretest scores were controlled, he alsofound no differences in teacher-reported beliefs and practices, job stress levels, ordepression. Finally, observers did not rate classroom quality as different in inter-vention or control classrooms at the time of the posttest. What Gilliam did obtain,however, were positive effects in teacher ratings of child externalizing behaviorsand total behaviors for the two children in each classroom having behavior prob-lems of greatest concern.
A clue to the lack of differences in teacher improvement in the Gilliam (2007)study may be found in the research of Alkon et al. (2003), who evaluated the effectsof mental health consultation provided through the Early Childhood Mental HealthInitiative in San Francisco. This initiative provided ongoing consultation given bymental health professionals from four agencies to staff in 40 child care centers usinga variety of approaches, but focusing on building staff capacity to help children de-velop socially and emotionally and to manage difficult behavior. Alkon and her co-workers compared outcomes for staff in 9 centers that had more than one year ofconsultation to those for staff in 14 centers with less than one year of the service.Using a pretest/posttest comparison group design, the researchers found that centers
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1007
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
having a longer duration of consultation had significant improvements in centerquality as measured by the ECERS (Harms & Clifford, 1980) compared to centerswith consultation of shorter duration. Duration of consultation also significantly pre-dicted lower rates of teacher turnover, and increased teacher self-efficacy scores asrated using the Teacher Opinion Survey (TOS; Geller & Lynch, 1999).
Do teachers report that consultation is effective? Perry, Dunne, et al. (2008)evaluated the effects of consultation on outcomes for 192 children who had beenidentified as being at risk for expulsion from suburban Maryland child care set-tings. Teachers reported that only 13 of the 192 children (6.7%) changed theirplacements by being removed involuntarily. The researchers obtained post-consul-tation reactions of 25 randomly selected staff members who had referred childrento the study; the majority reported that the consultant became an ally who helpedthem transform problem behaviors and foster improved social skills for the chil-dren at risk for expulsion.
Perry’s findings were echoed in the results obtained by Green and her associates.Studying mental health consultation in Head Start settings, Green, Simpson, et al.(2004) used intensive, semistructured interviews with multiple types of participantsto draw empirical and theoretical conclusions regarding mental health consultationat multiple program sites. When consultants were well-integrated into Head Startstaff, and administrators, teachers, and teacher aides believed that fostering positivechild mental health was everyone’s responsibility, consultation services were seen asbeing more effective. Green and her coworkers (2006) then made use of the resultingtheoretical model to construct survey questions for a national survey of Head Startconsultants and staff with 655 respondents randomly drawn from staff lists of 74programs. They found that the most important predictor of perceived effectivenessof consultation was the quality of relationship of the staff member with the mentalhealth consultant. Results of hierarchical linear modeling analysis revealed that apositive teacher–consultant relationship was by far the strongest predictor of per-ceived effectiveness of consultation in reducing externalizing and internalizing be-havior, increasing positive behavior, and promoting staff wellness.
Overall Findings on Mental Health Consultation Outcomes
Although the 20 remaining studies included in this review have design limitationsand were not peer reviewed, they offer additional evidence that early childhoodmental health consultation may have a positive impact on staff and program out-comes. We will first discuss outcomes at the staff level and then turn our attentionto program-level findings.
Staff Outcomes
Teacher self-efficacy/confidence. Eleven studies were found that investi-gated teacher self-efficacy, which we defined as perceived operative capability
1008 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
(Bandura, 2007, p. 646) or confidence in their ability to work with children in theirclassroom, even those with difficult behavior. Nine of the 11 studies that investi-gated staff self-efficacy or confidence generally showed that mental health consul-tation was associated with staff perceptions of increases in their ability to managechildren’s behavior and perform their job. Alkon et al. (2003) found that mentalhealth consultation was related to a statistically significant increase in staff self-ef-ficacy as assessed through scores on the Teacher Opinion Survey (TOS). In addi-tion, qualitative analyses indicated that there was a shift in teachers’ attitudes to-ward themselves as well as their work. Similarly, Bowman Associates and Kagan(2003) established a statistically significant improvement in teacher self-efficacyusing the TOS. They also found enhancements in self-reported performance be-fore and after consultation services as measured through the Goal AchievementScale adapted from Kiresuk, Smith, & Cardillo (1994). In evaluating San Fran-cisco’s High Quality Child Care Mental Health Consultation Initiative, Bleecker,Sherwood, and Chan-Sew (2005) reported that most teachers indicated that con-sultation was helpful in managing their duties as measured with the TOS, with86% of teachers answering that the consultation “helped a lot” or “helped some.”However, neither Bleecker and Sherwood (2004; TOS) nor Field, Mackrain, andSawilowsky (2004; TOS-R) found significant changes from pretest to posttest.
Assessing the impact of PEARL—an early childhood program delivering men-tal health services in early learning environments in Denver, Colorado—Olmosand Grimmer (2004) found that participation in the intervention helped providersfeel more competent in addressing the socioemotional needs of children. As re-ported in the previous section, the Perry, Dunne, et al. (2008) survey found that themajority of child care providers indicated that consultation from behavioral spe-cialists was extremely helpful in building their skills in working with children ex-hibiting problem behaviors. A small minority of the child care providers in theMaryland study who reported that the consultation was not helpful were morelikely to be family child care providers.
Three studies provided additional evidence regarding improvements in staffconfidence in their performance of their jobs. Shelton, Woods, Williford, Dobbins,and Neal (2002) assessed the impact of mental health consultation models used asintervention strategies compared with a control group receiving a comprehensivebehavioral/developmental evaluation but no follow-up intervention. Teachers inthe intervention classrooms reported feeling significantly more confident in work-ing with children. Similarly, Brennan, Bradley, Ama, and Cawood (2003) reportedon qualitative data from staff and administrator interviews indicating staff supportprovided by mental health consultants increased staff confidence, allowing them tobetter address the socioemotional needs of children under their care. Hennigan,Upshur, and Wenz-Gross (2004) also found that teachers reported that their abilityto work with children and families had improved given the consultation they re-ceived over 4 to 6 months. Finally, in the national survey of Head Start staff dis-
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1009
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
cussed in the previous section, the vast majority of staff and administrators re-ported that mental health consultation supported staff to do their jobs better(Green, Everhart, et al., 2004).
Observed and self-reported competence. Staff levels of competence intheir interactions with children and parents were addressed in 10 studies, with allbut two showing positive outcomes linked to consultation. Raver et al. (2008)found that teacher classroom management was significantly better in interventionclassrooms after consultation when contrasted with classroom management incontrol classrooms (d = .57). In contrast, although Gilliam (2007) found that teach-ers receiving mental health consultation in their classrooms reported significantimprovements in child behavior, he found no differences between experimentaland control teachers’ reported beliefs or practices during a consultation interven-tion of short duration. Observation of staff interactions with children in the class-room using the CIS also did not show significant differences. Additionally,Bleecker and Sherwood (2003) did not find significant changes in self-reports ofteacher competency between baseline and follow-up. Finally, although Lehman,Lambarth, Friesen, MacLeod, and White (2005) found increases in teacher self-re-ports of competence, these differences did not reach statistical significance in theirlimited sample.
In related research, a report presenting findings from 802 staff surveys (repre-senting 79 programs) indicated that 85% of administrators and 67% of direct ser-vice providers reported that classroom staff did their jobs better because of mentalhealth consultation (Green, Everhart, et al., 2004). Quality of staff–consultant rela-tionships and frequency of consultation efforts were both linked with perceivedconsultant helpfulness in improving staff members’ ability to cope with difficultchild behavior (Green et al., 2006). Four studies also indicated that mental healthconsultation helped staff increase their ability to involve parents in improving chil-dren’s social–emotional development (Brennan et al., 2003; Elias, 2004; Pawl &Johnston, 1991; Shelton et al., 2002).
Sensitivity. Four of five studies investigating staff–child interactions foundthat early childhood staff who received mental health consultation demonstratedincreased sensitivity, that is, responsiveness to the expressed needs of children.Bleecker and Sherwood (2004) found significantly improved ratings of teachersensitivity over time using the Arnett Scale of Caregiver Interaction (CIS; Arnett,1989). This finding reinforces the results of Raver and her coworkers (2008), whoalso found that classrooms receiving mental health consultation had moderatelyimproved teacher sensitivity when contrasted with control classrooms. In theirevaluation of mental health consultation in child care centers, Bowman Associatesand Kagan (2003) also reported that teachers in intervention centers were rated asmore sensitive and less harsh on the CIS compared to teachers in the California
1010 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
sample of the Cost, Quality and Child Outcomes study (Helburn, 1995). In theBowman Associates and Kagan evaluation, higher center quality was associatedwith greater sensitivity among teachers. A qualitative analysis by Alkon et al.(2003) of staff interviews and focus groups revealed greater understanding of chil-dren’s behavior and greater empathy for children with behavior problems at fol-low-up than at baseline. In contrast, Gilliam (2007) found no statistically signifi-cant differences in CIS scores for intervention and control classroom teachers.Also, Bowman Associates and Kagan found no differences in CIS scores betweenstaff receiving consultation of longer versus shorter duration, in contrast to theBleecker and Sherwood (2004) findings.
Reduction of job-related stress. Three of four studies included in this re-view that investigated job-related stress found an association between mentalhealth consultation and reduction in job-related stress, defined as discomfortingreactions to the demands of the position and the lack of control the teacher feelsover work. Teachers reported feeling less stressed as a result of mental health con-sultation, and lower levels of burnout and staff turnover were found after consulta-tion provided by the Colorado PEARL project (Olmos & Grimmer, 2004). Amultimethod program evaluation that included telephone surveys of 39 directors ofprograms in the Early Childhood Mental Health Initiative of Ohio also indicatedthat reduced staff stress was an outcome of mental health consultation (Langkamp,2003). However, Gilliam (2007) reported that there were no differences in self-re-ported stress levels between teachers in intervention and control classrooms atposttest. In contrast, Green et al. (2006) found that higher self-reported levels ofstaff wellness were associated with higher quality relationships with consultantsand more frequent program and individual consultation.
Program Outcomes
Center quality. Seven of the reviewed studies assessed center quality, de-fined as presence of desirable physical and social environmental conditions andpositive classroom activities, with five finding favorable results. As establishedabove, Alkon et al. (2003) found that improvements in center quality were pre-dicted by duration of mental health consultation in 25 urban child care centers.Centers that had mental health consultation programs for more than a year showedsignificant improvement in overall child care quality compared with centers hav-ing consultation support for less than a year. Bleecker and Sherwood (2003) foundthat 90% of centers in a randomly selected sample of early childhood settings re-ceiving consultation had total ECERS-R ratings of “good” or “excellent.” Simi-larly, Bowman Associates and Kagan (2003) found evidence that mental healthconsultation supported higher quality care, but they did not find a statistically sig-nificant difference between ECERS-R scores of comparison groups with shorter
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1011
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
and longer durations of intervention. Findings from a telephone survey of programdirectors who received mental health consultation as part of the Early ChildhoodMental Health Initiative in Ohio indicated that 92% of those surveyed stronglyagreed or agreed that overall program quality had improved since a mental healthconsultant began providing services to their center (Langkamp, 2003). Finally,Raver and her coworkers (2008) reported that intervention classrooms receivingconsultation services had improved positive classroom climates when comparedwith control classrooms.
In contrast, Tyminski (2001) found a decrease in ECERS-R scores followingmental health consultation (pre- and post-intervention scores 5.68 vs. 5.35, respec-tively). However, quality among these centers was still within the “good” range.The researcher attributed the statistically significant reduction in center qualityscores to the timing of baseline and follow-up measurements. Similarly, Gilliam(2007) found no differences in ECERS-R scores between intervention and controlclassrooms at posttest.
Staff turnover. Six of the studies addressed staff turnover, of which five pro-vided evidence that early childhood mental health consultation is associated withreduced numbers of staff leaving programs. Olmos and Grimmer (2004) con-ducted in-depth interviews and focus groups and found that mental health consul-tation was reported to reduce stress, which in turn was perceived to help decreaseburnout and turnover. Gould (2003) found some evidence of reduced staff turnoverin the evaluation data of Colorado’s Consolidated Child Care Pilot Mental Healthmini-grants. Bleecker et al. (2005) reported that staff who found consultation help-ful were also more likely to have plans to continue their child care careers. Alkon etal. (2003) also found a high correlation between frequency of mental health con-sultation and lower staff turnover rates (r = –.43, p < .05). In addition, duration ofmental health consultation was moderately correlated with low staff turnover inAlkon et al. Additionally, Hennigan et al. (2004) found that burnout scores werelower for staff in programs that had consultation for a longer duration than thosewith more recent introduction of consultation. However, less than one third of pro-gram directors who received mental health consultation as part of the Early Child-hood Mental Health Initiative in Ohio strongly agreed or agreed that staff turnoverwas reduced since a mental health consultant began providing services to their cen-ters (Langkamp, 2003).
DISCUSSION
The 26 studies summarized in this review represent a variety of approaches to eval-uating the effectiveness of mental health consultation in improving staff and pro-gram outcomes. We have highlighted the findings from the minority of studies that
1012 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
employed a randomized controlled trial design and those that have been publishedin the peer-reviewed literature. A variety of staff- and program-level outcomeswere assessed using a wide array of tools; the mostly commonly used instrumentswere a measure of overall program quality (ECERS), observer ratings of staff com-petence (CIS), and measures of teachers’ attitudes (TOS, GAS). Staff turnover,burnout, and stress were also measured in many of these studies.
Overall, the results of this review suggest that mental health consultation can beeffective in building the capacity of early childhood providers to cope with chil-dren’s difficult behavior by improving their competence in promoting social andemotional development and in managing their classrooms. In addition, the dataseem to indicate that consultation increases teacher self-efficacy and reduces staffstress and turnover. How effective mental health consultation is in improving theoverall quality of the child care environment—as measured by the ECERS—is lessclear, with some studies reporting positive associations, others finding no relation-ship, and one even reporting a negative relationship over time. Additional researchis needed to determine if consultation by a mental health specialist will affect therange of quality indicators measured by the ECERS.
Limitations of the Review
Although this synthesis provides valuable information on the current state of theevidence base for early childhood mental health consultation, there are several lim-itations to our findings. First, our reliance on studies that were not published inpeer-reviewed venues means that we must interpret our results with caution. Al-though some of the papers we included may not have been published due to meth-odological concerns or weaknesses, others (especially those reports of more recentand rigorous studies) will find their way into peer-reviewed journals in the comingmonths. Some field-based evaluators may have been interested only in communi-cating their findings to their funders and the program staff rather than seeking tocontribute to the peer-reviewed evidence, whereas university-based researchersmay have additional incentives to pursue this (time-consuming) process. Finally,we have avoided giving the mistaken impression that the level of “evidence” is suf-ficient to warrant a meta-analysis and instead have chosen to characterize the qual-ity of the existing data using a systematic review and research synthesis approach(Mullen & Ramírez, 2006).
There are other limitations inherent in our approach to gathering the studies foranalysis. It is possible that additional studies reporting on mental health consulta-tion were omitted from this review. Because of the substantial diversity in howmental health consultation is defined and implemented across early childhood set-tings, studies may have been overlooked because of conceptual differences in thedefinition of mental health consultation. This problem may have been com-pounded by the fact that much of the current research is not published in peer-re-
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1013
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
viewed venues. In addition, a formal rating of interrater reliability for article inclu-sion was not calculated.
The studies included in the synthesis may not be representative of all mentalhealth consultation programs. Given our emphasis on staff- and program-level out-comes for this article, there may be systematic differences between the mentalhealth consultation programs included in this review and those that did not reporton these outcomes. Furthermore, the research presented in this synthesis evaluateda variety of specific staff- and program-level outcomes using several differentmeasures. There is currently no consensus in the research on mental health consul-tation about which outcomes should be assessed, nor is there a gold standard ofmeasurement. In addition, researchers (Bleecker & Sherwood, 2003; Bowman As-sociates & Kagan, 2003) reported that it was difficult to determine if the mentalhealth consultation was the variable responsible for producing the outcomes, or ifanother variable may have been responsible for the change.
The review was also limited in its scope due to the small sample sizes in manyof the studies leading to constraints on the type of analyses the researchers wereable to use. Only two studies (Gilliam, 2007; Raver et al., 2008) provided effectsizes, and few authors supplied detailed statistical reporting, making a meta-analy-sis of studies less viable. Difficulty in teasing out overall patterns in these studieswas also heightened due to the variety of measures that were used by investigatorsto detect intervention effects on staff and programs. An additional limitation is thereliance on staff-reported outcomes for many of the measures. Again, Gilliam(2007) and Raver et al. are leading the way for future studies, as they both includedindependent assessments of the classroom climate.
Implications of the Findings for Research, Practice,and Policy
These findings underscore several important research, practice, and policy issuesthat require additional attention. First, there is a need to identify the key compo-nents of effective mental health consultation, and this process should be driven byevaluation data. The studies reviewed in this article varied in the level of detail thatwas provided on the key aspects of the intervention and associations between studyvariables. There was insufficient information provided in most studies to deter-mine critical aspects of the consultation services—including the specific activitiesengaged in, frequency of contacts, level of intensity of intervention, and the con-tent or quality of the information being conveyed by the consultant. Green et al.(2006) found that the quality of the relationship between the consultant and theHead Start teacher was the single most important predictor of positive outcomes,but this variable was unmeasured and overlooked in most other studies. The qual-ity of this relationship may be influenced by the skills, expertise, and “consultativestance” (Johnston & Brinamen, 2005; Parlakian, 2001) that the consultant brings
1014 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
to the relationship; it may also be influenced by the attitudes, experience, and moti-vations of the child care provider.
Cohen and Kaufmann (2000) reported consensus at a roundtable of experts thatconsultation should be provided by a “licensed mental health professional,” whichin many states would require a minimum of a master’s-level education. Althoughthere is reason to believe that consultation provided by a professional with exper-tise in the mental health of young children could be more effective than that pro-vided by someone who lacks that specific expertise, there are currently no data tosupport that assumption. In addition, there were no studies that compared differentmodels of consultation within a single study, which limits our ability to assesswhich approaches may lead to better outcomes in similar populations.
Future research should focus on studies that compare variations in key elementsof the consultation (i.e., qualifications of the consultant, frequency and intensity ofthe consultation, modality through which the consultation is delivered) so thatthese relationships may be explored. Ideally, these studies would be based on ran-domized controlled trials or use rigorous statistical methods (such as regres-sion-discontinuity analyses) to adjust for lack of randomization. There is also aneed for longitudinal data to be collected, preferably linking the shorter term out-comes achieved through mental health consultation to longer term increases inschool readiness skills. Linking changes in staff- and program-level outcomes tochild- and family-level variables will also provide support for a theory of changethat posits these mediating pathways.
More recent studies, such as those conducted by Raver et al. (2008) and Gilliam(2007), demonstrate the value of articulating a theory of change—that is, specify-ing the pathways through which the consultation services are thought to affect arange of outcomes. This process helps ensure that researchers are both measuringthe mediators of such consultative models as well as linking specific aspects of theconsultation process to measured outcomes. Several studies used the Early Child-hood Environmental Rating Scale (ECERS) as an outcome measure, for example;this implies that one goal of mental health consultation is to improve the overallquality of the early childhood classroom environment. However, in those studieswhere improvements in the ECERS scores were reported, variables that mighthave mediated those changes in the ECERS (such as specific consultation on top-ics such as changes in the routines of the classroom, improved transitions betweenactivities, changes in room arrangement) were not identified or specifically mea-sured. For studies where no change in program quality (as measured by theECERS) was observed, unresolved questions remain: Did the consultation not spe-cifically address these topics, or did teachers fail to implement these strategies asrecommended by the consultant? Were there changes in these aspects of theECERS, but were they perhaps not of sufficient magnitude to lead to detectabledifferences in the overall score? Without a well-constructed theory of change, weare left not knowing the how or why of many of these important questions.
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1015
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
The importance of identifying theories of change underscores one of the mostpersistent problems in evaluating the effectiveness of mental health consultation:the lack of valid and reliable tools for measuring mediating variables as well as sa-lient staff and program outcomes. Many of these programs were initiated in re-sponse to concerns about the prevalence of young children’s problem behaviors,and there are a wide variety of tools for assessing changes in children’s social skillsand problem behavior. On the other hand, there are few reliable and valid toolsavailable to measure the pathways through which mental health consultation mayaffect children’s behavior (e.g., changes in teachers’ attitudes, beliefs, practices;adoption of new skills to reduce problem behavior in a specific child; the quality ofthe relationship between the consultant and teacher). Systematically assessingthese constructs is likely to require a combination of self-report and observationaldata, such as those gathered by Raver et al. (2008). New tools will need to be devel-oped to examine the broader impact of changes in teachers’ behavior on other chil-dren in a class with a child identified with problem behaviors. These gaps in boththeory and measurement need to be addressed by the next generation of researchstudies.
There are other constructs of interest that would benefit from more attention totheory and measurement, such as the role cultural factors play in moderating theeffectiveness of mental health consultation in child care settings. These culturalfactors operate at multiple levels and require researchers to go beyond reports ofthe ethnicity of children served or demographic characteristics of the early careand education providers. The “culture” of child care, in contrast to the norms, prac-tices, and values of those in the mental health field, needs to be explored, as doesthe important role that culture plays in setting expectations about young children’sbehavior. For example, in the Gilliam (2005) preschool expulsion study, AfricanAmerican children were twice as likely to be expelled as Euro-American chil-dren and 4.5 times more likely than Asian American children. Investigatorsshould also examine the specific strategies that promote effective consultation ininfant/toddler versus preschool-age settings, or family day care versus child caresettings.
As states and communities expand their capacity to meet the mental healthneeds of young children and their caregivers (Finello & Poulsen, 2005), the needfor accurate, data-driven information about effective strategies to deliver mentalhealth consultation is growing (Wesley & Buysse, 2006). There remain many un-answered questions about the key components of effective consultation and thebest tools to evaluate the impact of consultation on staff and program outcomes.Researchers must team with state and local policymakers and program managersto contribute to the knowledge base of what works for whom in which settings(Hepburn et al., 2007; Perry, Woodbridge, & Rosman, 2007). The lessons learnedfrom this review should help position all stakeholders to better address these gapsin our evidence base.
1016 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
ACKNOWLEDGMENTS
Preparation of this manuscript was supported in part by the Research and TrainingCenter on Family Support and Children’s Mental Health of the Regional ResearchInstitute for Human Services, Portland State University, through NIDRR GrantH133B990025 and by the National Technical Assistance Center for Children’sMental Health, Georgetown University, through the Center for Mental Health Ser-vices SAMHSA Grants RX4285309 and SM-05-013. A preliminary version ofthis study was presented at the 19th Annual Research Conference “A System ofCare for Children’s Mental Health: Expanding the Research Base,” University ofSouth Florida, Louis de la Parte Florida Mental Health Institute, Research andTraining Center for Children’s Mental Health, Tampa, February 2006. We aregrateful to Beth Green and Roxane Kaufmann for their reviews of an earlier ver-sion of this article; to Adey Tsega for her work on a preliminary review paper; andto Ariel Holman, Stacey Sowder, and Anna Malsch for their help with manuscriptpreparation.
REFERENCES
References marked with an asterisk (*) indicate studies included in the research review, and thosemarked with a double asterisk (**) indicate sources of measures used in the studies. References markedwith a triple asterisk (***) both are included in the review and have provided a measure used in one ormore studies.
*Alkon, A., Ramler, M., & MacLennan, K. (2003). Evaluation of mental health consultation in childcare centers. Early Childhood Education Journal, 31(2), 91–99.
**Arnett, J. (1989). Caregivers in day-care centers: Does training matter? Journal of Applied Develop-mental Psychology, 10, 541–552.
Bandura, A. (2007). Much ado over a faulty conception of perceived self-efficacy grounded in faultyexperimentation. Journal of Social and Clinical Psychology, 26, 641–658.
Berkowitz, I. H. (2001). Evaluations of outcome in mental health consultation in schools. Child and Ad-olescent Psychiatric Clinics of North America, 10(1), 93–103.
*Bleecker, T., & Sherwood, D. L. (2003). A final report of the 2001–2002 evaluation of the High Qual-ity Child Care Mental Health Consultation Initiative. San Francisco: Department of Public Health,Community Health Programs.
*Bleecker, T., & Sherwood, D. L. (2004). San Francisco High Quality Child Care Mental Health Con-sultation Initiative. San Francisco: Department of Public Health, Community Behavioral Health Ser-vices.
*Bleecker, T., Sherwood, D., & Chan-Sew, S. L. (2005). San Francisco High Quality Child Care Men-tal Health Consultation Initiative. San Francisco: Department of Public Health, Community Behav-ioral Health Services.
*Bowman Associates, & Kagan, S. L. (2003). Evaluation report on mental health consultation tochildcare centers. San Francisco: Jewish Family and Children’s Services. Retrieved November 8,2008, from www.jfcs.org/Services/Children,_Youth,_and_Families/Parents_Place/Early_Childhood_Mental_Health_Consultation/JFCS-MentalHealthEvaluationReport.pdf
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1017
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
Brennan, E. M., Bradley, J. R., Allen, M. D., Perry, D. F., & Tsega, A. (2005, March). The evidence basefor mental health consultation in early childhood settings: Research synthesis and review. Paper pre-sented at Establishing the Evidence Base for Early Childhood Mental Health Consultation, Tampa,FL.
*Brennan, E. M., Bradley, J. R., Ama, S. M., & Cawood, N. (2003). Setting the pace: Model inclusivechildcare centers serving families of children with emotional or behavioral challenges. Portland,OR: Portland State University, Research and Training Center on Family Support and Children’sMental Health.
Brennan, E. M., Bradley, J. R., Gettman, M. G., & Ama, S. (2006). Initiatives supporting children withemotional or behavioral challenges in child care settings. In C. Newman, C. Liberton, K. Kutash, &R. M. Friedman (Eds.), The 18th annual research conference proceedings: A system of care for chil-dren’s mental health: Expanding the research base (pp. 247–250). Tampa: University of SouthFlorida, Louis de la Parte Florida Mental Health Institute, Research and Training Center forChildren’s Mental Health.
Byrne, B. M. (1991). The Maslach Burnout Inventory: Validating factorial structure and invarianceacross intermediate, secondary, and university educators. Multivariate Behavior Research, 26,583–605.
Capizzano, J., & Adams, G. (2003). Children in low-income families are less likely to be in cen-ter-based care (Snapshots of America’s Families III, No. 16). Washington, DC: Urban Institute.
Cassidy, D. J., Hestenes, L. L., Hegde, A., Hestenes, S., & Mims, S. (2005). Measurement of quality inpreschool child care classrooms: An exploratory and confirmatory factor analysis of the Early Child-hood Environment Rating Scale–Revised. Early Childhood Research Quarterly, 20, 345–360.
Cohen, E., & Kaufmann, R. (2000, May). Early childhood mental health consultation. Washington,DC: Center for Mental Health Services of the Substance Abuse and Mental Health Services Admin-istration, and the Georgetown University Child Development Center.
Colorado Division of Mental Health. (2006). Interrater reliability study: CCAR 2006. Denver, CO: De-partment of Human Services. Retrieved January 27, 2008, from www.cdhs.state.co.us/dmh/PDFs/DE_CCAR_interraterccar2006rpt.pdf
Cook, T. D., Cooper, H., Cordray, D. S., Hartmann, H., Hedges, L. V., Light, R. J., et al. (1992).Meta-analysis for explanation: A casebook. New York: Russell Sage Foundation.
Cooper, H. M., & Lindsay, J. J. (1998). Research synthesis and meta-analysis. In L. Bickman & D. J.Rog (Eds.), Handbook of applied social research methods (pp. 315–337). Thousand Oaks, CA: Sage.
**Curbow, B., Spratt, K., Ungaretti, A., McDonnell, K., & Breckler, S. (2000). Development of theChild Care Worker Job Stress Inventory. Early Childhood Research Quarterly, 15, 515–536.
Cytrynbaum, S., Ginath, Y., Birdwell, J., & Brandt, L. (1979). Goal attainment scaling: A critical re-view. Evaluation Quarterly, 3, 5–40.
Denham, S. A. (2006). Social-emotional competence as support for school readiness: What is it andhow do we assess it? Early Education and Development, 17, 57–89.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2000). Handbook of qualitative research (2nd ed.). ThousandOaks, CA: Sage.
Donohue, P., Falk, B., & Provet, A. G. (2000). Mental health consultation in early childhood. Balti-more: Brookes.
*Early Head Start National Resource Center. (2004). Evaluation of the Pathways to Prevention initia-tive of the Early Head Start National Resource Center at Zero to Three. Washington, DC: Early HeadStart National Resource Center at Zero to Three.
*Elias, C. F. R. (2004). Mental health consultation services in community based early childhood set-tings: A survey of preschool teachers. Dissertation Abstracts International, 64(11-B), 5778.
**Ellis, R. H., Wackwitz, J. H., & Foster, M. (1991). Use of an empirically derived client topologybased on level-of-functioning: Twelve years of the CCAR. Journal of Mental Health Administration,18, 88–100.
1018 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
**Erchul, W. P., Hughs, J. N., Meyers, J., Hickman, J. A., & Braden, J. (1992). Dyadic agreement con-cerning the consultation process and its relation to outcome. Journal of Educational and Psychologi-cal Consultation, 3, 119–131.
*Field, S., & Mackrain, M. (2004). Michigan child care expulsion prevention projects: Evaluation re-port, October, 2002–September, 2003. Birmingham: Michigan Department of Community Health.
*Field, S., Mackrain, M., & Sawilowsky, S. (2004). Caring for Kids initiative: Evaluation report, May1, 2003–April 30, 2004. Detroit, MI: Wayne State University, College of Education.
Finello, K. M., & Poulsen, M. K. (2005). Developing standards for training in infant and preschoolmental health. In K. M. Finello (Ed.), The handbook of training and practice in infant and preschoolmental health (pp. 307–325). San Francisco: Jossey-Bass.
**Geller, S., & Lynch, K. (1999). Teacher Opinion Survey. Richmond: Virginia Commonwealth Uni-versity Intellectual Property Foundation, and Wingspan.
**Geller, S., & Lynch, K. (2000). Teacher Opinion Survey–Revised. Richmond: Virginia Common-wealth University Intellectual Property Foundation, and Wingspan.
Gilliam, W. S. (2005). Prekindergarteners left behind: Expulsion rates in state prekindergartener sys-tems. New Haven, CT: Yale University, Child Study Center.
*Gilliam, W. S. (2007, May). Early Childhood Consultation Partnership: Results of a random-controlledevaluation. Final report and executive summary. New Haven, CT: Yale University, Child Study Center.
Gilliam, W. S., & Shahar, G. (2006). Pre-kindergarten expulsion and suspension: Rates and predictorsin one state. Infants & Young Children, 19, 228–245.
*Gould, M. (2003). Evaluation of the Early Intervention Program for Young Children and the Consoli-dated Child Care Pilot Mental Health mini grants. Denver: University of Colorado at Denver, Gradu-ate School of Public Affairs, Center for Human Investment Policy.
***Green, B. L., Everhart, M., Gettman, M. G., Gordon, L., & Friesen, B. (2004). Mental health con-sultation in Head Start: Selected national findings. Portland, OR: Portland State University, Re-search and Training Center on Family Support and Children’s Mental Health.
*Green, B. L., Everhart, M., Gordon, L., & Gettman, M. G. (2006). Characteristics of effective mentalhealth consultation in early childhood settings: Multilevel analysis of a national survey. Topics inEarly Childhood Special Education, 26, 142–152.
*Green, B. L., Simpson, J., Everhart, M. C., Vale, E., & Gettman, M. G. (2004). Understanding inte-grated mental health services in Head Start: Staff perspectives on mental health consultation. NHSADialog, 7(1), 35–60.
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulle-tin, 82, 1–20.
Gross, D., Sambrook, A., & Fogg, L. (1999). Behavior problems among young children in low-incomeurban day care centers. Research in Nursing & Health, 22, 15–25.
Hamre, B. K., Mashburn, A. J., Pianta, R. C., Locasle-Crouch, J., & La Paro, K. M. (n.d.). ClassroomAssessment Scoring System: Technical appendix. Retrieved January 27, 2008, from http://classobservation.com/research/class_tech_manual.pdf
**Harms, T., & Clifford, R. M. (1980). Early Childhood Environment Rating Scale. New York:Teachers College Press.
**Harms, T., Clifford, R. M., & Cryer, D. (2002). Early Childhood Environment Rating Scale (Rev.ed.). New York: Teachers College Press.
Helburn, S. W. (Ed.). (1995). Cost, quality, and child outcomes in child care centers. Technical reports,public report, and executive summary. Denver, CO: University of Colorado. Retrieved November 8,2008, from Educational Resources Information Center: http://eric.ed.gov/ERICdocs/data/ericdocs2sq1/content_storage_01/00019b/80/14122/e6.pdf
*Hennigan, L., Upshur, C., & Wenz-Gross, M. (2004). Together for kids: Second year report. Wor-cester: Health Foundation of Central Massachusetts. Retrieved November 8, 2008, from www.hfcm.org/CMS/images/TFK%20Year%20Two%20Report.pdf
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1019
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
Hepburn, K. S., Kaufmann, R. K., Perry, D. F., Allen, M. D., Brennan, E. M., & Green, B. L. (2007).Early childhood mental health consultation: An evaluation tool kit. Washington, DC: GeorgetownUniversity, Technical Assistance Center for Children’s Mental Health; Johns Hopkins University,Women’s and Children’s Health Policy Center; and Portland State University, OR, Research andTraining Center on Family Support and Children’s Mental Health.
**Hoffman, A., Field, S., & Sawilowsky, S. (2001). Self-determination Educator’s Scale. Austin, TX:PRO-ED.
*Hutchison, M. A. (1986). Strengthening Head Start families: Reducing high risk through mentalhealth prevention/intervention (MHP/IP). Overview and final report. Washington, DC: Administra-tion for Children, Youth, and Families.
Jaeger, E., & Funk, S. (2001). The Philadelphia Child Care Quality Study: An examination of quality inselected early education and care settings. Philadelphia: Saint Joseph’s University.
Johnston, K., & Brinamen, C. (2005). Integrating and adapting infant mental health principles in thetraining of consultants to childcare. Infants & Young Children, 18, 269–281.
Johnston, K., & Brinamen, C. (2006). Mental health consultation in child care: Transforming relation-ships among directors, staff, and families. Washington, DC: Zero to Three.
**Kiresuk, T. J., Smith, A., & Cardillo, J. E. (Eds.). (1994). Goal attainment scaling: Applications, the-ory, and measurement. Hillsdale, NJ: Erlbaum.
Kupersmidt, J. B., Bryant, D., & Willoughby, M. (2000). Prevalence of aggressive behaviors amongpreschoolers in Head Start and community child care programs. Behavioral Disorders, 26(1), 42–52.
*Langkamp, D. L. (2003). Evaluation of the Early Childhood Mental Health Initiative. Akron: OhioDepartment of Mental Health, Office of Program Evaluation and Research.
**La Paro, K., Pianta, R., & Stuhlman, M. (2004). Classroom Assessment Scoring System (CLASS):Findings from the pre-k year. Elementary School Journal, 104, 409–426.
Layzer, J. I., Goodson, B. D., & Moss, M. (1993). Observational study of early childhood programs, fi-nal report: Vol. I. Life in preschool. Cambridge, MA: Abt Associates.
LeBuffe, P. A., & Naglieri, J. A. (1999a). The Devereux Early Childhood Assessment (DECA): A mea-sure of within-child protective factors in preschool children. NHSA Dialog, 3(1), 75–80.
**LeBuffe, P. A., & Naglieri, J. A. (1999b). Devereux Early Childhood Assessment Technical Program.Lewisville, NC: Kaplan Press.
*Lehman, C., Lambarth, C. H., Friesen, B., MacLeod, M., & White, D. (2005). Evaluation of the In-credible Years and mental health consultation to child care centers. Unpublished manuscript, Port-land State University, Regional Research Institute for Human Services.
Lombardi, J. (2003). Time to care: Redesigning child care to promote education, support families andbuild communities. Philadelphia: Temple University Press.
Madison-Boyd, S., Raver, C., Aufmuth, E., Jones-Lewis, D., Barden, K., & Williams, M. (2006).The Chicago school readiness mental health consultation manual. Chicago: University of Chi-cago.
**Marcon, R. A. (1999). Differential impact of preschool models on development and early learning ofinner-city children: A three-cohort study. Developmental Psychology, 35, 358–375.
**Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). The Maslach Burnout Inventory (3rd ed.). PaloAlto, CA: Consulting Psychologists Press.
Medway, F. J. (1979). How effective is school consultation? A review of recent research. Journal ofSchool Psychology, 17(3), 275–280.
Medway, F. J., & Updyke, J. F. (1985). Meta-analysis of consultation outcome studies. American Jour-nal of Community Psychology, 13, 489–505.
Mullen, P. D., & Ramírez, G. (2006). The promise and pitfalls of systematic reviews. Annual Review ofPublic Health, 27, 81–102.
NICHD Early Child Care Research Network. (2006). Child care effect sizes for the NICHD Study ofEarly Child Care and Youth Development. American Psychologist, 61, 99–116.
1020 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
*Olmos, A., & Grimmer, M. (2004). The Parent/Provider Effectiveness in Early Learning Environ-ments (PEARL) project. Denver, CO: Mental Health Corporation of Denver.
Parlakian, R. (2001). Look, listen, and learn: Reflective supervision and relationship-based work.Herndon, VA: Zero to Three.
*Pawl, J. H., & Johnston, K. (1991). Daycare consultants: A component of the Infant–Parent Programof the University of California San Francisco. Process evaluation report. San Francisco: Universityof California San Francisco.
Perry, D. F., Allen, M. D., Brennan, E. M., & Bradley, J. R. (2008). The evidence base on mental healthconsultation in early childhood settings: Research synthesis addressing children’s behavioral out-comes. Manuscript in preparation.
*Perry, D. F., Dunne, M. C., McFadden, L., & Campbell, D. (2008). Reducing the risk for preschool ex-pulsion: Mental health consultation for young children with challenging behavior. Journal of Childand Family Studies, 17, 44–54.
Perry, D. F., Woodbridge, M. W., & Rosman, E. A. (2007). Evaluating outcomes in systems deliveringearly childhood mental health services. In D. F. Perry, R. K Kaufman, & J. Knitzer (Eds.), Social andemotional health in early childhood: Building bridges between services and systems (pp. 121–146).Baltimore: Brookes.
Qi, C. H., & Kaiser, A. P. (2003). Behavior problems in preschoolers from low-income families: Re-view of the literature. Topics in Early Childhood Special Education, 23, 188–216.
**Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the generalpopulation. Applied Psychological Measurement, 1, 385–401.
*Raver, C. C., Jones, S. M., Li-Grining, C., Metzger, M., Smallwood, K., & Sardin, L. (2008). Im-proving preschool classroom processes: Preliminary findings from a randomized trial implementedin Head Start settings. Early Childhood Research Quarterly, 23, 10–26.
Raver, C., & Knitzer, J. (2002). Ready to enter: What research tells policymakers about strategies topromote social and emotional school readiness among three- and four-year-old children. New York:Columbia University, National Center for Children in Poverty.
Reddy, L. A. (2000). Clinical focus of consultation outcome research with children and adolescents.Special Services in the Schools, 16, 1–22.
Roach, M. A., Riley, D. A., Adams, D., & Edie, D. (2005). Evaluation of a state initiative to improvechild care quality. Early Education and Development, 16, 69–84.
Sadish, W. R. (1992). Do family and marital psychotherapies change what people do? A meta-analysisof behavioral outcomes. In T. D. Cook et al. (Eds.), Meta-analysis for explanation: A casebook (pp.129–208). New York: Russell Sage Foundation.
*Safford, P., Rogers, L., & Habashi, J. (2001). A qualitative study of Day Care Plus: Children, provid-ers, and the consultation process. Cleveland, OH: Case Western Reserve University, Schubert Centerfor Child Development.
**Schaefer, E., & Edgerton, M. (1985). Parental and child correlates of parental modernity. In I. E.Sigel (Ed.), Parental belief systems: The psychological consequences for children (pp. 287–318).Hillsdale, NJ: Erlbaum.
Shefler, G., Canetti, L., & Wiseman, H. (2001). Psychometric properties of goal-attainment scaling inthe assessment of Mann’s time-limited psychotherapy. Journal of Clinical Psychology, 57, 971–979.
*Shelton, T. L., Woods, J. E., Williford, A. P., Dobbins, T. R., & Neal, J. M. (2002). System of care in-terventions for hard to manage preschoolers in Head Start. In C. Newman, C. Liberton, K. Kutash, &R. M. Friedman (Eds.), The 14th annual research conference proceedings: A system of care for chil-dren’s mental health: Expanding the research base (pp. 213–218). Tampa: University of SouthFlorida, Louis de la Parte Florida Mental Health Institute, Research and Training Center forChildren’s Mental Health.
**Sparrow, S. S., Balla, D. A., & Cicchetti, D. V. (1985). Vineland Adaptive Behavior Scales—Class-room edition. Circle Pines, MN: American Guidance Service.
THE EVIDENCE BASE FOR MENTAL HEALTH CONSULTATION 1021
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
***Tyminksi, R. (2001). A final report of an evaluation of mental health consultation in child care cen-ters: San Francisco’s High Quality Child Care Initiative (1999–2001). San Francisco: San FranciscoDepartment of Children, Youth and Their Families, Department of Human Services, and Departmentof Public Health.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with early-onset conductproblems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Childand Adolescent Psychology, 33, 105–124.
Wesley, P. W., & Buysse, V. (2006). Ethics and evidence in consultation. Topics in Early ChildhoodSpecial Education, 26, 131–141.
West, J., Denton, K., & Reaney, L. M. (2001). The kindergarten years: Findings from the Early Child-hood Longitudinal Study, kindergarten class of 1998–1999. Washington, DC: National Center forEducation Statistics.
Zins, J. E. (2007). Has consultation achieved its primary prevention potential? Journal of Educationaland Psychological Consultation, 17, 133–150.
1022 BRENNAN, BRADLEY, ALLEN, PERRY
Dow
nloa
ded
by [
Uni
vers
ity o
f N
ebra
ska,
Lin
coln
] at
06:
27 1
8 O
ctob
er 2
014
