EA OURNEY JACKSONVILLE J RLY LEARNING P ROGRAM

JACKSONVILLE JOURNEYEARLY LEARNING PROGRAM

Early Learning Coalition of DuvalEpiscopal Children’s Services

Child Guidance CenterFlorida Institute of Education at theUniversity of North Florida

Evaluation Report Fall 2011

Jacksonville Journey Early Learning Program: Evaluation Report Fall 2011 Prepared by the Florida Institute of Education at the University of North Florida

The Jacksonville Journey Early Learning Program

Evaluation Report

Fall 2011

Prepared by

Florida Institute of Education at the University of North Florida

In collaboration with

Early Learning Coalition of Duval, Inc.

Episcopal Children’s Services

Child Guidance Center

TABLE OF CONTENTS

Jacksonville Journey Early Learning Program: Evaluation Report Fall 2011 Page i Prepared by the Florida Institute of Education at the University of North Florida

INTRODUCTION .......................................................................................................................... 1 EVALUATION PLAN .................................................................................................................... 2 SECTION 1: EARLY LEARNING CENTER OUTCOMES ................................................................. 3

Expected Outcome 1: At the end of the contract period (September 30, 2011), at least 21 of the 35 centers will increase their star rating by one star or maintain a 5-star rating............ 4

SECTION 2: SCHOOL READINESS OUTCOMES ............................................................................ 5 Expected Outcome 2: At the end of the contract period, at least 70% of Jacksonville Journey ELP School Readiness students will measure “ready for kindergarten” on Florida School Readiness measures................................................................................................ 10 Expected Outcome 3: Jacksonville Journey Early Learning Program children will demonstrate positive readiness outcomes as measured by the Bracken Basic Concept Scale—Third Edition: Receptive ........................................................................................ 25 Expected Outcome 4: Jacksonville Journey Early Learning Program children will demonstrate positive readiness outcomes as measured by the Florida Kindergarten Readiness Screener ............................................................................................................ 13 Expected Outcome 5: Prekindergarten children enrolled in Journey Early Learning Program child care centers will demonstrate significant and meaningful gains in early reading achievement .......................................................................................................... 30

SECTION 3: MENTAL HEALTH OUTCOMES .............................................................................. 31

Expected Outcome 6: 85% of clients will improve their social/emotional functioning as evidenced by CFARS pre/post assessment ......................................................................... 37 Expected Outcome 7: 85% of parents and teachers will indicate that the mental health services received were beneficial, via survey instrument .................................................... 40 Expected Outcome 8: 85% of clients enrolled in the program will complete treatment ....... 41

Jacksonville Journey Early Learning Program: Evaluation Report Fall 2011 Page 1 Prepared by the Florida Institute of Education at the University of North Florida

Jacksonville Journey Early Learning Program

INTRODUCTION

In September of 2008, the Jacksonville City Council initiated the Jacksonville Journey: A

Comprehensive Community-Wide Anti-Crime Initiative1 designed to combat escalating crime in

Jacksonville by: (a) increasing police presence and eliminating crime, (b) assisting ex-offenders

in building job skills and becoming responsible community members, and (c) keeping children

safe and productive, particularly after school and in the summer.

Funding provided by this initiative is also invested in improving children’s early literacy

and school readiness through the Jacksonville Journey Early Learning Program (ELP). The

overarching 2010-11 Jacksonville Journey ELP goal was to transition 40 custodial child care

centers located in Jacksonville’s downtown area into early learning centers that not only address

children’s educational development but also their mental/social/emotional health. A variety of

support services were offered to directors, staff, children, and families in these centers.

Coaching, technical assistance, and environmental support were provided to center directors and

staff to improve the overall quality of delivered programs. Educational scholarships were

provided to participating teachers. Programs were developed to encourage the engagement of

families in their children’s early care and learning. Children were screened and mental health

services provided for children identified in need of mental health services.

1 City of Jacksonville (2009, October 28). The Jacksonville Journey Oversight Committee: An overview of the first six months. January 1 – June 30, 2009. Retrieved from http://www.coj.net/NR/rdonlyres/etrh5of2qv4dsrpnzxowjwf6fnurggvo7tpggy7wqfe33h5g2vsv4zh7y5cq4xjgbjztp4z5mgbfuqs37jha24iu4ue/JJ+ANNUAL+REPORT+Aug2009.pdf


EVALUATION PLAN

The purpose of the evaluation plan is to measure the extent that the eight expected

outcomes of Jacksonville Journey ELP were achieved. The report is organized in three sections:

Early Learning Center Outcomes, School Readiness Outcomes, and Mental Health Outcomes.

This evaluation report covers the contract period from August 2010 through September 2011.


SECTION 1: EARLY LEARNING CENTER OUTCOMES

Expected Outcome 1: At the end of the contract period (September 30, 2011), at least 21 of the 35 centers will increase their star rating by one star or maintain a 5-star rating. The Guiding Stars of Duval, an initiative of the Early Learning Coalition of Duval, Inc.,

is a quality rating system for child care centers in Jacksonville. Centers participating in Guiding

Stars are committed to program improvement targeting higher quality care and education to

ensure that all children enter kindergarten with the skills necessary to succeed. The Guiding Stars

rubric is used to evaluate centers across five domains: learning environment, staff qualifications

and professional development, ratio and group size, curriculum, and program operations (family

engagement and staff administration). The Guiding Stars rubric is used to evaluate centers across

five domains: learning environment, staff qualifications and professional development, ratio and

group size, curriculum, and program operations (family engagement and staff administration).

Table 1 depicts the variability of the weight used to calculate the star rating. Domain 1 uses a

scientifically-based tool and measures elements of process as well as structure and receives a

heavier weight than the other domains.

Table 1 Variability of Weights Used to Calculate Guiding Star Rating

Domain Weight Domain 1: Learning Environment 40% Domain 2: Staff Qualifications and Professional Development 20% Domain 3: Ratio and Group Size 10% Domain 4: Curriculum 15% Domain 5: Program Operations (including Family Involvement 15%

Initially, 40 child care centers participated in the Jacksonville Journey ELP during the

2010-11 program year. During the year, four centers closed, one center withdrew from

participation, and one was waived from Guiding Stars validation due to extenuating


circumstances. Of the remaining 34 centers, some centers increased their ratings, some ratings

decreased, and some remained the same. The star ratings, completed in August and September of

2011, are as follows:

3 centers are on a corrective plan for repeat ERS assessments,

12 centers’ star ratings remained the same,

6 centers’ star ratings dropped by 1 star,

1 center maintained its 5-star rating

4 centers improved their star rating by 1,

1 center improved its star rating by 2,

! center improved its star rating by 3,

5 centers, new to Jacksonville Journey, received a star rating of 3, and

1 center, new to Jacksonville Journey, received a star rating of 2.

Outcome

Results of the ELC Guiding Stars of Duval process document that 12 child care centers

increased their star rating by at least one star and one child care center maintained a 5-star rating.

The expected outcome was not achieved.


SECTION 2: SCHOOL READINESS OUTCOMES Expected Outcome 2: At the end of the contract period, at least 70% of Jacksonville Journey ELP School Readiness students will measure “ready for kindergarten” on Florida School Readiness measures. Expected Outcome 4: Jacksonville Journey ELP children will demonstrate positive readiness outcomes as measured by the Florida Kindergarten Readiness Screener.

Expected Outcomes 2 and 4 will be evaluated together as the children of interest in

Expected Outcome 2 are a subset of the group of children evaluated in Expected Outcome 4. The

measure used to determine “ready for kindergarten” in the first instance and “positive readiness”

in the second instance is the Florida Kindergarten Readiness Screener (FLKRS). The level of

school readiness of the 2010-11 Jacksonville Journey ELP prekindergarten children and the

subset described as School Readiness students will be assessed as kindergarteners by Duval

County Public Schools (DCPS) using the fall 2011 FLKRS. Not all 2010-2011 Jacksonville

Journey ELP prekindergarten children are assessed by DCPS. Some will not enter DCPS

kindergarten, some will no longer reside in Duval County, and some who chose not to attend

DCPS kindergarten were not served by the VPK program. DCPS will provide data for children

with sufficient identification to match their names with FLKRS scores. The evaluation will be

included in the 2012 Jacksonville Journey report. However, these outcomes were also part of the

2009-10 Jacksonville Journey ELP school year and the fall 2010 FLKRS data are available for

the 2010-11 report. We address Expected Outcome 2 and Expected Outcome 4 for the 2009-10

contract period in this report.

Florida School Readiness Screener (FLKRS)

FLKRS data were collected across all elementary school settings in the Jacksonville

community in fall 2010. Children attending public school kindergarten for the first time, were

assessed using FLKRS which is composed of two parts: the Early Childhood Observation


System (ECHOS) and the Florida Assessments for Instruction in Reading—K (FAIR-K). The

ECHOS portion of FLKRS provides an overview of development in language and literacy,

mathematics, social and personal skills, science, social studies, physical development and

fitness, and creative arts—all domains of the Florida Voluntary Prekindergarten Education

Standards. FAIR-K includes measures of letter naming and phonemic awareness, aspects of the

language and literacy domain.

Each child’s kindergarten teacher administered the ECHOS screening during the first 30

days of the school year. The readiness results of the ECHOS are calculated from the

observational ratings of the teachers. To obtain a numerical value for a total readiness score,

the observed categories Consistently Demonstrating, Emerging/Progressing, and Not Yet

Demonstrating are given values of 2, 1, and 0, respectively. The total ECHOS readiness score,

the sum of the 19 item scores, is then categorized as Consistently Demonstrating,

Emerging/Progressing, or Not Yet Demonstrating. Scores in the Consistently Demonstrating and

Emerging/Progressing categories are further classified as Ready.

The FAIR-K letter naming task is composed of 10 items requiring children to identify

the names of presented letters. The FAIR-K phonemic awareness task is composed of 10 items

involving blending or onset and rime in which the child blends words at the word part and

phoneme levels. Data are used to calculate a Probability of Reading Success (PRS) score. The

PRS score represents the probability of future reading success (defined as scoring at least at the

40th percentile). For example, a PRS score of 85% is interpreted as the child having 85%

probability of scoring at least at the 40th percentile on a future measure of reading. To calculate

the PRS score, the numbers of correct responses on the letter naming and phonemic awareness

tasks are combined using the probability matrix shown in Table 2. Cells of the matrix are color-


coded with red indicating PRS scores predicting a 15% or less probability of success on a future

measure of reading and with green indicating PRS scores predicting an 85% or better

probability of success on a future measure of reading. Scores greater than 67% are considered

Ready for kindergarten. Scores below the heavy black line in Table 2 are designated Ready and

scores above the heavy black line are designated Not Ready.

Table 2 FAIR Probability of Reading Success Scores

Participating Children with FLKRS Scores

The sample of children used to identify scores of Jacksonville Journey ELP 2009-10

prekindergarten children in the fall 2010 DCPS FLKRS kindergarten data was the children

assessed in fall 2009 and spring 2010 using the Bracken Basic Concept Scale—Third Edition:

Receptive. The 155 sampled children were enrolled in 21 childcare centers receiving assistance

from Jacksonville Journey and had informed parental consent for assessment. Fifty-one of the

children were designated School Readiness (children from low-income families receiving

subsidized childcare).


Expected Outcome 2: At the end of the contract period, at least 70% of Jacksonville Journey ELP School Readiness students will measure “ready for kindergarten” on Florida School Readiness measures. Discussion of the children’s “readiness for kindergarten” first presents characteristics of

the sample of the Jacksonville Journey ELP School Readiness students with fall 2010 FLKRS

scores. Thirty-four of the 51 2009-10 children (66.7%) had FLKRS scores. Table 3 presents the

demographic information of the students.

Table 3 Characteristics of the Sample of 2009-10 Jacksonville Journey ELP ‘School Readiness’ Prekindergarteners as Fall 2010 DCPS Kindergarteners

Characteristic n % Sex

Boys 12 35.3 Girls 22 64.7

Ethnicity Black 34 100.0

The sample of children included more girls (65%) than boys; the only ethnicity

represented was Black. At the time of the fall 2010 FLKRS assessments, the 34 Jacksonville

Journey School Readiness children from 15 of the 2009-10 Jacksonville Journey childcare

centers attended 27 DCPS schools with each school enrolling three or fewer of the 34 children.

Not all Jacksonville Journey School Readiness children had reportable scores on both

FLKRS measures: All children had reportable FAIR-K scores and 33 had reportable ECHOS

scores. Table 4 presents the scores of the children by percentage for each scoring category and

readiness status.


Table 4 FLKRS Scores of the 2009-10 Jacksonville Journey ELP ‘School Readiness’ Prekindergarten Children as 2010 DCPS Kindergarteners

Measure Category n % Kindergarten

Readiness % Ready

DCPS % Ready

Florida % Ready

ECHOS D 17 51.5 93.9 87.7 88.5 E 14 42.4 N 2 6.1 FAIR-K Ready 21 61.8 61.8 69.7 64.8 Not Ready 13 38.2

Note. ECHOS D is Demonstrating, E is Emerging/Progressing, and N is Not Yet Demonstrating More than 70% of the Jacksonville Journey ELP School Readiness children were Ready

for kindergarten based on the ECHOS measure; less than 70% of the of the Jacksonville Journey

ELP School Readiness children were Ready for kindergarten based on the FAIR-K measure. The

percentage of Jacksonville Journey ELP 2009-10 School Readiness prekindergarten children’s

ECHOS Ready scores was greater than the DCPS and Florida percentages; the percentage of

DCPS and Florida Ready FAIR-K scores exceeded the percentage of Ready scores of the

Jacksonville Journey ELP School Readiness children. None of 13 FAIR-K scores classified Not

Ready was located in the red zone of the PRS matrix shown in Table 2 on page 7 of this report.

Furthermore, of the 21 FAIR-K scores classified Ready, 14 (67%) were located in the green zone

of the same PRS matrix.

Not all Jacksonville Journey ELP 2009-10 School Readiness prekindergarten children

had reportable ECHOS and FAIR-K scores. Ninety-seven percent (33 children) had reportable

scores on both measures. Table 5 shows the percentage of children having Ready scores on both

measures, having Ready scores on either the ECHOS or FAIR-K measure, and having Ready

scores on neither measure.


Table 5 FLKRS ‘Ready’ Scores of the Jacksonville Journey ELP 2009-10 Prekindergarten ‘School Readiness’ Children as DCPS Kindergarteners

Ready Scores n % No Ready Scores 2 6.0 Only ECHOS Ready Score 11 33.3 Only FAIR-K Ready Score 0 0.0 Two Ready Scores 20 60.6

Ninety-four percent of the children had Ready scores on at least one of the two FLKRS

measures. No children had Ready FAIR-K scores but not Ready ECHOS scores while 33% of the

children had Ready ECHOS but not Ready FAIR scores. Sixty-one percent of the children had

Ready scores on both measures.

School Readiness Children in Kindergartener Summary

Of the 51 2009-10 prekindergarten Jacksonville Journey ELP School Readiness children, 34 (67%) had identifiable fall 2010 DCPS FLKRS scores.

Sixty-five percent of the children were girls and all of the children were Black.

The children were assessed using the FLKRS measures in fall 2010 in 27 DCPS schools.

Ninety-four percent of the School Readiness kindergarteners had Ready scores on the ECHOS measure. This percentage surpasses the expected outcome, 70%.

Sixty-two percent of the School Readiness kindergarteners had Ready scores on the FAIR-K measure. This percentage does not meet the expected outcome, 70%.

Outcome

More than 70% of the Jacksonville Journey ELP School Readiness children were Ready

for kindergarten based on the ECHOS; less than 70% of the Jacksonville Journey ELP School

Readiness children were Ready for kindergarten based on the FAIR-K.


Expected Outcome 4: Jacksonville Journey ELP children will demonstrate positive readiness outcomes as measured by the Florida Kindergarten Readiness Screener.

Discussion of the children’s positive readiness outcomes will first present the

characteristics of the sample of 2009-10 Jacksonville Journey ELP prekindergarten children with

identifiable scores in the fall 2010 DCPS FLKRS data. One hundred two of the 155 (66%)

children had scores. Table 6 presents demographic information for the sample.

Table 6 Characteristics of the Sample of 2009-10 Jacksonville Journey ELP Prekindergarteners as Fall 2010 DCPS Kindergarteners

Characteristic n % Sex

Boys 49 48.0 Girls 53 52.0

Ethnicity Black 96 94.1 Asian 1 1.0 Mixed 1 1.0 White 4 4.0

The sample of children included more girls (52%) than boys; the largest ethnicity

represented was Black (94%). At the time of the fall 2010 FLKRS assessments, the 102 children

who had attended prekindergarten in 21 Jacksonville Journey childcare centers were assessed in

67 DCPS schools. Table 7 presents the names of the six DCPS schools enrolling at least three

2009-10 Jacksonville Journey ELP prekindergarten children.

Table 7 Elementary or K-8 Schools Enrolling at Least Three Jacksonville Journey ELP Children as 2010 DCPS Kindergarteners

School Name n Chaffee Trail 3 Crystal Springs 3 Loretto 5 North Shore 3 Sadie T. Tillis 4 Twin Lakes Academy 4


Not all 102 Jacksonville Journey children had reportable scores on both FLKRS

measures, but all had a reportable score on at least one measure. Two children did not have

reportable ECHOS scores and three did not have reportable FAIR-K scores. Table 8 presents the

scores of the children by percentage for each scoring category and readiness status.

Table 8 Kindergarten 2010 FLKRS Scores of the 2009-10 Jacksonville Journey ELP Prekindergarten Children

Measure Category n % Kindergarten

Readiness % Ready

DCPS % Ready

Florida % Ready

ECHOS D 46 46.0 89.0 87.7 88.5 E 43 43.0 N 11 11.0 FAIR-K Ready 65 65.7 65.7 69.7 64.8 Not Ready 34 34.3

Note. ECHOS D is Demonstrating, E is Emerging/Progressing, and N is Not Yet Demonstrating More than 70% of the Jacksonville Journey ELP children were Ready for kindergarten

based on the ECHOS measure; less than 70% of the of the Jacksonville Journey ELP children

were Ready for kindergarten based on the FAIR-K measure. The percentage of Jacksonville

Journey ELP 2009-10 prekindergarten children’s ECHOS Ready scores was more than the DCPS

and Florida percentages; the percentage of DCPS but not Florida Ready FAIR-K scores exceeded

the percentage of Ready scores of the Jacksonville Journey ELP children. Of the 34 FAIR-K

scores classified Not Ready, one (2.9%) score was located in the red zone of the PRS matrix

shown in Table 2 on page 7 of this report. Furthermore, of the 65 FAIR-K scores classified

Ready, 44 (67.8%) were located in the green zone of the same PRS matrix.

Ninety-five percent of the children (97) had reportable scores on both FLKRS measures.

Table 9 shows the percentage of children having Ready scores on both measures, having Ready

scores on either the ECHOS or FAIR-K measure, and having Ready scores on neither measure.


Table 9 FLKRS ‘Ready’ Scores of the Jacksonville Journey ELP 2009-10 Prekindergarten Children as 2010 DCPS Kindergarteners

Ready Scores n % No Ready Scores 9 9.3 Only ECHOS Ready Score 24 24.7 Only FAIR-K Ready Score 1 1.0 Two Ready Scores 63 64.9

Ninety-one percent of the children had Ready scores on at least one of the two FLKRS

measures. One percent of the children had Ready FAIR-K scores but not Ready ECHOS scores

while 25% of the children had Ready ECHOS but not Ready FAIR-K scores. Sixty-five percent

had Ready scores on both measures.

School Readiness Children in Kindergarten Summary

Of the 155 Jacksonville Journey ELP 2009-10 prekindergarten children, 102 (66%) had fall 2010 DCPS FLKRS scores.

Fifty-two percent of the children were girls and 96% were Black.

The children were assessed using the FLKRS measures in fall 2010 in 67 DCPS schools.

Eighty-nine percent of the Jacksonville Journey ELP kindergarteners had Ready scores on the ECHOS measure. Across DCPS schools, 88% of the ECHOS scores were Ready.

Sixty-six percent of the Jacksonville Journey ELP kindergarteners had Ready scores on the FAIR-K measure. Across DCPS schools, 70% of the FAIR-K scores were Ready scores.

Outcome

A sample of Jacksonville Journey ELP 2009-10 prekindergarteners with fall 2010 DCPS

FLKRS scores demonstrated positive readiness outcomes measured by the ECHOS scores—89%

had Ready scores. FAIR-K results indicated that 66% of the children’s scores were Ready. The

sample of children with both ECHOS and FAIR-K scores indicated that scores of 65% of the

children were Ready on both measures. As no benchmark is provided for this outcome, we will

leave it to the reader to determine whether these outcomes constitute positive outcomes.


Expected Outcome 3: Jacksonville Journey Early Learning Program children will demonstrate positive readiness outcomes as measured by the Bracken Basic Concept Scale—Third Edition: Receptive. Expected Outcome 5: Children enrolled in the Jacksonville Journey Early Learning Program childcare centers will demonstrate significant and meaningful gains in early literacy achievement. Expected Outcomes 3 and 5 will be reported together as the random sample of children

used to evaluate Expected Outcome 3 is the same sample used to evaluate Expected Outcome 5.

However, different measures were used to assess “positive school readiness” and “early literacy

achievement.”

Participating Children

Classes in all Jacksonville Journey ELP participating early learning and care centers were

stratified by the children’s age as preschoolers (children eligible for public school kindergarten

during the 2012-13 school year) or as prekindergarteners (children eligible for public school

kindergarten during the 2011-12 school year). The criterion used for the selection of children

from classes required the random selection of at least four and no more than eight children with

informed consent from no more than two classes serving preschool children and no more than

two classes serving prekindergarten children at each childcare center. This sampling resulted in

statistically usable assessments from 139 preschoolers and 165 prekindergarteners.

Data were collected in fall 2010 from October through late November and, in spring 2011,

from late April through May. Fall assessments were collected from children enrolled in 30

Jacksonville Journey ELP childcare centers. However, the preschool children (184) were

enrolled in only 27 of the 30 childcare centers, and the prekindergarten children (205) were

enrolled in 28 of the 30 centers. The remaining 10 Jacksonville Journey childcare centers either

enrolled too few preschool and/or prekindergarten children or enrolled only infants and toddlers.


In spring, preschool children who were assessed in the fall were available in 26 childcare centers

and prekindergarten children who were assessed in the fall were available in 27 childcare

centers. The attrition of sites was due to the closing of one site enrolling sampled children during

the 2010-11 school year.

Table 10 presents information describing the final sample of 139 preschool and 165

prekindergarten children. Ethnicity data were collected across five categories; however, three

categories (Hispanic, Mixed, and Other) were collapsed into one category, Other, because of

their small representation. The sample of children included more girls than boys, and almost all

of the children were Black. Additionally, 42% of the children were School Readiness children

(children from low-income families receiving subsidized child care).

Table 10 Characteristics of the Sampled Preschool and Prekindergarten Children

Characteristic All Children Preschool Prekindergarten n % n % n %

Sex Boys 146 48.0 69 49.6 77 46.7 Girls 158 52.0 70 50.4 88 53.3

Ethnicity Black 284 93.4 133 95.7 151 91.5 White 16 5.3 5 3.6 11 6.7 Other 4 1.3 1 0.7 3 1.8

School Readiness Yes 128 42.1 69 49.6 59 35.8

Measures

The extent to which children enrolled in Jacksonville Journey ELP childcare centers

demonstrated significant and meaningful gains in school readiness and early literacy

achievement was determined by whether sampled children made statistically significant fall to

spring gains on the selected measures of school readiness and early literacy. The Bracken Basic


Concept Scale—Third Edition: Receptive (BBCS-3:R)2 was used to assess school readiness. The

Test of Preschool Early Literacy (TOPEL)3 and the Preschool Word and Print Awareness

(PWPA)4 measures were used to assess important components of children’s early reading

readiness. Florida Institute of Education-trained assessors administered the BBCS-3:R, TOPEL,

and PWPA in one-on-one settings.

BBCS-3:R

The BBCS-3:R is a battery of 10 scales designed to measure understanding of concepts

that traditionally have been thought relevant to the development of young children. The first five

scales form the School Readiness Composite (SRC) and assess foundational concepts that are

frequently and intentionally taught to children before and during formal schooling. These

concepts include colors, letters, numbers, shapes, and size. The remaining scales

(Direction/Position, Self-/Social Awareness, Texture/Material, Quantity, and Time/Sequence)

measure basic concepts that are less frequently taught at home and more often taught in formal

school settings. These concepts are incorporated in Florida early childhood standards.

The Direction/Position scale measures relational terms that describe position and

placement of objects including positional words (inside, outside, and above), directional words

(near, far, right, and left), and properties of motions (backward and side-to-side). The Self-

/Social Awareness scale measures knowledge of concepts that describe emotional states and

kinship including personal characteristics (gender and age) and roles of the family. The

Texture/Material scale measures knowledge of concepts that describe attributes of objects 2 Bracken, B. A. (2006). Bracken Basic Concept Scale-Third Edition: Receptive. San Antonio, TX: Harcourt

Assessment, Inc. 3 Lonigan, C. J., Wagner, R. K., & Torgesen, J. K. (2007). Test of Preschool Early Literacy. Austin, TX: Pro-Ed. 4 Justice, L. M., Bowles, R. P., & Skibbe, L. E. (2006). Measuring preschool attainment of print-concept knowledge:

a study of typical and at-risk 3- to 5-year-old children using item response theory. Language, Speech, and Hearing Services in School, 37, 224-235.


including difference in textures (smooth and rough) and sorting objects by texture. The Quantity

scale measures knowledge of concepts that describe quantity and how it can be manipulated

including part/whole relationships (whole, half, and piece), language that compares quantity

(more, several, less, greater than), and comparison of objects by capacity/volume (empty, full,

and enough). The Time/Sequence scale measures concepts that describe occurrences on a

continuum (such as calendar events) and the degree of speed and the time of day (morning and

night), vocabulary related to time (before and after), relative position sequence (ordinal

numbers), and sequence of events (beginning, middle, and end).

TOPEL

The TOPEL measures abilities related to early literacy including print knowledge,

definitional vocabulary, and phonological awareness―abilities correlated with decoding and

reading comprehension. Print Knowledge is a 36-item scale measuring knowledge of the nature

and purposes of printed words and alphabet knowledge. Children are asked to point to

representations of the correct response to verbal stimuli and to produce letter sounds.

Definitional Vocabulary is a 35-item scale simultaneously measuring surface and deep

vocabulary knowledge. The child, upon being shown a picture, is asked to say what the picture

represents and to describe an important attribute of what is pictured. Phonological Awareness is

a 27-item scale designed to measure elision and blending abilities. For the first elision items, the

child looks at a page of pictures as the examiner names each object. The examiner then says a

word and asks the child to point to the picture of what is left when a sound is dropped. For the

beginning blending ability items, the examiner asks the children to combine sounds to form a

word by pointing to the picture corresponding to the newly formed word. For both types of

items, higher numbered, more difficult items are presented without pictures. The Early Literacy


Index is a composite of the three scales and, according to the test developers, is the single best

measure of children’s early literacy.

PWPA

PWPA is a measure of children’s knowledge of 14 concepts of print including

identification of the front and back of a book; knowledge that print is read from right to left and

top to bottom; and differentiation of letters as a unit of print, first letter of a word, and capital and

lowercase letters. The assessment is conducted in an authentic setting with an adult engaging the

child in a shared reading of a commercially available book.

Standard Scores

Children undergo rapid development over the age span covered by the assessments;

therefore, raw scores are converted to standard scores relative to a normative group of children

covering the age range of the test. This process adjusts scores for natural maturation and allows

detection of gains resulting from programs. A change or lack of change in standard scores is

always relative to the normative populations and does not represent an absolute gain in

knowledge. Table 11 shows the ranges of the BBCS-3:R, TOPEL, and PWPA scores relative to

the national normative population.

Table 11 BBCS-3:R, TOPEL, and PWPA Scores by Ability Categories

Score Categories

Very Poor

Poor Below Average

Average Above Average

Superior Very Superior

BBCS-3:R Below 4

4-5 6-7 8-12 13-14 15-16 17-20

TOPEL Below 70

70-79 80-89 90-110 111-120 121-130 Above 130

PWPA Below 70

70-79 80-89 90-110 111-120 121-130 Above 130

Percentile Ranking

2nd or lower

3rd to 9th

10th to 25th 26th to 75th

76th to 90th 91st to 97th

98th or higher


Standard BBCS-3:R scores have a mean of 10 and a standard deviation of 3. Scores

below 4 are classified Very Delayed, scores ranging from 4 to 6 are classified Delayed, scores

ranging from 7 to 13 are classified Average, scores ranging from 14 to 16 are classified

Advanced, and scores greater than 16 are classified Very Advanced. TOPEL and PWPA scores

have a mean of 100 and a standard deviation of 15.

Data Analyses

The Jacksonville Journey ELP sampling design used a stratified sampling frame because

children were nested in centers; the children learned together in classes. Hierarchical Linear

Modeling (HLM) accounts for this dependency in data and was used to analyze the data. The

two-level HLM analysis (with data modeled at the child- and center-levels) confounds the impact

of teacher and center, but is a sufficient model for such a small sample.

Expected Outcome 3: Jacksonville Journey Early Learning Program children will demonstrate positive readiness outcomes as measured by the Bracken Basic Concept Scale—Third Edition: Receptive. Discussion of the school readiness results will first present analysis of child-level attrition,

then summary descriptive statistics from the BBCS-3:R scores, and finally analytic results.

Attrition

The study of attrition of the participating children was based on the children’s BBCS-3:R

scores. A determination of the importance of attrition involved investigating whether the attrition

was random or systematic. Systematic attrition can result in a final sample of children that is

different from the original sample relative to children’s initial school readiness or demographic

characteristics. The BBCS-3:R fall scores of the participating children who were not

administered the BBCS-3:R in the spring (incomplete) were compared to fall scores of children

who were administered the BBCS-3:R in the spring (complete). Table 12 presents the BBCS-3:R


SRC fall mean scores of the preschool and prekindergarten children with complete and

incomplete data and the results of the analysis of potential differences in their scores.

Table 12 BBCS-3:R SRC Fall Mean Scores for Preschool and Prekindergarten Children with Complete and Incomplete Scores

Preschool Prekindergarten Complete Incomplete Complete Incomplete n M n M p-value n M n M p-value

139 8.22 45 8.73 .312 165 9.16 40 7.33 .001

The number of incomplete BBCS-3:R fall scores represents attrition of 24% for

preschoolers and 20% for prekindergarteners. The analysis of the BBCS-3:R SRC scores

indicated that the SRC fall mean score of the preschool children who did not complete the school

year in their original classes was not statistically different from the SRC fall mean score of

children who did complete the year in their original classes. However, the SRC fall mean score

of the prekindergarten children who did not complete the school year in their original classes was

statistically less than the SRC fall mean score of children who did. Thus, there is evidence that

attrition was systematic relative to the prekindergarten children’s initial school readiness

achievement. Results from further analysis of this scale should be interpreted with caution.

In fall and spring, prekindergarten children (205 children) were also administered the five

BBCS-3:R scales that do not form the SRC. Table 13 presents the BBCS-3:R fall mean scores of

the prekindergarten children with complete and incomplete data and the results of the analysis of

potential differences in their mean scores.


Table 13 Fall Mean Scores for Children with Complete and Incomplete Scores

Scale Complete Incomplete M SD M SD p-value

Direction/Position 8.97 2.64 7.15 2.29 .003*** Self-/Social Awareness 8.22 2.14 7.68 2.75 .276 Texture/Material 8.49 2.37 7.43 2.30 .073* Quantity 8.58 2.40 7.20 2.56 .218 Time/Sequence 8.80 1.95 7.85 2.38 .039**

Note. n = 165 BBCS-3:R complete scores; n = 40 incomplete BBCS-3:R scores. *** indicates p < 01; ** indicated p < .05; and * indicated p < .10.

The analysis of the BBCS-3:R scores indicated that the Direction/Position,

Texture/Material, and Time/Sequence scales fall mean scores of children who did not complete

the school year in their original classes were statistically different from the fall mean scores of

children who did complete the year. Therefore, one should use caution in interpreting results on

these scales as the children who were not available for the spring assessment had lower scores,

on average, than the children who were available.

Results

Table 14 presents the summary statistics and HLM results for the BBCS-3:R scales, and

as can be seen, on all scales with the exception of the BBCS-3:R Texture/Material and

Time/Sequence scales, the children made statistically significant fall to spring gains. Table 14

also reports effect sizes, and BBCS-3:R effect sizes ranged from a low of 16% of a standard

deviation to a high of 33% of a standard deviation.


Table 14 BBCS-3:R, Summary Statistics and HLM Analyses Results

Scale Fall Spring

Center Varianc

e

Effect Size

M SD M SD p % School Readiness Composite (3) 8.22 2.96 9.20 2.85 <.001*** 17.4 .327 School Readiness Composite (4) 9.16 3.00 9.79 2.49 <.001*** 0.9 .210 Direction/Position 8.49 2.66 8.97 2.46 .001*** 3.2 .160 Self-/Social Awareness 8.17 2.54 8.82 2.14 .002*** 1.6 .217 Texture/Material 8.21 2.49 8.49 2.37 .111 0.0 Quantity 7.75 2.56 8.58 2.39 <.001*** 3.4 .277 Time/Sequence 8.70 2.33 8.80 1.95 .567 1.3

Note. Effect sizes are based on the BBCS-3:R normative population. Effect sizes of 20 to 49 percent of a standard deviation are small, between 50 and 79 percent of a standard deviation are medium, and 80 percent or more of a standard deviation are large; n = 165 BBCS-3:R scores. ***p < .01;**p < .05; and *p < .10. Very little of the variance in the children’s BBCS-3:R scores was accounted for by

childcare center contexts with the exception of the preschool children’s SRC scores, and there

the context of the center really mattered. For the other BBCS-3:R scales, the findings suggest

that the childcare center context did not account for any substantial, measureable amount of the

variance in the children’s scores. Thus, the mean gain would have occurred regardless of any

perceived differences in the context of the sampled childcare centers.

Another way to look at the impact of this achievement is to look at the percentage of

children’s scores categorized from Very Delayed to Very Advanced. (See page 19.) Figure 1

shows the percentage of scores in the BBCS-3:R SRC achievement categories in the fall and

spring. As can be seen, 80% of the preschool children and 96% of the prekindergarten children

exhibited at least Average school readiness at the end of the 2010-11 school year.


Figure 1. The percentage of children’s BBCS-3:R SRC scores categorized as Very Delayed to Very Advanced in the fall and spring of the school year (n = 139 preschool and 165 prekindergarten children).

Another way to look at the school readiness achievement of the Jacksonville Journey ELP

children is to look at the fall to spring differences in the percentile rankings of the BBCS-3:R

mean scores (Figure 2).


Figure 2. Summary of the fall to spring changes in the percentile rankings of the BBCS-3:R mean scores of the preschool children and the prekindergarten children. TEX* and TIME* indicate that the fall to spring gains were not statistically different than zero. All spring means scores ranked at least at the 31st percentile. The spring SRC mean score

of the prekindergarten children was the only BBCS-3:R scale mean score ranked higher than the

40th percentile in the spring. The lowest ranked spring mean scores were the TEX* and QUA

mean scores, 31st and 32nd percentile, respectively. The largest statistically significant fall to

spring differences in the percentile rankings were the preschool children’s SRC mean scores (11

percentile increase) and for the prekindergarten children’s Quantity mean scores (9 percentile

increase).

28

39

31

2728

23

33

39

47

3735

3132

34

0

5

10

15

20

25

30

35

40

45

50

Preschool SRC Pre-K SRC DIR SELF TEX* QUA TIME*

Perc

entil

e Ra

nkin

g of

Fal

l and

Spr

ing

Mea

n Sc

ores

fall spring

BBCS-3:RBBCS-3:RSRC


Summary

Analysis of the BBCS-3:R SRC fall to spring mean scores indicated that the preschool and prekindergarten children made a statistically significant fall to spring gains with resulting effect size of 33 and 21 percent of a standard deviation, respectively.

Analysis of the BBCS-3:R SRC mean scores indicated an increase in fall to spring

percentile ranking of the SRC mean scores from the 28th to the 39th percentile for the preschool children’s scores and from the 39th to the 47th percentile for the prekindergarten children’s scores.

Eighty percent of the preschool and 96% of the prekindergarten children’s SRC scores

were categorized as Average or better at the end of the 2010-11 school year.

The prekindergarten children’s scores, on average, indicated that they made statistically significant fall to spring gains on the Direction/Position, Self-/Social Awareness, and Quantity scales with resulting effect sizes of 16, 22, and 28 percent of a standard deviation.

Analyses of the prekindergarten children’s BBCS-3:R mean scores indicate that the Direction/Position, Self-/Social Awareness, and Quantity mean scores increased from fall to spring in percentile ranking from the 31st to the 37th, from the 27th to the 35th, and from the 23rd to the 32nd percentiles, respectively.

Outcome

Results of the analyses of the Jacksonville Journey ELP data indicated that Jacksonville

Journey ELP children demonstrated positive readiness outcomes measured by the Bracken Basic

Concept Scale—Third Edition: Receptive. The preschool children’s spring mean SRC score

ranked at the 39th percentile and 80% of their scores were categorized as Average or better. The

prekindergarten children’s spring mean SRC score ranked at the 47th percentile and 96% of their

scores were categorized as Average or better. The remaining five BBCS-3:R scores of the

prekindergarten children indicate a need for greater improvement.


Expected Outcome 5: Children enrolled in the Jacksonville Journey Early Learning Program childcare centers will demonstrate significant and meaningful gains in early literacy achievement. Discussion of the early reading achievement results will first present analysis of child-

level attrition, then summary descriptive statistics from the TOPEL and PWPA scores, and

finally analytic results.

Attrition

Table 15 presents the TOPEL and PWPA fall mean scores of the Jacksonville Journey

ELP children who were available for spring assessments (complete) and who were not available

(incomplete). The 40 children who were not tested in the spring represent an overall attrition rate

of 20%. However, the TOPEL Early Literacy Index, Print Knowledge, and Definitional

Vocabulary fall mean scores of children who did not complete the school year in their original

classes were statistically less than the fall mean scores of children who did. Thus, there is

evidence that attrition was systematic relative to the prekindergarten children’s early literacy

achievement. Results from further analysis of this scale should be interpreted with caution.

Table 15 Fall Mean Scores for Children with Complete and Incomplete Early Literacy Scores

Scale Complete Incomplete M SD M SD p-value

TOPEL Early Literacy Index 101.72 13.44 87.45 15.37 .006*** Print Knowledge 109.21 10.70 95.83 16.54 .003*** Definitional Vocabulary 96.19 12.15 90.23 13.41 .026** Phonological Awareness 99.90 16.51 85.33 16.51 .186

PWPA 95.27 13.08 94.26 13.16 .662

Note. n = 165 TOPEL and PWPA complete scores; n = 40 incomplete scores. *** indicates p < 01; ** indicates p < .05.


Results

Table 16 presents the summary statistics and HLM results for the TOPEL and PWPA

scales, and, as can be seen, on all scales with the exception of the TOPEL Definitional

Vocabulary scale, the children made statistically significant fall to spring gains. Table 16 also

reports effect sizes, and TOPEL effect sizes ranged from a low of 23% to a high of 47% of a

standard deviation. The children made statistically significant fall to spring gains on the PWPA

scale (effect size 74% of a standard deviation).

Table 16 TOPEL and PWPA Summary Statistics and HLM Analyses Results

Scale Fall Spring

Center Varianc

e

Effect Size

M SD M SD p % TOPEL

Early Literacy Index 94.61 14.75 101.72 13.44 <.001*** 20.6 .474 Print Knowledge 104.15 15.41 109.21 10.70 <.001*** 15.1 .337 Definitional Vocabulary 95.30 12.67 96.19 12.15 .251 11.6 Phonological Awareness 88.95 15.27 99.90 16.51 <.001*** 20.6 .230

PWPA 95.27 13.04 106.39 16.81 <.001*** 7.0 .741

Note. Effect sizes are based on the TOPEL and PWPA normative population. Effect sizes of 20 to 49 percent of a standard deviation are small, between 50 and 79 percent of a standard deviation are medium, and 80 percent or more of a standard deviation are large; n = 165 TOPEL and PWPA scores. ***p < .01;**p < .05; and *p < .10. Between 12% and 21% of the variance in the children’s TOPEL scores was accounted for

by childcare center contexts. These findings suggest that the childcare center context did account

for a substantial, measureable amount of the variance in the children’s scores. Thus, the mean

gain was dependent on elements of the context, including classroom quality, of the sampled

childcare centers. The portion of the variance in the children’s PWPA scores accounted for by

childcare contexts was small.


Another way to look at the early literacy achievement of the children enrolled in

Jacksonville Journey childcare centers is to look at the fall to spring differences in the percentage

of Early Literacy Index scores ranked in the TOPEL achievement categories. (See page 18, Table

11 for the TOPEL categories.) Figure 3 shows these differences.

Figure 3. The distribution of TOPEL Early Literacy Index fall and spring scores (n = 165).

In spring, 18% of the TOPEL Early Literacy Index scores ranked in the bottom quartile

of national scores (Very Poor through Below Average), and 35% of the TOPEL Early Literacy

Index scores ranked in the top quartile of national scores (Above Average through Very

Superior).

4.2

15.2 15.2

51.5

12.1

1.2 0.62.4

4.9

10.3

54.6

24.2

3.6

0.00

10

20

30

40

50

60

Very Poor Poor Below Average Average Above Average Superior Very Superior

Perc

ent o

f Fal

l and

Spr

ing

TOPE

L Ea

rly L

itera

cy In

dex

Scor

es b

y Ca

tego

ry

fall spring


Changes in Percentile Rankings of the Mean Scores

Another way to look at the early literacy achievement of the children enrolled in

Jacksonville Journey childcare centers is to look at the fall to spring differences in the percentile

rankings of the TOPEL and PWPA mean scores (Figure 4).

Figure 4. The percentile ranking of fall and spring mean scores if the fall to spring gain was statistically different than zero. DV* indicates that the gain was not statistically different than zero. The largest gain in percentile ranking (28 percentiles) was for the PWPA, however, the

standard PWPA scores are the same for all prekindergarten children regardless of their age.

Thus, the process used to develop standard scores for this measure does not adjust the score for

children’s natural development. Among the TOPEL scores, the largest gains in percentile

rankings (27 percentile gain) occurred for TOPEL Phonological Awareness scale. The TOPEL

36

61

38

23

38

55

73

4850

66

0

10

20

30

40

50

60

70

80

ELI PK DV* PA PWPA

Perc

entil

e Ra

nkin

gs o

f the

Fal

l and

Spr

ing

Scor

es

fall spring

PrintConceptsTOPEL


Print Knowledge spring mean score ranked at the 73rd percentile, and both the fall and spring

Print Knowledge mean scores ranked above the national normative population mean score, the

50th percentile. All spring mean scores close to or above the 50th percentile (the national mean),

and the lowest ranking was the 48th percentile.

Summary

The children demonstrated statistically significant gains on the TOPEL Early Literacy Index, Print Knowledge, and Phonological Awareness scales with effect sizes of 47%, 34%, and 73% of a standard deviation, respectively.

The TOPEL Early Literacy Index, Print Knowledge, and Phonological Awareness spring

mean scores ranked at the 55th, 73rd, and 50th percentile, respectively.

The children demonstrated statistically significant gains on the PWPA measure, and, in the spring, the PWPA mean score ranked at the 66th percentile.

Outcome

Results of the analyses of support the finding that prekindergarten children enrolled in the

Jacksonville Journey ELP childcare centers demonstrated significant and meaningful gains in

early reading achievement based specifically on the statistically significant gains on the TOPEL

Early Literacy Index. The Early Literacy Index spring mean score ranked at the 55th percentile,

above the national average, and 82% of the children’s scores ranked Average or higher. The

PWPA spring mean score ranked at the 66th percentile, above the national average.


SECTION 3: MENTAL HEALTH OUTCOMES

Context and Overview of Mental Health Services

The mental health services implemented by the Child Guidance Center were delivered to

the 37 Jacksonville Journey ELP child care centers. The intent of the Jacksonville Journey ELP

services was to impact the educational development and social/emotional development of the

participating Jacksonville Journey ELP children.

Measurement Instruments Three screening and assessment instruments5 were used in the project:

1. Ages & Stages Questionnaires® (ASQ-3) 2. Ages & Stages Questionnaires®: Social-Emotional (ASQ:SE) 3. The Achenbach Child Behavior Checklist 1.5-5 (CBCL)

The Achenbach Systems of Empirically Based Assessments (ASEBA) is a comprehensive

evidence-based assessment system developed to assesses competencies, adaptive functioning,

and behavioral, emotional, and social problems from age 1½ to over 90 years. The preschool

forms and profiles span ages 1½ -5 years. The forms seek information regarding parents’,

daycare providers’, and teachers’ ratings of 99 problem items plus descriptions of problems,

disabilities, what concerns parents or respondent most about the child, and the best things about

the child. The title of the preschool form is Child Behavior Checklist for Ages 1½ -5.6

5 Bricker, D., & Squires, J. (with Mounts, L., Potter, L., Nickel, B., Twombly, E., & Farrel, J.). (1999). Ages and Stages Questionnaires® ): A Parent Completed, Child-Monitoring System (3rd ed.). Baltimore: Paul Brookes. Squires, J., Bricker, D., Twombly, E. (with Yockelson, S., Davis, M.S., & Kim, Y.). (2002). Ages & Stages Questionnaires®: Social-Emotional: A Parent Completed, Child-Monitoring System for Social-Emotional Behaviors. Baltimore: Paul H. Brookes. 6 Achenbach T., (1999) ASEBA. Caregiver-Teacher Report Form for Ages 1.5-5. Burlington, VT: University of Vermont; Achenbach, T., Rescorla, L. (2000) ASEBA Child Behavior Checklist for Ages 1.5-5. Burlington, VT: University of Vermont. Retrieved November 1, 2010 from http://www.aseba.org/preschool.html


Continuum of Services

The Jacksonville Journey ELP continuum of services employs a wide array of strategies

involving the children, family members, caregivers, and center staff. Five steps guided the

implementation of early intervention services and the identification of children at risk of

developmental delays and social-emotional concerns:

Awareness and Consent Process Screening Process and Results Services and Supports for Families, Children, and Teachers Referral Process Evaluation of Service Delivery Consent Collection and Screening Process In October 2010, approximately 1,501 children were enrolled in the participating

Jacksonville Journey ELP child care centers. Informed consents were distributed to all families.

In an effort to increase awareness, a definition of screening and assessment was included on the

consent form. The intended goal (better understanding the child’s physical and social-emotional

development) was also highlighted to encourage families and teachers to complete two

questionnaires—the ASQ:SE and the ASQ-3. Of the approximately 1,501 children enrolled in

Jacksonville Journey ELP child care centers, 1,383 consent forms were returned.

Once informed consents were collected, the ASQ:SE and/or the ASQ-3 questionnaires

were distributed to the families and teachers of children with informed consents. A letter,

providing more information about the purpose and procedures of screening, was attached to each

screening questionnaire. This process began in the fall and was continued as new centers were

identified. These questionnaires were collected and scored by CGC Early Childhood

Consultants. Of the 938 children with ASQ-SE questionnaires completed by a family member,


teacher, or both, 81 required further action (e.g., additional referral services and/or support). Of

the 222 children with completed ASQ-3 questionnaires, 83 required further action.

Services and Support for Children, Families, and Teachers

If children are to be successful, the active engagement of families and teachers working

together to provide high-quality early learning and language experiences is critical. The

Jacksonville Journey ELP Early Childhood Consultants worked closely with families and

teachers to facilitate communication and provide activities and strategies they could use to

enhance children’s early learning and social-emotional development.

Services and Support for Families

Upon completion of the screening process, letters were sent to all families

communicating the results and inviting further discussion. Jacksonville Journey ELP Early

Childhood Consultants worked with families to provide additional information, to provide access

to needed services (e.g., health, speech, further evaluation) for their children, and to initiate the

development of specific plans addressing the children’s identified needs.

A federal law, the Individuals with Disabilities Education Act (IDEA), makes services

available for infants, toddlers, and preschoolers with disabilities. Each state outlines an early

intervention system that provides specialized health, education, and therapeutic services to

children from birth to 36 months by accessing Part C of the IDEA, or Early Intervention. States

individually decide on the label for specific early intervention system; in Florida, Early Steps, a

recognized state program, provides a range of services to children at-risk of having

developmental concerns or disabilities and to their families. When a child meets the criteria, the

family’s concerns, resources, goals, and the evaluation results guide the provision of services.


Some children transition out of the early intervention system after receiving the needed

supports as they catch up and function within typical ranges of other children their age. Others

transition into the next phase of services or are identified at 3 years of age. When a 3-year-old

child is in need of services, services are made available through the public school system, using

Part B of the IDEA. In Florida, services from the school system are available from the Florida

Diagnostic and Learning Resource System (FDLRS), and children needing services are referred

to Child Find. The services are structured to address the education needs and possible

developmental delays that challenge children’s ability to learn. Child Find assists in matching

education programs (including community supports and parent consultations) with the children’s

needs. During the year, 9 referrals were made to Early Steps and 10 referrals were made to Child

Find. The CGC Early Childhood Consultants also facilitated communication between families

and teachers.

Services and Support for Children

Key to children’s success involves a coordinated system of support. Jacksonville Journey

ELP Early Childhood Consultants developed plans that coordinated support for children among

key partners—teachers, families, and Early Childhood Consultants. The Individual Positive

Behavior Plans (IPBPs) addressed social-emotional needs of children identified by the ASQ:SE

or classroom observations in need for further intervention. The plans, designed to engage

children in specific activities addressing identified needs, were developed in consultation with

families and teachers. Jacksonville Journey ELP Early Childhood Consultants monitored

children’s progress and regularly communicated with families and teachers to facilitate

children’s continued positive development.


Services and Support for Teachers

Teachers were invited to attend center- and/or community-based training sessions

provided by ECS, CGC, and ELC. The sessions provided an array of strategies that teachers

could use to support children’s social-emotional development. In-depth training sessions focused

on research-based strategies for classroom organization and management. The Jacksonville

Journey ELP Early Childhood Consultants visited centers regularly and provided teachers

individualized support and training as requested.

Referral Process

Based on the degree of risks indicated on the ASQ:SE and referrals from the director,

teacher, coach, or parent, the CGC Early Childhood Consultant initiated the evaluation process

that included observing the child and meeting with the teacher. Following a teacher/director

conference the Early Childhood Consultant contacted the parent to schedule an appointment. If

the parent/guardian was interested in individual therapeutic services an evaluation including the

CBCL was completed. From October 1, 2010, through September 30, 2011, 29 children from 28

families were referred to Child Guidance. During the year, the cases of 27 children in 24 families

were closed.

CGC provided therapeutic services in the child care center, at home, and in the office

depending on the child’s needs and the parent’s preference. Services always involved contact

with the parent/guardian, the constant source of support in the child’s life. Services to address the

children’s needs were individualized and took into consideration the context of the child, family,

and child care center. Child Guidance Center activities for 2010-11 are reported in Table 17.


Table 17 Summary of Child Guidance Center Activities during 2010-11

Outcome # Children

# Families

2010/11 Referrals Declined Services 3 3 Referral Withdrawn or Child Withdrawn from Center 3 3 Unable to Contact 0 0 In process of Contacting 0 0 Open Cases 23 22 Referral Total 29 28

Cases Closed in 2010/11 Closed after 3 or More Months of Service (Post CBCL)

Cases Carried over from 2009 6 5 New Cases 7 7

Less Than 3 Months of Service (No Post CBCL) 14 12 Closed-Case Total 27 24

Cases Carried over to 2011/12 Open Cases 10 10 Cases in Process of Opening 4 4 Carried-Over Total 14 14

Expected Outcome 6: 85% of clients will improve their social/emotional functioning as evidenced by CBCL pre/post assessment.

Findings

Response to Expected Outcome 6 was determined by the CBCL pre/post assessment used

to rate children’s functioning. Higher scores represent more impaired functioning. CBCLs were

completed at the beginning and end of treatment for children who remained in treatment for at

least 3 months. Thirteen children received 3 or more months of service and completed treatment

during the 2010-2011 year. Of those 13, 12 showed improvement (a lower score on the scales

that were of most concern); one stayed the same. The percentage of children showing

improvement was calculated by counting the number of children who had an improved score on

the post CBCL (12 children) divided by the number of children with pre and post scores (13

children).


Outcome

The percentage of children showing improved social/emotional functioning as evidenced

by CBCL pre/post assessment was 92.3% which represents more than the 85% expected

outcome.

Expected Outcome 7: 85% of parents and teachers will indicate that the mental health services received were beneficial, via survey instruments.

Three survey instruments were developed to respond to Expected Outcome 7. Each

survey instrument addresses the perception of the benefit of services provided to children. The

Teacher Benefit Survey has five items and four response options: Not Beneficial, Somewhat

Beneficial, Beneficial, and Very Beneficial. The Family Benefit Survey has five items and four

response options: Not Beneficial, Somewhat Beneficial, Beneficial, and Very Beneficial. The

Family Child Guidance Center Benefit Survey has five items and four response options: Not

Beneficial, Somewhat Beneficial, Beneficial, and Very Beneficial. Jacksonville Journey ELP

Early Childhood Consultants, with center director support, distributed and collected the surveys.

Summaries of the responding teachers and families are presented next.

Teachers who participated in Jacksonville Journey ELP activities during the 2010-2011

school year were asked to complete the Teacher Benefit Survey. Table 18 summarizes the

responses of 26 teachers who were in only 15 of the 37 participating Jacksonville Journey ELP

child care centers. The number of responding teachers from the individual child care centers

ranged from one to four teachers per center.


Table 18 Teacher Benefit Survey Items and Survey Response Summary (n=26 Teachers)

Item Not

Beneficial Somewhat Beneficial Beneficial Very

Beneficial Missing

n % n % n % n % n % 1. I am better able to identify

children who do not display developmentally appropriate behavior.

1 4 1 4 11 42 12 46 1 4

2. I am better able to manage children with behavior problems. 4 15 13 50 8 31 1 4

3. I know what to do when a child shows behavior problems. 4 15 13 50 8 31 1 4

4. I now have the tools I need to support children’s mental health (social emotional development).

1 4 5 19 12 46 6 23 2 8

5. I am better able to implement the strategies listed on the Individual Positive Behavior Plan (IPBP) in my classroom.

5 19 10 38 8 31 2 8

All (100%) of the responding teachers indicated that they thought the Jacksonville

Journey mental health services were beneficial in managing children’s inappropriate behavior, in

knowing what to do when a child misbehaves, and in implementing strategies listed on the

children’s IPBPs. Sixty-nine percent of the respondents rated the services Beneficial or Very

Beneficial.

Family Benefit Surveys were distributed to 18 families whose children remained enrolled

in Jacksonville Journey ELP centers at the time data were collected. Fifteen families whose

children were enrolled in only 9 of the 37 participating Jacksonville Journey ELP child care

centers completed surveys. The number of responding families with children enrolled in

individual child care centers ranged from as few as one family per center to as many as three

families per center. Table 19 presents a summary of the families’ responses.


Table 19 Family Benefit Survey Items and Survey Response Summary (n=15 Families)

Item Not


Beneficial N/A Missing

n % n % n % n % n % n % 1. I have learned new skills to

help manage my child’s behavior.

4 27 11 73

2. I am better able to work with my child’s teacher to reinforce positive behaviors.

2 13 6 46 7 47

3. I better understand my role as a parent. 4 27 8 53 3 20

4. My child is happier at the child care center. 4 27 11 73

5. I now know what to do when my child misbehaves. 10 67 5 33

All responding families indicated that the Jacksonville Journey ELP mental health

services were beneficial to them in managing their child’s behavior and in better understanding

their role as parents. Additionally, they indicated that their child was happier at the child care

center and that they were better able to work with their child’s teacher.

Family Child Guidance Center Benefit Surveys were distributed to 15 families

representing 18 children. This number includes families whose children completed treatment and

those who were in treatment at least 3 months. Surveys were received from12 families

representing 14 children. Table 20 presents a summary of the families’ survey responses.

Table 20 Family Child Guidance Center Benefit Survey Items and Survey Response Summary (n=12 )

Item Not


Beneficial N/A

n % n % n % n % n % 1. I have learned new skills to help

manage my child’s behavior. 2 16 10 83

2. I am better able to work with my child’s teacher to reinforce positive behaviors.

3 25 9 75

3. I better understand my role as a parent. 4 33 5 42 3 25

4. My child is happier at the child care center. 3 16 10 83

5. I now know what to do when my child misbehaves. 6 50 6 50


Responses indicate that 100% of the responding families thought the Child Guidance

Center services were beneficial relevant to survey items 1, 2, 3, and 5, respectively. Sixty-seven

percent of respondents indicate they better understand their role as a parent.

For each survey item across all three surveys, the percentage of Somewhat Beneficial,

Beneficial, and Very Beneficial responses was determined. These percentages were averaged

within the surveys to calculate an average response for each survey. Table 21 presents the survey

response percentages and their averages for each of the three surveys.

Table 21 Percentage of Survey Responses Indicating a Benefit by Item Averaged Within Surveys

Item Percentage of Somewhat Beneficial, Beneficial, and Very Beneficial Responses Teacher Survey Family Survey Family Child Guidance Survey

1. 92% 100% 100% 2. 96% 100% 100% 3. 96% 100% 100% 4. 88% 100% 100% 5. 98% 100% 100% Average 94% 100% 100%

Outcome

More than 85% of the parents and teachers (94% of teachers, 100% of families, and

100% of the families receiving Child Guidance Center services) indicated that the mental health

services received were beneficial, via survey instruments.


Expected Outcome 8: 85% of clients enrolled in the program will complete treatment.

Individual Positive Behavior Plans (IPBPs) were developed for 56 children whose ages

ranged from infants to 5-years old. Of the children initially receiving IPBPs, 6 withdrew from the

participating centers. Another 5 attended a center which closed suddenly and transferred to non-

participating centers. Of the remaining 45 enrolled children, 27 children completed their IPBPs

and 18 continue to be served.

Outcome

Twenty-seven (60%) children completed their IPBPs. Due to the individual needs of

children, the remaining 18 children continue to be served.

Documents

EA OURNEY JACKSONVILLE J RLY LEARNING P ROGRAM