INSPIRE Jamaica - UNICEF

INSPIRE Jamaica: Screening and Early Intervention System for Children and Families

INSPIRE Jamaica:

Early Intervention Screening

Project and Referral

Final Report

Grant no: UWI UNICEF 914

9/19/12-1/31/14

Early Intervention Program

University of Oregon

Jane Squires, Principal Investigator

1


Introduction to the Final Report

History Since its establishment in 2003, the Early Childhood Commission (ECC) in

Jamaica has worked strategically with partners and stakeholders to identify priorities and an

action plan for development of a comprehensive, effective early childhood system focused on

achieving optimal development for all Jamaican children. Early Child Development (ECD)

sectors, including Health, Education, and Social Security, identified an Effective Screening and

Early Intervention System for Children at Risk as the critical foundation of this system.

Development of valid and reliable screening tools was a critical first step for building this

early childhood system, with three focal components: 1) Screening and documentation system to

identify households at risk, 2) Screening and documentation system to identify young children at

risk for developmental or behavioral delays and 3) School readiness screening to determine

whether children have the necessary skills to learn and succeed in primary school and to identify

preschool children at risk. These tasks were the focus of INSPIRE Jamaica, a collaborative

project in partnership with the Government of Jamaica including the Early Childhood

Commission and agencies serving young children and families, UNICEF, children and families

in Jamaica, the Early Intervention Program at the University of Oregon, and the University of

West Indies.

This final report summarizes the research conducted as part of the INSPIRE Jamaica

project, including the history of the development and adaptation of measures for each system

component, as well as the data collection, analyses, and results of psychometrics studies of these

measures. Finalized protocols for screening tools and user’s guides were developed and are

included to facilitate accurate administration of the tools in health and educational settings in

Jamaica.

Acknowledgements University of Oregon researchers would like to acknowledge the

contributions of the University of West Indies research team for their work and dedication to

Project INSPIRE. Many thanks to Maureen Samms-Vaughan for envisioning the development

of this system of supports for children and families in Jamaica and her on-going contributions to

this project. Special thanks to Sydonnie Shakespeare for leading the on-site data collection in

Jamaica. Sydonnie and her team of research assistants, Simone Lee, Nara Anderson-Figueroa

2


and Allana Ingram, remained positive despite numerous data collection challenges. This

research never could have happened without their hard work and dedication to project INSPIRE.

Finally, we would like to thank the service providers, families and children in Jamaica that

participated in Project INSPIRE. We hope the tools developed will further the system of

supports for young children and families in Jamaica.

3


Table of Contents

Chapter 1: Technical Report Introduction………………..………………….5

Chapter 2: Family Risk Screening Technical Report…………..……………7

Chapter 3: Child Development Screening Technical Report…..…………..18

ASQ-Jamaica Technical Report……………………..……….18

SWYC Technical Report……………………….…………….36

Chapter 4: School Readiness Screening Technical Report………….……..51

Appendix A: Utility Survey………………………………………………..69

Appendix B: Child and Family Information……………………………….71

Appendix C: References……………………………………………………73

4


Chapter 1: Introduction

In 2012, researchers from the University of Oregon’s Early Intervention Program and

from the University of West Indies collaborated to develop culturally appropriate, valid and

reliable screening tools for use in Jamaica’s early childhood screening system. The screening

system consists of three main components: family support and risk, child development, and

school readiness. See Figure 1 for an outline of the screening components of the INSPIRE

Jamaica Early Childhood Screening System, and proposed administration schedule.

The Family Support Screening tool (FSS), originally developed by the Early Childhood

Commission in Jamaica after significant research on family risk factors, has been reformatted

and revised. The purpose of the FSS is to gather information about family and household risk in

order to provide families with needed support and resources. The FSS asks basic demographic

questions about families, along with questions about parent-child interactions, family health and

lifestyle, and family safety at home and within the community.

The child development component aims to identify those children “at-risk” and/or those

identified as having concerns as part of developmental and behavioral surveillance used during well-

child visits in Jamaican clinics. Two tools, both developed in the United States, were identified and

researched for use with children and families in Jamaica. The Ages and Stages Questionnaires

(ASQ) is a parent-completed child-development screening tool that screens a child’s development in

communication, motor, cognitive and personal-social areas of development. As part of Project

INSPIRE, an adapted version of this tool (ASQ-Jamaica) was developed with input from Jamaican

parents and service providers about the cultural appropriateness of the tool. In addition, the Survey

of Well Being of Young Children (SWYC), which screens for developmental and behavioural risk,

was researched for use with Jamaican families.

Finally, the purpose of the School Readiness tool is to screen children for child development

and readiness for primary school at the end of the four-year preschool. The school readiness

screening tool contains three sections: the ASQ-J child development screening tool for 4-5 year old

children; the Child Behavior Rating Scale; and the Jamaica School Readiness Screening Skills.

The psychometric properties of these screening tools were investigated as part of a

research grant funded through UNICEF. Research questions, utility and psychometric studies for

each of the screening tools are described in detail in the following report.

5

INSPIRE Jamaica

Screening and Early Intervention System for Children and Families

Figure 1. Components and proposed Administration Schedule for the INSPIRE Jamaica

Early Childhood Screening System.

6


Chapter 2: Family Risk Screening

Introduction and Tool Development

The Family Support Screening (FSS) tool was designed to gather information from families in order to

assess family and household risk and to provide families with needed support and resources. The FSS

tool is intended to be completed by professionals or paraprofessionals (e.g. health providers, teachers,

PATH Social Workers) through an interview with a main caregiver, an adult who resides with the child

and is responsible for decisions regarding the child’s care and wellbeing. The FSS tool includes five

sections, with between six and eleven questions in each. Each section of the FSS tool is listed below;

sections are described in detail in the Family Support Screening User’s Guide.

Section 1: Demographic and Socio-Economic Profile of the Household

Section 2: Welfare of Children in the Household

Section 3: Household Health and Lifestyle

Section 4: Family Safety in the Household and in the Community

Section 5: Parental Stress and Parent-Child Interaction

Development of the Family Risk Screening. The Family Support Screening Tool (FSST) was first

developed by the Early Childhood Commission in Jamaica after significant research on family risk

factors. In the Fall of 2012, researchers from the University of Oregon reviewed the most recent

version of the Family Support Screening tool and made some formatting changes, including

eliminating scoring guidelines, to create a pilot version of the tool. Before beginning the pilot phase,

these researchers conducted a focus group in Kingston, Jamaica on the pilot version of the FSS tool

with parents, teachers, Jamaican stakeholders, and University of West Indies (UWI) research

assistants. Feedback provided during this focus group was generally positive. The FSS tool was

revised in the following ways based on focus group feedback.

Section 5: Parental Stress and Parent-Child Interactions had been cut from an earlier version of

the tool and was added back into the pilot version.

A separate Antenatal version of the tool was created for use in antenatal clinics; this antenatal

version includes initial questions relevant to pregnant women.

Individual items were revised for clarity and appropriateness; this included adding in examples

and making subtle changes to wording of items.

In order to guide referrals during the pilot phase, a question was added to each section of the

tool that inquires whether families would like support or assistance to address concerns in that

particular area.

7

Finally, an interviewer guide was created to provide further guidance and clarity on particular

questions for practitioners.

Research Objectives and Questions

One of the key objectives of Project INSPIRE was to study the psychometric properties of the

Family Support Screening. Research was divided into 2 phases; Phase 1 focused on the utility of the

ASQ-J for Jamaican children and families. Phase 2 focused on descriptive statistics, reliability and

validity studies. Following are research questions for each phase:

1) Phase 1. Research Questions

a. What is the utility (including cultural sensitivity) of the Family Support Screening for

families and service providers in Jamaica?

b. What modifications need to be made to the Family Support Screening?


a. What are descriptive statistics (e.g., means, standard deviations) of the Family Support

Screening for Jamaican households?

b. What is the validity of the Family Support Screening (e.g., sensitivity, specificity) when

compared with other psycho-social measures, such as the Center for Epidemiological

Studies—Depression Scale (CES-D), (Radloff, 1977); Parenting Stress Index (PSI),

(Abidin, 1983); and Ages and Stages Questionnaires: Social Emotional (ASQ: SE).

Phase 1. Utility Data Collection on the Pilot Family Support Screening

During the Pilot Phase, UWI research assistants completed the FSS tool through interviews with

parents attending well child check-ups at health clinics and parents of children in preschool classrooms.

Parents provided utility feedback on the clarity, appropriateness and meaningfulness of FSST items.

One hundred utility surveys were collected on the FSS tool. Table 2.1 summarizes FSS tool utility

data. A copy of the utility survey can be found in Appendix A at the end of this technical report.

8

Table 2.1

Summary of Responses to Utility Statements*

Total Strongly

Agree

Agree No

Opinion

Disagree Strongly

Disagree

Missing

N N (%)

N (%)

N (%)

N (%)

N (%)

N (%)

In general,

questions clear

and easy to

understand

100 47 (47%)

49 (49%) 2 (2%) 0 (0%) 0 (0%) 2 (2%)

Questions

appropriate for

child and

family’s culture

100 33 (33%) 63 (63%) 1 (1%) 2 (2%) 0 (0%) 0 (0%)

Completing tool

provided

meaningful

Information

100 17 (17%) 59 (59%) 13 (13%) 9 (9%) 1 (1%) 1 (1%)

Would like to

use this tool

again

100 11 (11%) 69 (69%) 8 (8%) 10 (10%) 1 (1%) 1 (1%)

*Response to statements (e.g., In general, were the questions clear and easy to understand?) were on a

5 point Likert Scale, (Strongly Agree, Agree, No Opinion, Disagree and Strongly Disagree).

Additional Written Comments on Utility Surveys:

Add open questions.

First ask questions-if there is crime and violence in the community, if person says no, give

examples

This tool would have taken less time to complete if I had assistance.

Question needs to be disguised so that it doesn’t make people uncomfortable.

The tool could look at questions such as whether the child is engaged versus withdrawn. If

parent can cope.

Extend the tool to much more schools.

The tool should improve on more questions about parent.

University of Oregon researchers conducted a second round of focus groups with parents and

UWI research assistants after the FSS tool had been piloted. Participant feedback in these focus groups

9

was overwhelmingly positive, though a few suggested specific item changes. In addition, the following

advantages and disadvantages were shared by parents in the focus group.

Advantages of the tool:

o Good to get conversation started and getting information out there.

Disadvantages:

o People might hesitate to answer questions but desire for help could motivate people to

answer.

o Questions about wanting support or assistance could be more generally different for the

whole household as opposed to for the main caregiver.

Summary of Utility Findings Utility data on the FSS tool were quite positive. Almost all--96%

of respondents—strongly agreed/agreed that the questions on the FSST are clear and easy to

understand. Ninety-six percent of respondents strongly agreed/agreed that questions were appropriate

for their child and family’s culture. Seventy-six percent of respondents strongly agreed/agreed that

completing the tool provided meaningful information about the child; Eighty percent of respondents

strongly agreed/agreed that they would like to use this screening tool again. Current findings and

anecdotal reports from the UWI research team suggest that the FSST will be an appropriate and useful

screening tool for determining Jamaican family risk and needs.

Phase 2. Psychometric Data Collection on the Family Support Screening

The second phase of data collection involved collecting psychometric data on the field version

of the FSS questionnaire. Research staff from the University of West Indies assisted in identifying

health clinics and preschool classroom settings that provided as well-stratified a sample of Jamaican

families and children as possible, given financial and logistical constraints of the project. All

parents/caregivers in targeted settings were provided with information about the child and family risk

screening projects and asked if they would be willing to participate. As part of participation in the

research, main caregivers received an FSS questionnaire from the teacher or researcher along with a

form asking for the child’s demographic information and a research consent form. The FSS

questionnaire was completed through an interview with a research assistant. The research assistant

assisted the parent with referrals to community support services as needed. Procedures ensuring

protection of human participants were approved by the University of West Indies and the University of

Oregon institutional review boards and were followed in all research phases.

The following data are based on 251 FSS questionnaires completed by parents of children

10

between 1 and 66 months of age.

Population Sample for the Family Support Screening

The data analyses that are contained in this report are based on 206 completed questionnaires.

Each parent or caregiver who completed a questionnaire was asked to complete a demographic form.

Demographic data included information on the child’s age, gender, parish the child resides, setting

screening took place, who completed screening, and number of family household resources (as a proxy

of risks). It should be noted that there were 103 missing demographics, so percentages reported are

based on the 148 data for which demographic information was collected. In some cases, individual

item data was also missing. The demographic data for the population sample are displayed in Tables

2.2 through 2.6. A copy of the demographic form can be found in Appendix A at the end of this

technical report.

Gender of child and parish where child resides. As shown in Table 2.2, the gender distribution

for the sample was 59.9% male and 40.1 % female. Table 2.3 contains data on the parish where the

child lived at the time of the data collection. The majority of children lived in Kingston (41.9%), St.

Andrew (35.8%), or St. Catherine (12.4%) parishes. A small percentage of children (1.4%) resided in

other parishes in Jamaica.

Settings Where Screenings Took Place. Table 2.4 contains data on the setting where the FSS

screening took place. A large percentage (61.5%) of FSS screenings occurred in health settings, 32.4%

occurred in educational settings, and 6.1% took place in “other” settings. Approximately 76% of these

settings were public and 22% were private settings. Ninety-six percent (98.7%) of screenings were

conducted in urban settings (4.1% of the total sample was considered “inner” urban), while only 1.4 %

of screening occurring in rural settings. See the Limitations of Study section for a discussion of the

imbalance or urban/rural settings and recommendations for future research on this tool.

Table 2.2

Gender of Children

Frequency Valid Percent

Male 85 59.9

Female 57 40.1

Total 142 100.0

Missing 109

11

Total N 251

Table 2.4

Setting Where Screening Took Place

Person Completing Questionnaires. Table 2.5 contains data on the person completing the

questionnaires. The majority of individuals completing the questionnaires were mothers (92%). It

should be noted that the majority of these (98.6%) were completed with the support of a research

assistant. While FSS can be completed independently, data on the FSS tool was collected primarily by

research assistants through an interview format. Further research would need to be conducted to

determine how much support is required in actual health and educational settings in Jamaica to

complete the FSS tool, and whether main caregivers are able to complete it independently.

Table 2.3

Parish Where Child Resides

Parish Frequency Valid Percent

Kingston 62 41.9

St Andrew 53 35.8

St Catherine 31 12.4

Other 2 1.4

Total 148 100.0

Missing 103

Total N 251

Setting Frequency Valid Percent

Antenatal 15 10.1

Clinic 76 51.4

Early Childhood

Educational 26 17.6

Primary School 3 2.0

Nursery 19 12.8

Other 9 6.1

Total 148 100.0

Missing 103

Total N 251

12

Table 2.5

Person Completing Questionnaire

Household resources (child risk). Data on the number of household resources was collected as

a proxy of child risk status. Parents were asked to self-report household resources from a list of 26

resources (e.g., telephone, car, water heater). A total number of resources was entered for each family;

data was not entered in a way that identified specific resources held by each family. Household

resource data are displayed in Table 2.6. A little more than two percent (2.1%) of the sample reported

having between 1 and 4 household resources, 12.8% reported having between 5 and 8 resources,

24.8% reported having between 9 and 12 resources; 33.3% reported having between 13 and16

resources; 22% reported having 17-20 resources, and 5% reported having more than 21 household

resources.

Cutoff Scores and Percent Identified of Population for the Family Support Screening

A main goal of Phase 2 was to establish cutoff scores for the Family Support Screening (FSS)

tool in order to provide guidance to practitioners in identifying families with a high number of

concerns and in initiating referrals and other support. A discussion of validity studies follows.

However, validity samples were not adequate to inform the establishment of cutoff scores. Instead,

cutoff scores were derived using the normative sample (N=251; Ante-natal N=20); cutoff scores were

determined based on percentage of families identified.

Frequency Valid

Percent

Mother 136 92

Father 6 4.1

Grandmother 2 1.4

Aunt 2 1.4

Other-family 2 1.4

Total 148 100.0

Missing 103

Total N 251

13

Table 2.6

Total Number of Household Resources

# of Resources Frequency Valid Percent

1-4 3 2.1

5-8 18 12.8

9-12 35 24.8

13-16 47 33.3

17-20 31 22

21-24 7 5

Total 141 100

Missing 110

Total N 251

Initially researches established an overall cutoff score of 8 total concerns on the FSS tool. This cutoff

identified approximately 16 % of the normative sample. Jamaica may decide to use this overall cutoff

score. However, researches recognized that the intent of the FSS tool was to identify and follow up on

family risk factors. Because the overall cutoff score did not identify individual FSS sections with

higher concern scores, researchers created separate cutoff scores for individual sections that identified

approximately the same percentage of the normative population per section. As illustrated in Table 2.7,

each individual section of the FSS tool was assigned a unique cutoff score that would identify between

8 and 10 percent of the normative population. These cutoff scores are included in the summary sheet

on the final page of the FSS tool, and instructions are provided in the User’s Guide for using these

section cutoff scores to guide follow-up with families. Overall, the percentage of families in the

normative sample who were below the cutoff score in one or more of the individual FSS sections was

27%. Thus, if the normative sample is representative of the Jamaican population, providers can expect

to identify approximately 27% of families using these cutoff scores.

As discussed above, the section cutoff scores were initially established in order to identify a

particular percentage of the population. Researchers also analyzed the data to determine mean scores

and standard deviations within each section. Using means and standard deviations, cutoff scores are

set at two standard deviations above the mean score for each section, as illustrated in Table 2.8. Cutoff

scores are similar when computed based on percent identified and means/standard deviations.

14

Table 2.7

FSS Tool Cutoff Scores

Section N Cutoff Score Percent of Normative

Sample Identified

A (Ante-natal) 20 3 15

2 251 4 8.4

3 251 2 9.2

4 251 4 9.2

5 251 3 8.4

Table 2.8

Mean Total Concern Scores, Standard Deviations, Cutoff Scores, and Percent Identified by Section of

the Family Support Screening

Section 2 Section 3 Section 4 Section 5

Valid 251 251 251 251

Missing 0 0 0 0

Mean Total Concerns 1.38 .47 1.41 1.32

Standard Deviation 1.33 .77 1.32 .90

Cutoff Score 4 2 4 3

Percent Identified 8.4 9.2 9.2 8.4

As indicated above, Phase 2 research also included a plan to collect an FSS validity sample

using three corroborating tools: the CES-D (Center for Epidemiological Studies-Depression Scales),

the PSI (Parenting Stress Index) and the ASQ: SE (Ages and Stages Questionnaires Social-Emotional).

A validity sample size of 50 was targeted for all three tools. The total data collected included the 33

PSI, 28 CES-D, and no ASQ:SE.

Researchers analyzed correlations between total scores on the PSI and total concerns on the

FSS tool and found an overall correlation of .235. This correlation was weak and not significant.

Researchers also analyzed correlations between total scores on the CES-D and total concerns on the

FSS tool. Because eight of the CES-D scales had items for which responses were missing, researchers

calculated mean scores to replace missing item responses. The overall correlation between the FSS and

the CES-D was .459. Though this correlation is significant at the .05 level (2-tailed), it is also

15

considered weak. Correlations are illustrated in Tables 2.9 and 2.10 below.

Table 2.9

Correlations between FSS Total Concerns and PSI Total Stress Score

FSS Total Concerns PSI Total Score

FSS Total Concerns Pearson Correlation 1 .229

Sig. (2-tailed) .200

N 36 33

PSI Total Score Pearson Correlation .229 1


N 33 47

Researchers also analyzed the agreement between FSS cutoff scores and PSI and CES-D total scores.

For this purpose, two respondents who exhibited defensive responding on the PSI were eliminated

from the sample, resulting in a sample size of 31 for this analysis. Researchers looked at agreement

between those identified by the PSI and those families who had one or more area above the cutoff on

the FSS. The agreement was .74. However, no one in the validity sample was identified by the PSI as

having stress, this agreement does not provide meaningful information.

The fact that no one in the PSI sample was identified as having stress could be due to the fact that this

sample was not randomized, to the fact that participation by families was optional and/or to the fact

that the surveys were administered by unfamiliar researchers rather than by health providers known to

the families. Future research efforts might include a randomized sample representing a broader range

of families.

Limitations and Future Directions. There are several limitations to this study that should be considered

when implementing the Family Support Screening and considering future research directions. It is

important to note that this was an ambitious study, conducted within an exceptionally short time frame

and with limited resources. As discussed earlier, demographic information was not collected from 103

families, so percentages reported are based on the 148 families from whom demographic information

was collected.

16

Table 2.10

Correlations between FSS Total Concerns and CES-D Total Score

FSS Total Concerns CES-D Total Score

FSS Total Concerns Pearson Correlation 1 .459*


N 28 28

CES-D Total Score Pearson Correlation .459* 1


N 28 49

* Correlation is significant at the .05 level (2-tailed)

Data was also missing for some individual questions on gathered demographic forms. Because

demographic data is missing, it is not clear whether the normative sample is representative of the

general Jamaican population. Collected demographics indicate that the majority of data was collected

in three parishes: Kingston, Saint Catherine and Saint Andrew. As also discussed earlier, 98.7% of the

FSS screenings for which demographic information was collected took place in urban settings, while

only 1.4 % took place in rural settings. Future studies on the FSS tool should include a representative

sample of the Jamaican population.

As also discussed above, the small size of the validity sample combined with the weak and

insignificant correlations between chosen concurrent validity tools required researchers to use a

different method to establish cutoff scores. Cutoff scores were derived based on percentage of families

identified and presumed a normative sample that was stratified in terms of number of risk factors, and

included households with a low, medium and high number of risk factors. Because the FSS tool covers

a broad range of risk factors and areas of potential family distress (health, violence, stress, community,

parent child interactions etc.) it was challenging to identify concurrent validity tools that measured

similar areas. It may be helpful for future studies to conduct a factor analysis to identify categories of

risk factors covered by the FSS tool and to examine concurrent validity using additional measures that

have been used or validated in Jamaica and that correspond to the identified categories.

17


CHAPTER 3: Child Development Screening


Project INSPIRE research staff studied the utility and psychometric properties of two child

development screening tools--one with a developmental focus, the Ages &Stages

Questionnaires: Jamaican version (ASQ-J), and one that is recommended to be used when social-

emotional/behavioral concerns are identified, the Survey of Wellbeing for Young Children

(SWYC). This chapter provides information on utility studies and psychometric studies for both

tools. The first section covers technical information on the ASQ-J, followed by the SWYC

technical report.

Part I

ASQ-J Technical Report

This section offers a range of technical information about the Ages & Stages

Questionnaires: Jamaican version (ASQ-J) and the development of the Ages & Stages

Questionnaires (ASQ-J) system, including focus group and utility studies that led to selected

questionnaire revisions. Normative studies in Jamaica resulted in recruiting and collecting 876

questionnaires. The collected data have been used to examine selected psychometric parameters

of the ASQ-J. In addition to describing the demographic characteristics of the samples, analyses

address test–retest reliability and measures of internal consistency. A comparison of

questionnaire performance by groups of risk and non-risk children is presented. Validity analyses

include descriptions of how the cutoff points were determined and of measures of concurrent

validity.

Introduction and Background. The Ages and Stages Questionnaires (ASQ) are a

series of parent-completed questionnaires designed to accurately identify infants and young

children who are in need of further evaluation and may benefit from early intervention or early

childhood special education services. Because the questionnaires are completed by parents or

caregivers and have flexible administration procedures, they have been easily incorporated into a

variety of health, educational and social service settings in the United States, (Squires, Bricker,

Twombly, & Potter, 2009) as well as a number of other countries including China (Bian, Yao,

Squires, Wei, Chen, & Fang, 2010), Korea, (Heo, Squires, & Yovanoff, 2008), Turkey (Kapci,

Kucuker, & Uslu, 2010) and the Caribbean (Roving Caregiver Program; http://www.ccsi-

18


info.com/).

University of Oregon Project INSPIRE staff recommended using the Ages and Stages

Questionnaires (ASQ) as the in-depth developmental screening tool for the most “at-risk”

children and/or those identified as having concerns based on the current developmental

surveillance method used during well child visits in Jamaican clinics. The flexibility of the ASQ

system should work well given Jamaica’s objective of developing a screening tool that can be

used by parents, primary health care, and early childhood providers in Jamaica. Secondly, while

the ASQ-3 was developed in the United States (US), and the data used to investigate the

psychometric properties and establish cutoff points have been gathered primarily from families

and children who reside in the US, a number of researchers and practitioners from outside of the

US have translated or adapted the tool for their use. The ASQ has been translated into over ten

languages, including Korean, French-Canadian, French, Turkish, Norwegian, Vietnamese,

Chinese, Portuguese, and most recently for Aboriginal families in Australia. The ASQ system

has been extensively studied by researchers in these countries and overwhelmingly has been

found to be valid and easy to translate and adapt for use with different cultures (Bian, Yao,

Squires, Wei, Chen, & Fang, 2010; Kapci, Kucuker, & Uslu, 2010; Heo, Squires, & Yovanoff,

2008; Dionne, Squires, & Leclerc, 2004; Janson & Squires, 2004). Finally, the tool has been

extensively studied and is recommended for use by the American Academy of Pediatrics, (AAP,

2006).

Development of the ASQ-J Translation and adaptation of a measure for use in a different

culture requires several safeguards. As a number of authorities have cautioned, the adoption of a

measure into a different culture requires careful consideration to ensure its appropriate use (e.g., Ball

& Janyst, 2008). Screening or assessment tools that are not culturally appropriate may yield

misleading or negative results and unintended consequences (Musquash & Bova, 2007). Between

August 2012 and November 2012, researchers from the University of Oregon’s Early Intervention

Program collaborated with researchers from the University of West Indies to discuss the

development of an adapted version of the ASQ for use with children and families in Jamaica. In

December 2012, University of Oregon researchers conducted a focus group in Kingston, Jamaica on

the original ASQ with parents, teachers, Jamaican stakeholders, and University of West Indies

research assistants. Focus group participants reported that the overall structure, intent and items on

the ASQ were very appropriate for children and families in Jamaica but suggested that minor

19


changes and adaptations to the tool would allow it to better reflect Jamaican language and culture.

Based on feedback, changes were made to the ASQ questionnaires, and a pilot version of the ASQ-J

was developed.

The Ages and Stages Questionnaires-Jamaica (ASQ-J) is an adaptation of the U.S.

version of the ASQ. The original ASQ system consists of a series of 21 questionnaires that

screen children at specific age intervals, between the ages of 1 to 66 months. Twelve ASQ-J

intervals were developed (from the original 21), and 8 were prioritized for research (i.e., the 6,

12, 18, 24, 30, 36, 54 and 60 month ASQ-J intervals). Each ASQ-J questionnaire contains 30

developmental items that are written in simple, straightforward language. The items are

organized into five areas: Communication, Gross Motor, Fine Motor, Problem Solving, and

Personal-Social. An Overall section addresses general parental concerns. The prioritized

intervals fit into key components of Jamaica’s current system of health and educational systems.

This pilot version includes examples in dialect (Patois) for a number of items in the

communication domain, both in how the question was asked, and examples of correct responses.

In addition, changes were made to item examples that better reflected materials, games, animals,

foods, etc. common in Jamaica (e.g., “mouse” was changed to “ant”; “applesauce” was changed

to “sand”). In addition, the direction “check the circle” when selecting an item response was

changed to “tick the circle”. Items that included units of measurement were changed from the

US system (e.g., inches) to the Jamaican system (e.g., centimeters). The ASQ-J is intended to

be used in the following settings in Jamaica:

Health Clinic Well Child Check-Ups. The ASQ-J will be used a second level screening

tool during well child checkups. During well child checkups, all children are currently

screened at 6, 9, 12, 18, 24 and 36 months using the “11 questions”, included in Jamaica’s

Child Health and Development Passport. The proposed ASQ-J should be administered as

a second level screening when any concern is indicated on the “11 Questions”.

Preschool Programs as part of School Readiness Assessment. The ASQ-J is the parent

component of the School Readiness Assessment that should occur during the end of the

child’s 4/5 school year. In the school setting, the ASQ-J should be completed by the

parent or by the parent and teacher together.

20



One of the key objectives of Project INSPIRE was to study the psychometric properties of

the ASQ-J. Research was divided into 2 phases; Phase 1 focused on the utility of the ASQ-J for

Jamaican children and families. Phase 2 focused on descriptive statistics, reliability and validity

studies. Following are research questions for each phase:


a. What is the utility (including cultural sensitivity) of the ASQ-J for children,


b. What modifications need to be made to the ASQ-J?


a. What are descriptive statistics (e.g., means, standard deviations) of the ASQ-J for

Jamaican children and families?

b. What is the reliability (test-retest) of the ASQ-J?

i. What is the internal consistency of the ASQ-J?

c. What is the validity of the ASQ-J (e.g., sensitivity, specificity) when compared to

known groups of children with/without disabilities?

Phase 1. Utility Data Collection on the Pilot ASQ-J

In early February 2013, the UWI research team began to collect pilot data on the utility of

the ASQ-J. Collection of utility data continued through mid-July of 2013. Research Assistants

collected ASQ-J’s from parents in Health Clinics and teachers in preschool classrooms. Teachers

and research assistants also completed utility forms indicating the clarity, appropriateness and

meaningfulness of ASQ-J items. Teachers were asked whether they planned to use the tool again

and how they might change it make it better. Results of utility surveys are summarized below. In

May 2013 during a visit to UWI, University of Oregon researchers conducted a second round of

focus groups with parents and teachers and research assistants on the piloted ASQ-J. Participants

in the May focus groups suggested minor formatting changes to the ASQ-J, along with a few

specific suggestions for item changes. These suggestions are listed below:

18 month ASQ GM#5: Change qualifier to “child may or may not hold onto the wall”.

Provide more illustrations across items to assist with low literacy.

Provide additional space for children to copy letters, draw person, etc.

21


60 month ASQ Com#2: Does your child use 4 and 5 word sentences? For example does

your child say, “I want the car”? Provide a Patois example.

60 month ASQ Com #5: What do you do when you get tired? Change to: What do you

do when you get tired at night? (Children have been answering “drink water.”)

Parents and teachers also described the following benefits of the ASQ-J based on their

experiences using the tool. Parents said that the ASQ-J:

Provided useful information

Made you think about things child/baby was doing

Was reassuring to know that child was okay, could do things asked

Brought to light things might not be seen as developmental progress

Highlighted things that could be tried with child or could encourage child to do. Gives

ideas of what to expect at which age.

Was a good way to keep track without doing a big developmental assessment

Domain aspect is very helpful gives idea of where to encourage child e.g., may need

encouragement in gross motor but not in communication.

Teachers made the following comments:

Get to know things you did not know about the child (e.g., medical history, hearing)

Helps parents to know the type of skills their child knows. Learning experience for

parents to know how to monitor their child for certain activities.

Utility Survey Data Collection One hundred ten (110) utility surveys were collected on

the Jamaican version of the Ages and Stages Questionnaires (ASQ-J) across intervals from 2-66

months. The target goal was to collect 80 utility surveys overall. Seventy-eight (78) utility

surveys were collected from parents, 30 from preschool teachers, one from a child’s auntie, and

one from a child’s grandmother. For some of the intervals, such as 24 months, it was more

difficult to recruit parents to complete surveys since these are not the typical times for well child

visits. However, in order to develop a comprehensive set of questionnaires, utility was collected

on all intervals. Eight of the intervals (highlighted below) are “priority intervals”. These intervals

correspond to well child checkups conducted in the Jamaican health care system, and the

proposed aged children will be screened to determine “School Readiness”. The following two

22


tables summarize the number of utility surveys collected by interval (Table 3.1) and responses to

the utility questionnaire (Table 3.2). See Appendix A for INSPIRE Utility Survey.

Table 3.1

Number of utility surveys collected by ASQ-J interval

ASQ-J

intervals

2

mo

6

mo*

10

mo*

12

mo

18

mo*

24

mo*

36

mo*

48

mo

54

mo*

60

mo*

Current

Utility

N

12 10 15 7 16 11 12 3 11 12

Table 3.2

Summary of Responses to Utility Statements*

Total Strongly

Agree Agree No Opinion

Disagree Missing

N N (%)

N (%)

N (%)

N (%)

N

In general, questions

clear and easy to

understand

110 54 (41.9%)

55 (50%) 0 (0%) 1 (.9%) 0 (0%)

Questions

appropriate for child

and family’s culture

110 31 (28.2%) 77 (70%) 1 (.9%) 1 (.9%) 0 (0%)

Completing tool

provided meaningful

Information

110 28 (28.5%) 62 (56.4%) 5 (4.5%) 15

(13.6%)

0 (0%)

Would like to use

this tool again

110 27 (24.8%) 76 (69.7%) 5 (4.6%) 1 (.9%) 1 (.9%)

Question

appropriate for

child’s age

110 30 (27.3%) 59 (53.6%) 18 (16.4%) 3 (2.7%) 0 (0%)

*Response to statements (e.g., In general, were the questions clear and easy to understand?) were

on a 5 point Likert Scale, (Strongly Agree, Agree, No Opinion, Disagree and Strongly Disagree).

In no cases, did participants use the “Strongly Disagree” option, so it is not included in the

summary table.

23


Additional Written Comments:

“Some questions she (the baby) wouldn’t reach yet”

“There are something that I wouldn’t expect baby to do”

“For now, instrument is well designed”

“It was really a privilege, helped me in my studies. Focuses on observation and not

standardized test is better”

How would you change this tool to make it better?

“Give the teachers the kit to work with and ask less questions”

“Separate questions from answers to make it not jumbled”

“Space should be provided for the child’s drawings”

Summary of Utility Findings Utility data on the ASQ-J were quite positive. Almost all—

99.1% of respondents--strongly agreed/agreed that the questions on the ASQ-J were clear and

easy to understand. Ninety-eight percent (98%) of respondents strongly agreed/agreed that

questions were appropriate for their child and family’s culture. Eighty-two (82%) percent of

respondents strongly agreed/agreed that completing the tool provided meaningful information

about the child; 94.5 % of respondents strongly agreed/agreed that they would like to use this

screening tool again. Nearly 81% of respondents strongly agreed/agreed that the questions were

appropriate for their child’s age. Preschool teachers felt that the ASQ-J added valuable

information to the School Readiness Tool and they strongly recommended keeping it for this

purpose.

Based on these findings and anecdotal reports from the research team from the UWI, it

was determined that the ASQ-J appears to be an appropriate and useful child developmental

screening tool for Jamaican populations. Some minor revisions were made to the pilot version of

the ASQ-J formatting and wording. For example, a larger space was provided within the

protocol on the 54 and 60 month ASQ-J intervals for older children to demonstrate their ability to

draw shapes and letters. On the 48 and 60 month intervals, focus group participants reported

problems with a question that asks children to answer the question, “What do you do when you’re

tired?”. Acceptable answers include such phrases as “go to bed”, but a number of children in

Jamaica were answering, “drink water”. The question was modified to, “What do you do at night

24


when you are tired.” Additional responses in Patois were added to a number of intervals. This

“field version” of the ASQ-J was used in Phase 2 data collection.

Phase 2. Psychometric Data Collection on the ASQ-J

The second phase of data collection involved collecting psychometric data on the field

version of the ASQ-J. Questionnaires were completed by parents and caregivers accessing

Jamaican health and educational settings. Teachers of preschool age children were also involved

in completing ASQ-J questionnaires on children older than age 36 months in school settings.

Research staff from the University of West Indies assisted in identifying settings that provided as

well stratified a sample of Jamaican families and children as possible, given financial and

logistical constraints of the project. All parents/caregivers in targeted settings were provided

with information about the child and family risk screening projects and asked if they would be

willing to participate. As part of participation in the research, parents received an ASQ-J

questionnaire from the provider along with a form asking for the child’s demographic

information and a research consent form. The ASQ-J questionnaire was completed either

independently by the parent or with assistance from a research assistant or an educational or

healthcare provider. The completed questionnaires were scored, and the results were shared with

the parent or caregiver. The provider assisted the parent with referrals to community support

services as needed. Procedures ensuring protection of human participants were approved by the

University of West Indies and the University of Oregon institutional review boards and were

followed in all research phases.

The following data are based on 876 questionnaires completed by parents of children

between 1 and 66 months of age. The numbers of questionnaires collected by interval are shown

in Table 3.3.

Population Sample

The data analyses that are contained in this report are based on 876 completed

questionnaires. Each parent or caregiver who completed a questionnaire was asked to complete a

demographic form. See Appendix B for INSPIRE Child/family Demographic Form.

Demographic data included information on the child’s age, gender, parish the child resides,

setting screening took place, who completed screening, number of family household resources

25


(as a proxy of risk), and whether the child was known to have any health risks or known

disabilities. It should be noted that there were 231 missing demographics, so percentages

reported are based on the 645 data for which demographic information was collected. The

demographic data for the population sample are displayed in Tables 3.4-3.7.

Table 3.3

Frequency and Percentage of Data Collected by ASQ-J Interval*

Interval Frequency Percent

2 25 2.9

6* 102 11.6

8 5 .6

10* 106 12.1

12* 92 10.5

18* 113 12.9

24* 94 10.7

30 4 .5

36* 104 11.9

48 18 2.1

54* 114 13.0

60* 99 11.3

Total 876 100.0

* These intervals were prioritized during data collection

Gender of child and parish child resides. As shown in Table 3.4, the gender distribution for the

sample was 53.5% male and 46.5 % female. Table 3.5 contains data on the parish where the

child lived at the time of the data collection. The majority of children lived in Kingston (59.8%),

St. Andrew (23.3%), or St. Catherine (7.9%) Parishes. A small percentage of children (1.5%)

resided in other parishes in Jamaica.

Settings Screenings Took Place Table 3.6 contains data on the setting where the screening took

place. A large percentage (87%) of screenings occurred in health settings, 9.1% occurred in

educational settings, and 3.9% took place in “other” settings. Approximately 92% of these

settings were public and 7.6% were private settings. Over sixty-two percent (62.4 %) of

26


screenings were conducted in urban settings, 32.6% were conducted in “inner” urban settings,

while only 5% of screening occurring in rural settings. See the Limitations of Study section for a

discussion of the imbalance or urban/rural settings and recommendations for future research on

this tool.

Table 3.4

Gender of Children


Male 345 53.5

Female 300 46.5

Total 645 100.0

Missing 231

Total N 876

Table 3.5


Person Completing Questionnaires Table 3.7 contains data on the person completing the

questionnaires. The majority of individuals completing the questionnaires were mothers (92.1%).

This finding is consistent with feedback from hundreds of screening professionals who report

that mothers are most apt to complete the questionnaires on their children. It should be noted

that the majority of these (87.1%) were completed with the support of a research assistant.

While ASQ-J questionnaires can be completed independently by parents, data collection methods

may have increased the amount of support research assistants provided. In order to determine

Parish Frequency Valid Percent

Kingston 396 59.8

St Andrew 204 30.8

St Catherine 52 7.9

Other 10 1.5

Total 662 100.0

Missing 214

Total 876

27


how much support is required in actual health and educational settings in Jamaica, further

research would need to be conducted.

Table 3.6



Antenatal 18 2.7

Clinic 557 84.1

PATH Home Visit 1 .2

Early Childhood Ed. 34 5.1

Primary School 3 .5

Nursery 23 3.5

Other 26 3.9

Total 662 100.0

Missing 214

Total 876

Table 3.7



Mother 610 92.1

Father 20 3.0

Grandmother 15 2.3

Aunt 6 .9

Stepmother 1 .2

Sister 1 .2

Other-family 7 1.1

Non-family 2 .4

Total 662 100.0

Missing 214

Total 876

28


Household resources (child risk). Data on the number of household resources were

collected as a proxy of child risk status. Parents were asked to self-report household resources

from a list of 26 resources (e.g., telephone, car, water heater). A total number of resources was

entered for each family. These data are displayed in Table 3.8. Nearly two percent (1.8%) of

the sample reported having between 1-4 household resources, 19.5% reported having 5-8

resources, 31.6% reported having 9-12 resources; 29.9% reported having 13-16 resources; and

17.1% reported having more than 17 household resources.

Table 3.8

Total Number Household Resources

Cutoff Scores and Percent Identified of Population

Because of the limited number of questionnaires available to establish “Jamaican”

cutoffs, an approach examining the percentage of children identified as “at-risk” was chosen to

determine cutoffs for the ASQ-J. Data were analyzed using the cutoffs from the third edition of

the U.S. version of the ASQ (the ASQ-3), which were established based on a stratified sample of

more than 18,000 children from the U.S. Concurrent validity studies conducted on the ASQ-3

indicated that the best domain cutoffs for optimizing sensitivity and specificity were at 2

standard deviations below the mean.

Using the INSPIRE data set, children were classified as “at-risk” if scores were below

established U.S. cutoffs, or “no-risk” if scores were above established U.S. cutoffs. There are 5


1-4 12 1.8

5-8 127 19.5

9-12 206 31.6

13-16 195 29.9

17-20 102 15.6

21-24 10 1.5

Total 652 100.0

Missing 224

Total 876

29


developmental areas or “domains” on the ASQ (e.g., communication, fine motor) and scores can

be below cutoff or “at-risk” in 0 to 5 areas. Targets of 12%–16% of children identified in one

developmental area (i.e., one area below the cutoff score) and 2%–7% identified in two or more

areas were adopted as the desired percentages to be identified for further assessment at each age

interval. These figures were based on U.S. Census Bureau and Centers for Disease Control and

Prevention prevalence data related to developmental disabilities in young children as well as

additional resources on worldwide disability prevalence (Cornell University, 2003–2009; U.S.

Census Bureau, 2004).

Table 3.9 contains data on the total number of Jamaican children identified as “at-risk” in

1 or more domains (17.7%), which is slightly higher than the 15.5 % of U.S. children identified

as “at-risk” in studies of the ASQ-3 (the range across intervals was 12-19.2%). Table 3.10

contains data on the total number of Jamaican children identified as “at-risk” in 2 or more

domains across intervals (4.7%) which is lower than the 5.9% of US children identified as “at-

risk” in ASQ-3 research (although still within the range across intervals which was 3.1-7.9 %).

Table 3.11 illustrates the percentage of children identified as “at risk” by number of areas low.

Eighty-two percent (82%) of all screenings conducted had no areas below cutoff. Two children

(.2%) received scores below cutoff in 5 areas on the ASQ.

It is not surprising that U.S. cutoffs may operate quite similarly for Jamaican populations.

Jamaican health and educational expectations are similar to US expectation and standards.

Similar materials (e.g., writing tools, scissors, puzzles) are present in preschool settings in

Jamaica (at least in the urban schools observed), and no materials were considered “culturally

inappropriate” during focus group sessions with Jamaican providers and parents. There will be

settings in Jamaica where materials are limited and children have not had opportunity to try

items on the ASQ-J, also common in the U.S. as well. Instructions for administering the tool

always recommend making materials available for children so they can practice skills prior to

administering the tool. Many times the process of screening provides opportunities for parents

and providers to think about how to make these new experiences available to young children.

Table 3.12 contains data on the percent of children identified as “at-risk” by the number of

reported household resources. Percentages range from a low of 15% to a high of 25% in

households with less than 4 reported resources. It is difficult to draw any conclusions based on

these data since there are only 12 children in the category reporting less than 4 household

30


resources. Demographic data, including the number of household resources, were missing in 224

cases. The overall percent of children identified as “at-risk” in 1 area on the ASQ-J, as reported

earlier, was 17.7%.

Table 3.9

Percent of Children Identified as “At-Risk” in 1 or More Domains Using U.S. Cutoffs

Table 3.10

Percent of Children Identified as “At-Risk” in 2 or More Domains using US Cutoffs

Reliability Studies

Reliability studies completed on the ASQ-3 include test–retest reliability and internal

consistency of ASQ-3 items. Internal consistency was examined using correlational analyses

and the Cronbach coefficient alpha (Cronbach, 1951). Each of these analyses is presented next.

Test–Retest Reliability. Test–retest reliability is designed to help determine the stability of test

outcomes over time. Test–retest reliability of the ASQ-J was examined by comparing two

questionnaires completed by the same caregiver at a 2-3 week time interval.

Child status

Frequency

Jamaica

Total %

Jamaica

Frequency

US data

Total %

US data

No-risk

(above cutoff)

721 82.3 15, 687 84.5%

At-risk (below

cutoff)

155 17.7 2885 15.5%

Total 876 100.0 18, 572

Child status

Frequency

Jamaica

Percent

Jamaica

Frequency

US data

Percent

US data

No-risk

(above cutoff)

835 95.3 17,478 94.1%

At-risk (below

cutoff)

41 4.7 1094 5.9%

Total 876 100.0 18, 572 100%

31


Table 3.11

Percent of Children Identified by Number of Areas below Cutoff “At risk”

Table 3.12

Percent Identified as “At-Risk” in 1 Area on ASQ-J by Number of Household Resources

That is, parents were asked to complete the same questionnaire interval for their child twice

within a 2-3 week time period between completions. Questionnaires completed by 45 parents

were included in this analysis. Parents were blind to the results of the first questionnaire when

they completed the second one. The two questionnaires completed by parents were then

compared for agreement on classifications (i.e., screened or not screened). As shown in Table

3.13 the percent agreement for the 45 parents was 86.7 %. Intraclass correlations, as shown in

# of areas below cutoff

“at-risk” Frequency Percent

0 areas 721 82.3

1 area 114 13.0

2 areas 30 3.4

3 areas 6 .7

4 areas 3 .3

5 areas 2 .2

Total 876 100.0

# of Resources N % Identified

“At-Risk”

1-4 12 25.0

5-8 127 15.7

9-12 206 15.0

13-16 195 15.4

Over 17 112 15.2

Missing 224 24.1

Total 876 17.7

32


Table 3.14, ranged from a low of .43 to a high of .71 indicating robust test–retest reliability

across ASQ developmental areas.

Table 3.13

Test-Retest Agreement

Table 3.14

Intraclass Agreement

Internal Consistency The internal consistency of the questionnaires was addressed by

examining the relationship between developmental area and overall scores. Correlational

analyses and Cronbach coefficient alpha (Cronbach, 1951) were calculated. Pearson product

moment correlation coefficients were calculated for developmental area scores with an overall

ASQ-J score for questionnaire age intervals. Correlations were computed on the 8 prioritized

ASQ-J intervals, which had sufficient data to analyze. As shown in Table 3.15, the correlations

by developmental area and overall ASQ-J score are consistent and generally ranged from .59 to

.86. The major exception is the Gross Motor area, in which 5 correlations were below .57. All

correlations are significant at p < .01. These findings suggest moderate to strong internal

consistency between developmental areas and total test score.

Time 1 & Time 2

Classification (Same

or Different) Frequency Percent

Different 6 13.3

Same 39 86.7

Total 45 100.0

Communication 0.673

Gross Motor 0.614

Fine Motor 0.718

Problem Solving 0.434

Personal-Social 0.642

33


Table 3.15

Correlations between Area and Overall Score

Table 3.16 contains the correlations between developmental area scores that have been

collapsed across the 8 questionnaire age intervals. Again, all correlations were significant,

suggesting congruence between developmental areas as well as between developmental areas

and overall ASQ-J scores.

Table 3.16

Correlations between Area Scores Collapsing Across Questionnaires

Age

Interval n Communication

Gross

Motor

Fine

Motor

Problem

Solving

Personal-

Social

6 102 0.59 0.73 0.71 0.78 0.71

10 106 0.62 0.53 0.71 0.72 0.70

12 92 0.63 0.41 0.64 0.79 0.76

18 113 0.68 0.44 0.68 0.73 0.61

24 94 0.71 0.44 0.82 0.73 0.77

36 104 0.59 0.57 0.79 0.55 0.67

54 114 0.76 0.67 0.73 0.86 0.69

60 99 0.69 0.74 0.80 0.75 0.67

Area Communication

Gross

Motor

Fine

Motor

Problem

Solving

Personal-

Social

Communication

Gross Motor 0.32

Fine Motor 0.25 0.24

Problem

Solving 0.20 0.24 0.48

Personal-Social 0.25 0.21 0.28 0.37

Overall 0.63 0.56 0.70 0.72 0.65

Note: All correlations are significant at p < 0.01

34


The reliability of the ASQ-J questionnaires has been studied by examining the internal

consistency and test–retest reliability of the questionnaires. Internal consistency analyses have

indicated strong relationships across items and within areas on the questionnaires. The

questionnaires also achieved substantial test–retest reliability. Parents’ evaluations of their

children using the questionnaires were consistent over time.

Validity Studies

Financial and logistical constraints of the grant made the gathering of validity data on the

ASQ-J impossible.

Summary of Findings on the ASQ-J

Initial utility and psychometric studies on the ASQ-J are promising. Both providers and

parents found that the ASQ-J to be an appropriate and useful child developmental screening tool

for Jamaican populations. The current U.S. cutoffs seem to be operating similarly for Jamaican

populations and identifying a similar percentage of children “at risk” and in need of further

evaluation. Research indicates good reliability with the ASQ-J, both across users and internally

in the test construction.

Limitations and recommendations. There are limitations to this study. The vast majority

of data collected were from 3 parishes in Jamaica (i.e., Kingston, St. Andrews and St.

Catherine’s) and the majority (95%) in urban or inner urban settings. More research on the

appropriateness of this tool across parishes and in rural settings would be recommended. Most

likely access to some materials required to demonstrate skills (e.g., scissors, crayons) may be

limited in rural community and outcomes may differ. It will be important for administrators to

consider how they can make screening materials available to children and parents prior to

completing the ASQ-J. How much support parents will need to complete questionnaires is

unknown, given research assistants supported most parents who completed questionnaires.

Again, screening administrators will need to consider that parents may need some support while

completing the ASQ-J (e.g., reading, interpreting items). The number of questionnaires, and the

stratification of the data are not sufficient to establish specific “Jamaican” cutoffs as yet, so

continuing to gather data on this tool would be recommended.

35


Part II

SWYC Technical Report

This section offers a range of technical information about the Survey of Wellbeing of

Young Children (SWYC) and the development of the SWYC-Jamaica system, including focus

group and utility studies that led to selected questionnaire revisions. Normative studies in

Jamaica resulted in recruiting and collecting 647 questionnaires. The collected data have been

used to examine selected psychometric parameters of the SWYC. In addition to describing the

demographic characteristics of the samples, analyses included in this report addresses test–retest

reliability and measures of internal consistency. Validity analyses include descriptions of how

the cutoff points were determined and of measures of concurrent validity.

Introduction and Background. The Survey of Well-being of Young Children (SWYC) is a

freely-available, comprehensive screening instrument for children under 5 years of age. There

are four segments of the original SWYC, though the Family Context portion was omitted for

Project INSPIRE, as this area would be screened thoroughly using the Jamaican Family Support

Screening Tool. The three SWYC components currently being studied through the INSPIRE

project are: (1) the Milestones: a short developmental component with questions about

cognitive, language and motor development; (2) the BPSC and PPSC: questions focused on

children’s social-emotional and behavioral development; and (3) the POS: questions focused on

screening for autism.

Development of the SWYC Translation and adaptation of a measure for use in a different

culture requires several safeguards. As a number of authorities have cautioned, the adoption of a

measure into a different culture requires careful consideration to ensure its appropriate use (e.g.,

Ball & Janyst, 2008). Screening or assessment tools that are not culturally appropriate may yield

misleading or negative results and unintended consequences (Musquash & Bova, 2007). In

December 2012, University of Oregon researchers conducted a focus group in Kingston, Jamaica

on the SWYC with parents, teachers, Jamaican stakeholders, and University of West Indies

(UWI) research assistants. Focus group participants reported that the overall structure, intent and

items on the SWYC were very appropriate for children and families in Jamaica and had no

significant suggestions for changes.

The SWYC is intended to be used as the second-level screening tool during well child

checkups, when any behavioral concerns were identified. During well child checkups, all

36


children are currently screened at 6, 9, 18, 24 and 36 months using the “11 questions”, included in

Jamaica’s Child Health and Development Passport. The SWYC would be administered as a

second level screening when any behavioral concern was indicated on the “11 Questions”.



the SWYC. Research was divided into 2 phases; Phase 1 focused on the utility of the SWYC for

Jamaican children and families. Phase 2 focused on descriptive statistics, reliability and validity

studies. Following are research questions for each phase:


a. What is the utility (including cultural sensitivity) of the SWYC for children,


b. What modifications need to be made to the SWYC?


a. What are descriptive statistics (e.g., means, standard deviations) of the SWYC for

Jamaican children and families?

b. What is the reliability (test-retest) of the SWYC?

c. What is the validity of the SWYC (e.g., sensitivity, specificity) when compared to

known groups of children with/without disabilities?

Phase 1. Utility Data Collection on the Pilot SWYC

In early February 2013, the UWI research team began to collect pilot data on the utility of

the SWYC. Collection of utility data continued through mid-July of 2013. Research Assistants

interviewed parents in Health Clinics to complete the SWYC. Parents also completed utility

surveys indicating the clarity, appropriateness and meaningfulness of SWYC items. Results of

utility surveys are summarized below. In May 2013 during a visit to UWI, University of Oregon

researchers conducted a second round of focus groups with parents and research assistants on the

piloted SWYC. Participants in the May focus groups were satisfied with the tool and did not

suggest any changes.

Utility Survey Data Collection Seventy-nine (79) utility surveys were collected on the

Survey of Wellbeing of Young Children (SWYC) across intervals from 2-36 months. Five of the

37


intervals (i.e., 6, 19, 18, 24, 36) are “priority intervals”. These intervals correspond to well child

checkups conducted in the Jamaican health care system; the additional intervals were more

difficult to gather utility data on since few children are seen at clinics at these times. Data were

collected across SWYC intervals, however, to have information on a comprehensive set of

questionnaires. The following tables summarize the number of utility surveys collected by

interval (Table 3.17) and responses to the utility questionnaire (Table 3.18). Seventy-five percent

(75%) of parents reported that the SWYC took less than 10 minutes to complete, (15 parents did

not respond to this question).

Table 3.17

Number of Utility Surveys Collected by SWYC Interval

Summary of Utility Findings Utility data on the SWYC collected thus far, while limited, are

positive. One hundred percent (100%) of respondents strongly agreed/agreed that the questions

on the SWYC clear and easy to understand. One hundred percent (100%) of respondents strongly

agreed/agreed that questions are appropriate for their child and family’s culture. A little over

ninety-three percent (93.4%) of respondents strongly agreed/agreed that completing the tool

provided meaningful information about the child while 93.6% of respondents strongly

agreed/agreed that they would like to use this screening tool again. More than 97% percent of

respondents strongly agreed/agreed that the questions were appropriate for their child’s age.

When asked how to change the tool to make it better, 93.7% said they had no changes. Based on

current findings and anecdotal reports from the research team from the University of West Indies,

the SWYC appears to be an appropriate and useful child developmental/ behavioral screening tool

for Jamaican populations.

SWYC

intervals

2

month

6

month

9

month

12

month

18

month

24

month

36

month

# of completed

utility surveys

2

12

13

7

17

11

13

38


Table 3.18

SWYC: Summary of Responses to Utility Statements*

Total Strongly

Agree Agree No Opinion Disagree Strongly

Disagree Missing

N N (%)

N (%)

N (%)

N (%)

N (%)

N (%)

In general,

questions clear

and easy to

understand

79 39 (49.4%)

40 (50.6%) 0 (0%) 0 (0%) 0 (0%) 0 (0%)

Questions

appropriate for

child and

family’s

culture

79 31 (39.2%) 46 (58.2%) 1 (1.3%) 1 (1.3%) 0 (0%) 0 (0%)

Completing

tool provided

meaningful

Information

79 18 (22.8%) 40 (50.6%) 9 (11.4%) 12 (15.2%) 0 (0%) 2 (6.7%)

Would like to

use this tool

again

79 25 (31.6%) 49 (62.0%) 3 (3.8%) 2 (2.5%) 0 (0%) 2 (6.7%)

Question

appropriate for

child’s age

79 22 (27.8%) 54 (68.4%) 1 (1.3%) 1 (1.3%) 0 (0%) 1 (1.3%)


on a 5 point Likert Scale, (Strongly Agree, Agree, No Opinion, Disagree and Strongly Disagree.

Phase 2. Psychometric Data Collection on the SWYC

The second phase of data collection involved collecting psychometric data on the field

version of the SWYC. Questionnaires were completed by parents and caregivers accessing

Jamaican health services. Research staff from the University of West Indies assisted in

identifying settings that provided as well stratified a sample of Jamaican families and children as

possible, given financial and logistical constraints of the project. All parents/caregivers in

targeted settings were provided with information about the child and family risk screening

projects and asked if they would be willing to participate. As part of participation in the research,

39


parents received an SWYC questionnaire along with a form asking for the child’s demographic

information and a research consent form. The SWYC questionnaire was completed either

independently by the parent or with assistance from a research assistant or a healthcare provider.

The completed questionnaires were scored, and the results were shared with the parent or

caregiver. The provider assisted the parent with referrals to community support services as

needed. Procedures ensuring protection of human participants were approved by the University

of West Indies and the University of Oregon institutional review boards and were followed in all

research phases.

The following SWYC data are based on 647 questionnaires completed by parents of

children between 1 and 36 months of age. The numbers of questionnaires collected by interval

are shown in Table 3.19.

Table 3.19

Frequency and Percentage of Data Collected by SWYC Interval

Population Sample

Each parent or caregiver who completed a questionnaire was asked to complete a

demographic form. Demographic data included information on the child’s age, gender, parish the

child resides, setting screening took place, who completed screening, number of family

household resources (as a proxy of risk), and whether the child was known to have any health

risks or known disabilities. It should be noted that there were some missing demographics, so

Interval Frequency Percent

2 23 3.6

6* 106 16.4

9* 106 16.4

12 102 15.8

18* 113 17.5

24* 92 14.2

36* 105 16.2

Total 647 100

* These intervals were prioritized during data collection

40


percentages reported are valid percentages, and based on the number of data for which

demographic information was collected. Demographic data for the population sample are

displayed in Tables 3.20-3.23.

Gender of child and parish child resides. As shown in Table 3.20, the gender distribution

for the sample was 54% male and 45.5% female. Table 21 contains data on the parish where the

child lived at the time of the data collection. The majority of children lived in Kingston (61.1%),

St. Andrew (30.9%), or St. Catherine (6.6%) parishes. A small percentage of children (1.5%)

resided in other parishes in Jamaica.

Table 3.20

Gender of Children

Table 3.21



Male 330 54

Female 276 45.5

Total 606 100

Missing 41

Total N 647

Parish Frequency

Valid

Percent

Kingston 380 61.1

St Andrew 192 30.9

St Catherine 41 6.6

Other 9 1.5

Total 622 100

Missing 25

Total 647

41


Settings Screenings Took Place Table 3.22 contains data on the setting where the

screening took place. A large percentage (92.8%) of screenings occurred in health settings,

primarily in health clinic settings during well child checkups. Approximately ninety-four

percent (94.2 %) of these settings were public and 5.5 % were private settings. Almost sixty

percent (59.6%) of screenings were conducted in urban settings, 34.7% were conducted in

“inner” urban settings, while only 5.6 % of screening occurring in rural settings. See the

Limitations of Study section for a discussion of the imbalance or urban/rural settings and

recommendations for future research on this tool.

Table 3.22


Person Completing Questionnaires Table 3.23 contains data on the person completing the

questionnaires. The majority of individuals completing the questionnaires were mothers (93.2%).

This finding is consistent with feedback from hundreds of screening professionals who report

that mothers are most apt to complete the questionnaires on their children. It should be noted

that the majority of these (86.2%) were completed with the support of a research assistant.

While SWYC questionnaires can be completed independently by parents, data collection

methods may have increased the amount of support research assistants provided. In order to


Antenatal 18 2.9

Clinic 558 89.7

PATH Home Visit 1 .2

Early Childhood

Educational

3 .5

Primary School 0 0

Nursery 19 3.1

Other 23 3.7

Total 622 100

Missing 25

Total 647

42


determine how much support is required in actual health and educational settings in Jamaica,

further research would need to be conducted.

Table 3.23


Household resources (child risk). Data on the number of household resources was



entered for each family. These data are displayed in Table 24. A little over two percent (2.3%)

of the sample reported having between 1-4 household resources, 19.4% reported having 5-8

resources, 31.6 having 9-12 resources, 30.8% having 13-16 resources, 14.7% having 17-20

resources, and 1.3% having 21-24 resources.

Cutoff Scores and Percent Identified of Population

Because of the limited number of questionnaires available to establish “Jamaican”

cutoffs, an approach examining the percentage of children identified as “at-risk” was chosen to

determine cutoffs for the SWYC.

Using the INSPIRE data set, children were classified as “at-risk” if scores were below

established U.S. cutoffs, or “no-risk” if scores were above established U.S. cutoffs. At first, three

Frequency

Valid

Percent

Mother 580 93.2

Father 16 2.6

Grandmother 14 2.3

Aunt 5 .8

Stepmother 0 0

Sister 1 .2

Other-family 5 .8

Non-family 1 .2

Total 622 100

Missing 25

Total 647

43


SWYC component were examined including, (1) the Milestones developmental screen (2) the

BPSC and PPSC social-emotional/behavioral screens and (3) the POS autism screen. Each

section of the SWYC was examined separately.

Table 3.24

Total Number Household

Targets of 12%–16% of children identified in the Milestones developmental segment of

the SWYC were adopted as the desired percentages to be identified for further assessment at

each age interval. These figures were based on U.S. Census Bureau and Centers for Disease

Control and Prevention prevalence data related to developmental disabilities in young children as

well as additional resources on worldwide disability prevalence (Cornell University, 2003–2009;

U.S. Census Bureau, 2004).

Targets of 16%-20% of children identified in the social-emotional components of the

SWYC (i.e., the BPSC, PPSC, POSI) were adopted as the desired percentages to be identified for

further assessment at each age interval. These figures were based on U.S. Census Bureau and

Centers for Disease Control and Prevention prevalence data related to social emotional

disabilities in young children as well as additional resources on worldwide disability prevalence

(Cornell University, 2003–2009; U.S. Census Bureau, 2004).

Table 3.25 contains data on the total number of Jamaican children identified as “at-risk”

on the Milestones developmental screening tool. For intervals 2-12 months, 6.5% of children


1-4 14 2.3

5-8 119 19.4

9-12 194 31.6

13-16 189 30.8

17-20 90 14.7

21-24 8 1.3

Total 614 100

Missing 33

Total 647

44


were identified as “at-risk” which is low compared to the target of identifying 12-16% of

children in one developmental area. For intervals 18-36 months, 13.3% of children are identified

as “at-risk”, which is within the target identification range. While these findings are promising,

during a consultation with researchers from Tufts University (the developers of the

SWYC), it was learned that the Milestones section of the SWYC was substantially revised

from the version available December 2012. For this reason, the data collected on the

Milestones component of the SWYC in this study are not valid. Other sections of the

SWYC have not been revised at this time and the data are valid for these sections.

Table 3.26 contains data on the percentage of Jamaican children identified as “at-risk” on

the BPSC, which is the social-emotional screening component of the SWYC, for children 2-12

months. Using the recommended cutoff of 3 or more concerns in any one of 3 sections, a high

percentage of children (46.4%) are identified. Because of this finding, an analysis was also run

on the percent of children identified as “at risk” using 4 or more concerns. This analysis resulted

in 30.5% of children being identified as “at-risk”. Even with an adjusted cutoff, this percentage

is high when compared to target identification rates of 16-20%.


the PPSC, which is the social-emotional screening component of the SWYC for children 18-36

months. Using the recommended U.S. cutoff of 9 or more concerns, 30.4% of children are

identified as “at-risk”. This identification rate is high for the recommended target identification

range.


the POSI, which is the autism screening component of the SWYC for children older than 18

months. Using the recommended U.S. cutoff, 26.5 % of children are identified as “at-risk” for

autism. Similar to the PPSC and BPSC, this percentage is high when compared to target

identification rates.

45


Table 3.25

Percent Identified on Milestones Component of SWYC, 2-12 & 18-36 Months

Table 3.26

Percent Identified on BPSC Using 3 or 4 Concerns as Cutoff*

Table 3.27

Percent Identified on PPSC

Classification

Frequency

(2-12)

Percent

(2-12)

Frequency

(18-36)

Valid Percent

(18-36)

Typical 287 93.5 242 86.7

“At-Risk” 20 6.5 37 13.3

Total 338 100 279 100

Missing 0 16

Classification

Frequency

(3 concerns)

Percent

(3 concerns)

Frequency

(4 concerns)

Percent

(4 concerns)

Typical 181 53.6 235 69.5

“At-Risk” 157 46.4 103 30.5

Total 338 100 338 100

*N=338 for intervals 2-12 months

Classification Frequency Percent

Typical 215 69.6

“At-Risk” 94 30.4

Total 309 100.0

*N=309 for intervals 18-36 Months

46


Table 3.28

Percent Identified on POSI

Reliability Studies

Reliability studies completed on the ASQ-3 include test–retest reliability and internal

consistency of ASQ-3 items. Internal consistency was examined using correlational analyses

and the Cronbach coefficient alpha (Cronbach, 1951). Each of these analyses is presented next.

Test–Retest Reliability. Test–retest reliability is designed to help determine the stability

of test outcomes over time. Test–retest reliability of the SWYC was examined by comparing two

questionnaires completed by the same caregiver at a 2-3 week time interval. That is, parents were

asked to complete the same questionnaire interval for their child twice within a 2-3 week time

period between completions. Parents were blind to the results of the first questionnaire when

they completed the second one. The two questionnaires completed by parents were then

compared for agreement on classifications (i.e., typical or “at-risk”). Test-Retest was analyzed

separately for each component of the SWYC (i.e., the BPSC, PPSC and POSI).

As shown in Table 3.29 the percent agreement for the BPSC component of the SWYC

(N=27) was 81.5 %, indicating strong test-retest agreement (intraclass correlation .86). Test-

retest agreement for the PPSC (N=17) was 82.4%, indicating strong test-retest agreement

(intraclass correlation .86). Test-retest agreement for the POSI (N=17) was 29.4%, which is

considered poor agreement (intraclass correlation of -2.1). It is difficult to draw conclusions on

these data given the low N, particularly for the analysis of the PPSC and the POSI. It is

recommended to continue to gather data on these tools.

Classification Frequency Valid Percent

Typical 227 73.5

“At-Risk” 82 26.5

Total 309 100.0

N=309 for 18-36 Month Intervals

47


Table 3.29

Test-Retest Agreement on Classification (typical/”at-risk”) on BPSC, PPSC, and POSI

Components of the SWYC

The reliability of the SWYC questionnaires has been studied by examining test–retest

reliability of the questionnaires. The questionnaires achieved substantial test–retest reliability for

the BPSC and PPSC social-emotional screening components of the SWYC. However, the POSI

component of the questionnaires had unacceptable test-retest reliability. These results need to be

interpreted with caution, however, given the small number of participants in test-retest analysis.

Validity Studies

Financial and logistical constraints of the grant made the gathering of validity data on the

SWYC impossible.

Summary

Initial utility studies on the SWYC are very promising. Both providers and parents found

that the SWYC to be an appropriate and useful social-emotional/behavioral screening tool for

Jamaican populations. Findings on the psychometric qualities of the SWYC have varying results

based on each component studied. The Milestones section of the SWYC has been substantially

revised from the version available December 2012 and data collected in this research are not

valid. The other components of the SWYC (i.e., the BPSC, PPSC and POSI) have not been

revised at this time and these components are the sections of the SWYC that provide information

on the child’s social-emotional/behavioral development (the targeted purpose for use of this tool

in Jamaica). Identification rates for the social-emotional screening components of the SWYC

(the BPSC for intervals 2-12 months and the PPSC for intervals 18 months and older) were

Time 1 & Time 2

Classification

BPSC

N (%)

PPSC

N (%)

POSI

N (%)

Same 22 (81.5%) 14 (82.4%) 5 (29.4%)

Different 5 (18.5%) 3 (17.6%) 12 (70.6%)

Total 22 (100%) 17 (100%) 17 (100%)

48


analyzed. The BPSC identified a very high percentage of Jamaican children as “at-risk” (46.4%)

using recommended U.S. cutoffs. Adjusting the cutoffs (i.e., using 4 or more concerns as “at-

risk”) resulted in 30.5% of Jamaican children being identified. This identification rate is still

high. The PPSC identified 30.4% of children as “at-risk”. This identification rate is high. The

POSI, the autism screening component of the SWYC for children18 months and older, identified

26.8% of children as “at-risk” for autism. Similar to the BPSC and PPSC, this percentage is high

when compared to target identification rates.

Test-retest reliability on the BPSC and PPSC were high (81.5 and 82.4, respectively),

while test-retest reliability on the POSI was poor (29.4). Reliability data are difficult to draw

conclusions from, given the low number of participants included in the analysis, which ranged

from 27 participants (for the BPSC) to 17 participants for the PPSC and POSI analysis. These

analyses should be repeated with a larger data sample.

Limitations and recommendations. There are limitations to this study. The vast majority

of data collected were from 3 parishes in Jamaica (i.e., Kingston, St. Andrews and St.

Catherine’s) and the majority (95%) in urban or inner urban settings. More research on the

appropriateness of this tool across parishes and in rural settings would be recommended. How

much support parents will need to complete questionnaires is unknown, given research assistants

supported most parents who completed questionnaires. Again, screening administrators will need

to consider that parents may need some support while completing the SWYC (e.g., reading,

interpreting items).

Because of the high percentage rates of children being identified as “at-risk” on the

BPSC, PPSC and POSI, screening administrators need to be prepared for a potentially large

percentage of screenings receiving a “red flag” that requires personnel to interpret screening

information and appropriate follow-up necessary (if any). It is difficult to know if these high

identification rates will persist as the tool is implemented in Jamaica and more data are collected.

It is possible that children in Jamaica have a higher number of social-emotional/behavioral

concerns than the estimated prevalence rates of these delays. It is also possible that, because

these components of the SWYC have only undergone preliminary studies, the tool may require

revisions to test items and/or recommended cutoff scores to bring down identification rates.

Because of the importance of screening for social-emotional and behavioral delays it is

recommended to continue using the SWYC as a second level screening when a parent indicates a

49


concern about a child’s behavior. It is also recommended that Jamaican screening system

developers keep up to date with changes to the SWYC. It may be important to update this tool

as additional research is conducted both in Jamaica and the U.S. For current protocols and

research on the SWYC, please go to: http://www.theswyc.org/ In addition, continuing to gather

psychometric data on this tool, including identification rates, reliability and validity data, is

highly recommended.

50

http://www.theswyc.org/


Chapter 4: School Readiness Screening


The Jamaica School Readiness Screening was designed to screen children for child

development and readiness for primary school in order determine whether additional

developmental evaluation is necessary, and to assist in curriculum-planning to support children’s

readiness for primary school. Preschool teachers are responsible for completion of the School

Readiness Screening with all children in four-year-old preschool classrooms during Easter term.

Teachers may complete the tool independently or in collaboration with a child’s parents or main

caregivers. The School Readiness tool includes 3 component parts: 1) Part 1: Ages and Stages

Questionnaires-Jamaica (ASQ-J) (Squires et al., 2009), the child development screening tool; 2)

Part 2: Child Behavior Rating Scale; and 3) Part 3: Approaches to Learning/Social-Emotional,

Literacy, and Numeracy Skills. Parts 2 and 3 are described briefly below. Part 1: The ASQ-

Jamaica (ASQ-J) is presented in Chapter 3 of the Technical Report. The ASQ-J is intended to

be completed by the parent with support from the child’s teacher as necessary, while the CBRS

and the School Readiness Skills are completed by the child’s preschool teacher.

Jamaica School Readiness Part 2: The Child Behavior Rating Scale (Bronson,

Goodson, Layzer, & Love, 1990).

The Child Behavior Rating Scale (CBRS) is a seventeen-item survey that assesses a child’s self-

regulatory skills, behaviour with other adults and children in a classroom setting, and social-

emotional development. The CBRS was selected as a component of the School Readiness tool

because self-regulatory and social emotional skills at kindergarten entry have been demonstrated

to be strong predictors of later school success. The CBRS has been used widely in research

studies and has demonstrated strong predictive validity with reading and math achievement in

elementary grades (Sektnan, McClelland, Acock, & Morrison, 2010). It also has been validated

in a wide range of cultural contexts.

Jamaica School Readiness Part 3: School Readiness Skills: Approaches to

Learning/Social Emotional, Early Literacy and Early Numeracy

The School Readiness Skills (SRS): Approaches to Learning/Social Emotional, Early

Literacy and Early Numeracy comprise Part of 3 of the School Readiness Screening. The

51


purpose of this component is to identify areas in which children need to develop competence in

order to be successful in primary school. The School Readiness Skills were developed in 2012

following a literature review of existing school readiness tools, other assessment measures and

existing definitions of school readiness to identify areas most predictive of school readiness;

researchers also looked at the Jamaica Early Childhood Curriculum. This process of review

guided the development and selection of items. The School Readiness Skills were then adapted

in 2013 to incorporate feedback from focus groups with teachers, parents and other key early

childhood stakeholders in Jamaica. The School Readiness Skills address skills universally

identified as important for school readiness and are completed by teachers through daily

observations of students in the classroom setting. Results from the School Readiness Skills may

guide teachers in preparing children for entry into primary school.

Development of the School Readiness Screening. In order to develop the School

Readiness Screening, the University of Oregon research team completed a literature review of

existing school readiness tools and of definitions of school readiness to identify areas most

predictive of school readiness. The Jamaica Early Childhood Curriculum was also reviewed and

consulted. In addition to the child development screening component (ASQ-J), the research team

identified behavior, readiness to learn, social emotional, early literacy and early numeracy as

important skills to include on a school readiness tool and skills with most predictive validity for

later school success. A draft version of the School Readiness screening tool was developed and

included three parts: 1) the ASQ-J child development screening, 2) the Child Behavior Rating

Scale (CBRS), and 3) the School Readiness Skills (SRS). Input on the draft version of all three

components of the School Readiness Screening was gathered from stakeholders during a focus

group. Based on the focus group feedback, items were adapted and/or revised and clarifying

examples included, resulting in the pilot version used for initial data collection.



the CBRS and SRS. Research was divided into 2 phases. Phase 1 focused on the utility of the

CBRS and SRS for Jamaican children and families; Phase 2 focused on psychometric properties

including descriptive statistics, reliability and validity studies. The goal of Phase 2 was to

examine the psychometric qualities of the tools as well as develop cutoff scores for identifying

52


children who need additional support to be ready to enter the primary school setting. Following

are research questions for each phase:


a. What is the utility (including cultural sensitivity) of the CBRS and School

Readiness Skills for children, families and teachers in Jamaica?

b. What modifications need to be made to the CBRS and School Readiness Skills?


a. What are descriptive statistics (e.g., means, standard deviations) of the CBRS and

School Readiness Skills for Jamaican children and families?

b. What is the reliability (test-retest) of the CBRS and School Readiness Skills?

i. What is the inter-observer reliability for the Early Literacy and Early

Numeracy areas of the School Readiness Skills?

ii. What is the internal consistency of the School Readiness Skills?

c. What is the validity of the CBRS and School Readiness Skills? (e.g., sensitivity,

specificity) when compared to known groups of children with/without

disabilities?

3) Phase 1. Utility Data Collection on the Pilot School Readiness Screening,

Parts 2 and 3

4) During the Pilot Phase, preschool teachers completed Part 2: CBRS and Part 3: School

Readiness Skills of the School Readiness Screening with children in their classrooms.

They completed Part 1: ASQ-J in collaboration with parents and main caregivers.

Teachers then provided utility feedback on the clarity, appropriateness and

meaningfulness of items on all three parts of the tool School Readiness Screening. A copy

of the utility survey can be found in Appendix A at the end of this technical report.

5) The results from Part 1: ASQ-J are contained in Chapter 3 of the Technical Report. The

utility data for Part 2: the CBRS (N = 39) and Part 3: the School Readiness Skills (N = 49)

are presented in Tables 4.1 and 4.2 below.

6) Thirty-nine utility surveys were collected on the CBRS. The following table summarizes

CBRS tool utility data.

53


Table 4.1

Part 2: CBRS Summary of Responses to Utility Statements*

Total Strongly

Agree

Agree No

Opinion

Disagree Strongly

Disagree

Missing

N N (%)

N (%)

N (%)

N (%)

N (%)

N (%)

In general,

questions clear

and easy to

understand

39 15

(38.5%)

23 (59%) 0 (0%) 1 (2.6%) 0 (0%) 0 (0%)

Questions

appropriate for

child and

family’s culture

39 11 (28 %) 27 (69%) 0 (0%) 1 (3%) 0 (0%) 0 (0%)

Completing tool

provided

meaningful

Information

39 10 (26%) 25 (64 %) 4 (10%) 0 (0%) 0 (0%) 0 (0%)

Would like to

use this tool

again

39 9 (23%) 29 (74%) 0 (0%) 1 (3%) 0 (0%) 0 (0%)

Question

appropriate for

child’s age

39 12

(30.8%)

26

(66.7%)

0 (0%) 1 (2.6%) 0 (0%) 1 (3%)



Forty-nine utility surveys were collected on the School Readiness Skills. The following

table summarizes School Readiness Skills utility data.

54


Table 4.2

Part 3: School Readiness Domains Summary of Responses to Utility Statements*

Total Strongly

Agree

Agree No

Opinion

Disagree Strongly

Disagree

Missing

N N (%)

N (%)

N (%)

N (%)

N (%)

N (%)

In general,

questions clear

and easy to

understand

49 15 (31%)

33 (67%) 0 (0%) 1 (2%) 0 (%) 0 (%)

Questions

appropriate for

child and

family’s culture

49 12 (25%) 34 (69%) 0 (0%) 3 (6%) 0 (%) 0 (%)

Completing tool

provided

meaningful

Information

48 12 (25%) 31 (65%) 2 (4%) 2 (6%) 0 (%) 1 (2%)

Would like to

use this tool

again

48 10 (21%) 37 (77%) 1 (2%) 0 (0%) 0 (%) 1 (2%)

Question

appropriate for

child’s age

47 13 (28%) 33 (70 %) 0 (0%) 0 (0%) 1 (2%) 2 (4%)



Additional written comments included the following responses to questions posed on the

utility survey.

Question 1: If you disagreed or strongly disagreed with any of the statements, please tell us

why.

Child Behavior Rating Scale

“I would change the scale and use: “Not all the time”, “Sometime”, “Most of the

time” as the scale was ambiguous and difficult to match the item and student

behavior.”

55


“Rubric wasn’t clear, a little confusing.”

School Readiness Skills

“I found this tool to be appropriate for the age group, but in the same breath

challenging to complete and manage a full class of over 30 students.”

“Not Age appropriate; some children just started school in September (have only

had one term of school). Just four this year.”

“The questions were in keeping with the basic achievement required for children

who are 5 years old. The instrument was quite comprehensive in covering the

affective and cognitive areas of development.”

Question 2: How would you change this tool to make it better?


“I would not change this tool for now.”

“Some of the questions a bit vague. Didn’t give a very clear picture of children’s

needs.”


“Change word number to numeral.”

“Make the questions a little more simpler/shorter.”

Question 3: We welcome further comments and suggestions. Feel free to write them below:


“After using this tool make appropriate helpline for the children.”

“Based on the age and stage of development of 5yr olds the questions were

consistent with same.”

“I think this program is well needed and should be widely expanded.”

This will help me to know my children even better, not only the ones being

assessed.”


“Felt was a very good assessment: it brings out what children could and could not

do so weaknesses could be worked on.”

“It is comparable to the assessment done at end of term to see how child is

progressing.”

56


“Questions opened eyes to child’s real and not assumed abilities.”

Summary of Utility Findings Utility data on the CBRS were quite positive. Almost all—97.5%

of respondents--strongly agreed/agreed that the questions on the CBRS are clear and easy to

understand. Ninety-seven percent of respondents strongly agreed/agreed that questions were

appropriate for their child and family’s culture. Ninety percent of respondents strongly

agreed/agreed that completing the tool provided meaningful information about the child; Ninety-

seven percent of respondents strongly agreed/agreed that they would like to use this screening

tool again. Current findings and anecdotal reports from the UWI research team suggest that the

CBRS will be an appropriate and useful screening tool for determining school readiness.

Utility data on the School Readiness Skills were also quite positive. Almost all--98% of

respondents--strongly agreed/agreed that the questions on the School Readiness Skills are clear

and easy to understand. Ninety-four percent of respondents strongly agreed/agreed that questions

were appropriate for their child and family’s culture. Ninety percent of respondents strongly

agreed/agreed that completing the tool provided meaningful information about the child. Ninety-

eight percent of respondents strongly agreed/agreed that they would like to use this screening

tool again. Current findings and anecdotal reports from the UWI research team suggest that the

School Readiness Skills will be an appropriate and useful screening tool for determining school

readiness.

Phase 2. Psychometric Data Collection on the School Readiness Screening,

Parts 2 and 3

The second phase of data collection involved collecting psychometric data on the field version of

the School Readiness Screening. Research staff from the University of West Indies assisted in

identifying 4-year preschool classrooms that provided a well-stratified a sample of Jamaican

families and children as possible, given financial and logistical constraints of the project. All

participating parents/caregivers of children in targeted settings were provided with information

about the School Readiness screening project and asked if they would be willing to participate.

As part of participation in the research, parents/caregivers received from their child’s teacher or a

UWI researcher a form asking for the child’s demographic information as well as a research

consent form. Teachers were also asked to complete a teacher demographic information form as

57


well as a research consent form. Part 2: Child Behavior Rating Scale (CBRS) and Part 3: School

Readiness Skills (SRS) of the School Readiness Screening were completed by teachers after a

researcher had provided them with an introduction to the screening project. Procedures ensuring

protection of human participants were approved by the University of West Indies and the

University of Oregon institutional review boards and were followed in all research phases.

The following data are based on 236 School Readiness screenings completed by teachers

of children between 42 and 66 months of age, in 4-year preschool classrooms.

Population Sample for the School Readiness Screening

Child and Family Information. The data analyses that are contained in this report are

based on 236 completed School Readiness screenings, which included both Part 1: CBRS and

Part 2: SRS. Each parent or caregiver who completed a questionnaire was asked to complete a

demographic form. Demographic data included information on the child’s age, gender, parish the

child resides, setting screening took place, who completed screening, and number of family

household resources (as a proxy of risks). It should be noted that there were 197 missing

demographics for Part 1: CBRS, and 195 missing demographics for Part 2: SRS; percentages

reported are based on the 41 CBRS data and 39 SRS data for which demographic information

was collected. The demographic data for the population sample are displayed in Tables 4.3 and

4.4. A copy of the demographic form can be found in the Appendix of this report.

Gender and age of child and parish where child resides. As shown in Table 4.3, the

gender distribution for the CBRS sample was 51% male and 49% female. Table 4.4 includes

information about the age of children included in the study. Because demographic information

was not available for these children, researchers looked at the age intervals of School Readiness

Screening Part 1:ASQ-J to determine approximate ages. These ages presume that children were

given the correct ASQ-J interval; birthdate information is not available to confirm this. Table 4.5

contains data on the parish where the child lived at the time of the data collection. The majority

of children lived in Kingston (51%), St. Andrew (26.8%), or St. Catherine (22%) parishes.

Settings Screenings Took Place. Table 4.6 contains data on the setting where the School

Readiness screening took place. A large percentage (93.7%) of School Readiness screenings

occurred in Early Childhood Educational and Primary school settings. 7.3% took place in “other”

settings. Approximately 68% of these settings were public and 32% were private settings. Close

to 98 percent (97.5%) of screenings were conducted in urban settings (15.4% of the total sample

58


was considered “inner” urban), while only 2.6% of screening occurring in rural settings. See the

Limitations of Study section for a discussion of the imbalance or urban/rural settings and

recommendations for future research on this tool.

Table 4.3

Gender of Children

CBRS School Readiness Skills

Frequency Valid Percent Frequency Valid Percent

Male 21 51.2 20 51.3

Female 20 48.8 19 48.7

Total 41 100.0 39 100.0

Missing 195 197

Total N 236 236

Note: Sample N’s were calculated separately for CBRS and School Readiness

Table 4.4

Age of Children by ASQ-J Interval


45-51 18 8.0

51-57 109 48.0

57-66 100 44.0

Total 227 100.0

Missing 9

Total N 236

59


Table 4.5


Table 4.6


Household resources (child risk). Data on the number of household resources was



entered for each family. These data are displayed in Table 4.6. A little less than twenty percent

(19.5%) reported having 5-8 resources, 34.2% reported having 9-12 resources; 29.3% reported

having 13-16 resources; 14.6% reported having 17-20 resources, and 2.4% reported having more

than 21 household resources.


Frequency Valid Percent Frequency Valid Percent

Kingston 21 51.2 21 53.8

St Andrew 11 26.8 9 23.1

St Catherine 9 22.0 9 23.1

Total 41 100.0 39 100.0

Missing 195 197

Total N 236 236


Setting Frequency Valid Percent Frequency Valid Percent

Early Childhood

Educational 35 85.4 35 89.7

Primary School 3 7.3 1 2.6

Other 3 7.3 3 7.7

Total 41 100.0 39 100.0

Missing 195 197

Total N 236 236

60


Teacher information. Demographic data were also collected from the preschool teachers

who completed School Readiness Screenings and included information on the teacher’s years of

experience working with preschool age children, setting where the screening took place, the

teachers’ job title or role, years of experience in her current job, age, level of education, degree

and additional college coursework. Data were collected from a total of 43 participating teachers,

between the ages of 27 and 61.

Table 4.7

Total Number of Household Resources

Years experience working with preschool children. Table 4.8 contains data on the number

of years of experience participating teachers had working with preschool Children. Only three

(7.3%) of teachers had 5 or fewer years of experience, while eight teachers (19.5%) had between

six and ten years; ten teachers (24.4%) had between 11 and 15 years; ten teachers had between

16 and 20 years; and only three teachers (7.3%) had over 30 years of experience working with

preschool children.

Setting where screening took place / Job title or role / Years in current job. Teachers

were also asked where they worked, their current job title or role, and the number of years they

had been in their current. Tables 4.9 through 4.11 present teacher responses. Twenty-five

# of

Resources Frequency Valid Percent

1-4 0 0.0

5-8 8 19.5

9-12 14 34.2

13-16 12 29.3

17-20 6 14.6

21-24 1 2.4

Total 41 100

Missing 195

Total N 236

61


(59.5%) of the teachers reported working in Public Early Childhood Educational Institutions,

while 17 (40.5%) reported working in Private Early Childhood Educational Institutions. The

majority of those participating were teachers (86%) with three Teacher Assistants (7%) and three

others, one principal, one Senior Teacher, and one Assistant Administrator/Supervisor. Eighteen

teachers (43.9%) reported being in their current job for 10 years or less, while 16 teachers (39%)

report working in their current job for between 11 and 20 years. Only seven teachers (17%)

indicated they had over 20 years of experience in their current job.

Table 4.8

Years of Experience Working with Preschool Children

# of Years Frequency Valid Percent

1-5 3 7.3

6-10 8 19.5

11-15 10 24.4

16-20 10 24.4

21-25 2 4.9

25-30 5 12.2

31-35 1 2.4

35-40 2 4.9

Total 41 100

Missing 2

Total N 43

62


Table 4.9

Where Do You Work

Table 4.10

Current Job Title or Role

Level of Education / Degree / College Coursework. Teachers also reported on their level

of education and coursework, illustrated in Tables 4.12 and 4.13 Thirteen of the teachers (31.7%)

had either an Associates or Bachelor’s Degree, while 18 others (43.9%) reported having some

college coursework. Twenty-five of the teachers (69.4%) reported that most of their college

coursework was related working with young children and families. It should be noted that

responses to this question were recorded for only 36 out of the 43 teachers.


Public Early Childhood

Educational Institution 25 59.5

Private Early Childhood

Educational Institution 17 40.5

Total 42 100

Missing 1

Total N 43


Teacher 37 86.0

Teacher Assistant 3 7.0

Other 3 7.0

Total N 43 100

63


Table 4.11

Years of Experience Working in Current Job

Table 4.12

Level of Education

# of Years Frequency Valid Percent

1-5 11 26.8

6-10 7 17.1

11-15 8 19.5

16-20 8 19.5

21-25 1 2.4

25-30 4 9.8

31-35 1 2.4

35-40 1 2.4

Total 41 100

Missing 2

Total N 43


Primary School 1 2.4

Second School Diploma 9 22.0

Some college 18 43.9

Associate’s Degree (AA) 3 7.3

Bachelor’s Degree 10 24.4

Total 41 100

Missing 2

Total N 43

64


Table 4.13

College Coursework Related with Working with Young Children and their Families

Cutoff Scores and Percent Identified of Population for the School Readiness Screening

A main goal of Phase 2 was to establish cutoff scores for the School Readiness Screening

in order to help teachers to determine whether additional developmental evaluation is necessary,

and to assist in curriculum planning to support children’s readiness for primary school. As

discussed earlier, cutoff scores for Part 1: ASQ-J of the School Readiness Screening are

discussed in Chapter 3. Cutoff Scores for Part 2 and Part 3 are discussed below. In order to assist

with the development of cutoff scores, researchers intended to collect a known groups validity

sample of School Readiness Screenings on children who had been previously identified with

developmental delays by teachers and/or developmental pediatricians. However, due to limited

resources, the collection of this sample was not possible.

Cutoff Scores for Part 2: Child Behavior Rating Scale. The cutoff score for the CBRS

was set at 6 or more total concerns. This cutoff score was based on analyses of the means and

standard deviations for concern scores in the normative sample. The mean number of concerns

for children in the normative sample was 2.7, while the standard deviation was 3.15. The cutoff

score of 6 was set at one standard deviation above the mean. The percentage of children in the

normative sample identified for additional support or referral by this cutoff was 16.5%.

Cutoff Scores for Part 3: School Readiness Skills. An individual cutoff score for each of

the three areas of the School Readiness Skills (Approaches to Learning/Social Emotional, Early

Literacy and Early Numeracy) were established using the same method as the CBRS. As

illustrated in Table 4.14, cutoffs were set at one standard deviation below the mean for each area


None 1 2.8

Some (25%) 6 16.7

Half (50%) 4 11.1

Most (75%) 25 69.4

Total 36 100

Missing 7

Total N 43

65


of the School Readiness Skills. Although one standard deviation below the mean in the Early

Numeracy Area is 8.4, this number was rounded up to a cutoff of 9 as the cutoffs are set as whole

numbers, with a score below the cutoff considered to be cause for follow-up. Using the

established cutoff scores 14.0% were below the cutoff score in the Approaches to

Learning/Social Emotional area, 15.3% were below in the Early Literacy area, and 15.7% were

below in the Early Numeracy area; 16.1% of children in the normative sample were below the

cutoff score in one or more areas.

Table 4.14

Means, Standard Deviations, Cutoff Scores and Percent Identified by Area of the School

Readiness Skills

Approaches to

Learning Early Literacy Early Numeracy

N Valid 236 236 236

Missing 0 0 0

Mean 13.44 13.97 12.24

Standard Deviation 3.88 5.01 3.84

Cutoff Score 10 9 9

Percent Identified 14.0 15.3 15.7

Reliability and Internal Consistency.

Researchers intended to examine test-retest reliability on both Part 1: CBRS and Part 2:

School Readiness Skills. However, limitations in terms of time and resources made it impossible

to collect test-retest data on both tools and the School Readiness Skills were prioritized given

that it was a newly-developed tool. In order to collect test-retest data, teachers were asked to

complete all three areas of the School Readiness Skills at Time 1 and again at Time 2, two to

three weeks later. A test-retest sample size of 50 was targeted; the actual sample size was 36. A

Pearson Product Correlation was used to analyze test-retest reliability, and results indicate strong

significant agreement in all three areas, as shown in Table 4.15.

Inter-observer reliability was also examined for the Early Literacy and Early Numeracy

areas of the School Readiness Skills. Both teachers and a researcher completed both areas

66


through observations and interactions with individual children during the same two-week period.

Researchers had hoped that the second rater could be someone familiar with the children and

their skills; however, each classroom had only one teacher and no assistants making it necessary

to utilize a researcher as a second observer. For this reason, the Approaches to Learning/Social

Emotional area was not included in the inter-observer study, as skills in this area require

observation over time by a rater with whom children have an ongoing relationship. For this

reason, it was assumed that inter-observer reliability would not be high in this area. The target

number of children for the inter-observer study sample was 50; this goal was met. Inter-observer

reliability was measured with Pearson Production Correlations, using total area scores. As

illustrated in Table 4.15, inter-observer correlations between total scores differed between the

two areas. The Early Literacy area had a correlation of .75, which is considered good, while the

correlation between total scores for the Early Numeracy area is .48, which is considered weak.

Both correlations were significant at the .01 level (2-tailed). Researchers also analyzed the

agreement between two observers, a teacher and a researcher, on classification, and found that

agreement was strong for both the Early Literacy (.90) and Early Numeracy areas (.90); 90% of

the time observers were in agreement as to whether a child was above or below the area cutoff

score.

Table 4.15

Summary of Test-Retest and Inter-Observer Reliability (Pearson Product Correlational Analyses)

Approaches to Learning/

Social Emotional Early Literacy Early Numeracy

Study N r N r N r

Test-retest 36 .85** 36 .75** 36 .70**

Inter-

observer 50 .75** 50 .48**

**Correlation is significant at the .01 level (2-tailed).

Internal consistency also was examined for all three areas of the School Readiness Skills

(Approaches to Learning/Social Emotional, Early Literacy and Early Numeracy) as well as for

the CBRS by examining the relation between item scores using correlational analysis and

67


Cronbach’s coefficient alpha. The standardized alpha for the Approaches to Learning (.81), Early

Literacy (.89), and Early Numeracy (.87) areas indicated strong internal consistency for all three

areas. Internal consistency was also examined for the CBRS, and the standardized alpha was .86,

also indicating strong internal consistency.

Limitations and Future Directions.

There are several limitations to this study that should be considered when implementing

the School Readiness Screening and when considering future research directions. It is important

to note that this was an ambitious study, conducted within an exceptionally short time frame and

with limited resources. As discussed earlier, demographic information was not collected from

197 families, so percentages reported are based on the 41 families from whom demographic

information was collected. Data were also missing for some individual questions on gathered

demographic forms. Because demographic data are missing, it is not clear whether the normative

sample is representative of the general Jamaican population. Collected demographics indicate

that the majority of school readiness screening data was collected in three parishes: Kingston

(51.2%), Saint Andrew (26.8%), and Saint Catherine (22%). As also discussed earlier, 97.5% of

the School Readiness screenings for which demographic information was collected took place in

urban settings, while only 2.6 % took place in rural settings. Cutoff scores were derived based on

means and standard deviations and presumed a normative sample that was stratified in terms of

number of risk factors, and included households with a low, medium and high number of risk

factors. However, since demographic information was limited, and researchers did not have the

ages of participating children, researchers were unable to examine the relationship between mean

scores and age. Without that demographic information it was also not possible to make

conclusions about the degree to which the sample represents the general Jamaican population.

Future studies on the School Readiness Screening should include a representative sample

of the Jamaican population. As also discussed above, future research should also investigate the

validity of established cutoff scores and ensure that the tool is measuring similar skills as other

established school readiness tools as well as identifying children most in need of support.

68

Project Title: INSPIRE Jamaica Child ID______________

University of Oregon IRB # 11162012.020 Research Assistant ID______________

Utility Survey

Please continue on the next page

1. Who is providing this feedback?

☐ Parent (1) ☐ Health Care Provider (2)

☐ Teacher (3) (ID#__________) ☐ Teacher’s Assistant (4) (ID#____________)

☐ Other: (5) ______________________

2. Which tool is feedback focused on? (Tick one response):

☐ SWYC (1) ☐ ASQ (2) ☐ Family Support (3) ☐ Teacher portion School Readiness (4)

2a. If ASQ, indicate interval completed (e.g., 12 months ______________ )

2b. If SWYC, indicate interval completed (e.g., 12-14 months ___________)

3. How long did it take to complete the tool (e.g., 10 minutes _________)?

Please provide your opinion about the following statements:

4. “In general, questions were clear and easy to understand.” (Tick one response)

☐Strongly agree (4) ☐Agree (3) ☐No opinion (2) ☐Disagree (1) ☐Strongly disagree (0)

5. “In general, questions were appropriate for child and/or family’s culture.” (Tick one)


6. “Completing the tool gave me meaningful information and child and/or family’s current

abilities and needs.” (Tick one response)


7. “ I plan (or would like) to use this tool again.” (Tick one response)


69

Thank you for participating in this study!

22 January 2013

If providing feedback on Family Support tool SKIP to question #9

8. “In general, questions were appropriate for child’s age” (Tick one response)


9. If you disagreed or strongly disagreed with any of the statements, please tell us why.

10. How would you change the tool to make it better?

11. We welcome further comments and suggestions. Feel free to write them below:

For Teachers/Professional providers (not intended for parents to complete)

12. How many times have you completed the above tool? _______ _(number of times completed)

13. Do you have a preferred way of completing the tool? For example, do you have an opinion

about where, how, or when it is administered?

☐No (0) ☐Yes (1) (if yes, please tell us more) ________________________

_________________________________________________________________________

_________________________________________________________________________

****************************************************************************************

70


University of Oregon IRB # 11162012.020 Research Assistant ID_____________

Child and Family Information: Parent Section Date Screening (s) Completed: ______________________________________ 1. What tool(s) were completed? (Tick all that are appropriate):

☐ Family Support Screening tool ☐ SWYC ☐ ASQ ☐ Teacher portion School Readiness (enter teacher ID #________________)

2. Child’s Date of Birth: _____________________

2a. For children under 2 years, was child 3 or more weeks premature? ☐ No (0) ☐ Don’t Know (2) ☐ Yes (1) (If yes, weeks premature?___________)

3. Child’s Gender: ☐ Male (1) ☐ Female (2)

4. Parish where child lives: (Tick where appropriate):

☐ Kingston (001) ☐ St Andrew (002) ☐ St Thomas (003) ☐ Portland (004) ☐ St Mary (005) ☐ St Ann (006) ☐ Trelawny (007) ☐ St James (008) ☐ Hanover (009) ☐ Westmoreland (010) ☐ St Elizabeth (011) ☐ Manchester (012) ☐ Clarendon (013) ☐ St Catherine (014) ☐ Don’t Know (99)

5. Setting where screening took place? (Tick where appropriate):

☐ Antenatal (1) ☐ Early Childhood Educational Institution (6) ☐ Clinic (2) ☐ Primary School (7) ☐ PATH Home Visit (3) ☐ Nursery (8) ☐ WCC (4) ☐ Other (9) (please specify__________)

☐ Pediatrician (5)

5a. Was the setting public or private? (Tick where appropriate): ☐ Public (government) (1) ☐ Private (2) ☐ Don’t Know (3) 5b. Setting geographic location (Tick where appropriate):

☐ Rural (1) ☐ Urban (2) ☐ Inner City Urban (3) ☐ Don’t Know (4)

6. Who completed tools (answered questions)? (Tick where appropriate): ☐ Mother (1) ☐ Aunt (6) ☐ Brother (11) ☐ Father (2) ☐ Uncle (7) ☐ Sister (12) ☐ Grandmother (3) ☐ Stepmother (8) ☐ Other-family (13) (specify_______________) ☐ Grandfather (4) ☐ Stepfather (9) ☐ Non-family (14) (specify________________) ☐ Teacher (5) ☐ Teaching Assistant (10)

71


University of Oregon IRB # 11162012.020 Research Assistant ID_____________

7. Who supported the completion of the questionnaires? (Tick where appropriate):

☐ No one (1) ☐ Research Assistant (3) ☐ Health Care Provider (5) ☐ Teacher (2) ☐ Teaching Assistant (4) ☐ Other (6) (specify_____________)

8. Which of the following resources does the family have in their household? (Tick all that apply):

☐ Television ☐ Cars or other vehicles ☐ Fans ☐ Water heater (solar) ☐ Cable/Satellite connection ☐ Telephone ☐ Gas Stove ☐ Generator ☐ Refrigerator ☐ VCR/DVD player ☐ Electric Stove ☐ Video equipment ☐ Freezer ☐ Computer ☐ Air Conditioners ☐ Game boy/Play station

☐ Living Room Set ☐ Internet connection ☐ CD/DVD Burner ☐Other electrical equipment ☐ Stereo Equipment ☐ Radio ☐ Water tank (toaster/microwave/blender) ☐ Washing Machine ☐ Sewing Machine ☐ Water heater (electric)

Enter total number of ticked items______

Section II. Additional Child Information

1. Date Screening (s) Completed: __________________________

2. Diagnosis: Does your child have a known delay or disability, or a medical condition with a high probability of

developmental delay (e.g., Down syndrome)?

☐ No (0) ☐ Don’t Know (2)

☐ Yes (1) If yes, what is disability/diagnosis?_______________________________________

_____________________________________________________________________________

72


Appendix C

Technical Report References

Abidin, R. R. (1983). Parenting Stress Index (PSI), 4th

Edition. Charlottesville, VA: Pediatric

Psychology Press.

American Academy of Pediatrics. (2006). Identifying infants and young children with developmental

disorders in the medical home: An algorithm for developmental surveillance and screening.

Pediatrics, 118(1), 405-420.

Ball, J., & Janyst, P. (2008). Enacting research ethics in partnerships with indigenous communities in

Canada: "Do it in a good way." Journal of Empirical Research on Human Research Ethics,

3(2), 33-52

Bian, X., Yao, G., Squires, J. Wei, M., Chen, C., & Fang, B. (2010). Studies of the norm and

psychometric properties of Ages and Stages Questionnaires in Shanghai children. Zhonghua Er

Ke Za Shi. Chinese Journal of Pediatrics, 48(7), 492–496.

Bronson, M., Goodson, B., Layzer, J., & Love, J. (1990). Child Behavior Rating Scale. Cambridge,

MA: ABT Associates.

Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 279-334.

Dionne, C., Squires, J., & Leclerc, D. (2004). Psychometric properties of a developmental screening

test. Using the Ages and Stages Questionnaires (ASQ) in Quebec and the U.S. Journal of

Intellectual Disability Research, 48(4-5), 408.

Heo, K., Squires, J., & Yovanoff, P. (2008). Cross-cultural adaptation of a preschool screening

instrument: Comparison of Korean and U.S. populations. Journal of Intellectual Disability

Research, 52(3), 195-206.

Janson, H., & Squires, J. (2004). Parent-completed developmental screening in a Norwegian

population sample: A comparison with U.S. normative data. Acta Paediatrica, 93(11), 1525-

1529.

Kapci, E. G., Kucuker, S., & Uslu, R. I. (2010). How applicable are Ages and Stages Questionnaires

for use with Turkish children? Topics in Early Childhood Special Education, 30, 176-188.

Musquash, C., & Bova, D. (2007). Cross cultural assessment and measurement issues. Journal of

Developmental Disabilities, 13(1), 53-66.

Radloff, L. S. (1977). The CES-D scale: a self-report depression scale for research in the general

population. Applied Psychological Measurement, 1:385-401.

73

Sektnan, M., McClelland, M. M., Acock, A., & Morrison, F. J. (2010). Relations between early family

risk, children’s behavioral regulations, and academic achievement. Early Childhood Research

Quarterly, 25(4), 464-479.

Squires, J., Bricker, D. & Twombly, E. (2002). Ages and stages questionnaires: Social-Emotional:

A parent-completed child-monitoring system. Baltimore: Paul Brookes.

Squires, J., Bricker, D., Twombly, E., & Potter, L. (2009). Ages and stages questionnaires user’s guide

3rd

edition: A parent-completed child-monitoring system. Baltimore, MD: Paul Brookes

Publishing.

74

Documents

INSPIRE Jamaica - UNICEF