ORIGINAL PAPER

Examination of the Korean Modified Checklist of Autismin Toddlers: Item Response Theory

HyeKyeung Seung1 · Juye Ji2 · Soo-Jin Kim3 · Inkyung Sung4 · Young-Ah Youn4 ·Gyunghun Hong3 · Hyeonjin Lee5 · Young Hwan Lee6 · Hyunsuk Lee7 ·Hyun Kyung Youm8

© Springer Science+Business Media New York 2015

Abstract The study examined the clinical utility and

psychometric properties of the Korean Modified Checklist

of Autism in Toddlers (K-M-CHAT)-2. A sample of 2300

parents of 16- to 36-month-old children was recruited

across South Korea. A phone interview was utilized to

follow up with participants who initially screened positive

for autism spectrum disorder (ASD). Item response theory

was applied to assess the psychometric properties of the

K-M-CHAT-2. Parents’ responses were substantially

changed after the follow-up, and the final screen-positive

rate was 2.3 %. Results indicated that the psychometric

properties of items 1, 3, 11, 18 and 22 were not as strong as

the other items. The K-M-CHAT-2 is a useful ASD

screening test when implemented with a follow-up.

Keywords ASD screening · Korean M-CHAT · Item

response theory · Population screening

Introduction

Autism spectrum disorder (ASD; APA 2013) is one of the

most prevalent developmental disorders (CDC 2014;

Fombonne 2009; Kogan et al. 2008; Rice et al. 2010; Wong

and Hui 2008; Zaroff and Uhm 2012). Children are

typically diagnosed with ASD on average between 3 and

6 years of age (Filipeck et al. 2000). However, there has

been strong interest in earlier detection of ASD because

earlier intervention could promote in better outcomes in

subsequent development (Anderson et al. 2014; Robins and

Dumont-Mathieu 2006). There have been ongoing scien-

tific efforts to develop and implement a screening

instrument for early detection of ASD (Daniels and Man-

dell 2013; Dietz et al. 2006; Eaves et al. 2006; Stone et al.

2000). Fortunately, it has been suggested that earlier

identification is now possible by conducting ASD-specific

screening in the general population (Chlebowski et al.

2013; Robins et al. 2001; Yama et al. 2012).

One of the most widely studied and utilized ASD

screening tests for the general population is the Modified

Checklist for Autism in Toddlers (M-CHAT; Robins et al.

2001). TheM-CHAT is an expansion of the 9-itemChecklist

for ASD in Toddlers (CHAT; Baron-Cohen et al. 1992). It

consists of 23 binary items asking about the presence or

absence of common ASD symptoms, including social and

communication deficits as well as repetitive behaviors. The

M-CHAT is relatively easy to administer and score and is

now available in various languages at the official M-CHAT

website (D. Robins), http://www.mchatscreen.com/Offi

cial_M-CHAT_Website.html. A substantial amount of

& HyeKyeung Seung

hseung@fullerton.edu

1 Department of Human Communication Studies, California

State University, 2600 E Nutwood Ave., Suite 420-1,

Fullerton, CA 92831, USA

2 Department of Social Work, California State University,

Fullerton, CA, USA

3 Department of Communication Disorders, Korea Nazarene

University, Cheonan, Republic of Korea

4 Department of Pediatrics, The Catholic University of Korea,

Seoul, Republic of Korea

5 Department of Early Childhood Education, Yeungnam

University, Gyeongsan, Republic of Korea

6 Department of Pediatrics, Yeungnam University, Gyeongsan,

Republic of Korea

7 The Children’s World Clinic, Seoul, Republic of Korea

8 Department of Education, Ewha Womans University,

Seoul, Republic of Korea

123

J Autism Dev Disord

DOI 10.1007/s10803-015-2439-0

research has investigated and supported the clinical utility

and psychometric properties of the M-CHAT. The focus of

these studies was to evaluate the sensitivity, specificity, and

positive predictive value of the M-CHAT by conducting an

initial screening, follow-up interviews on the failed items,

and ASD evaluations for those children who screened posi-

tive after the follow-up interview (Chlebowski et al. 2013;

Kleinman et al. 2008; Pandey et al. 2008; Robins et al. 2001).

These studies primarily assessed the accuracy of the

M-CHAT in detecting children at risk of ASD who were

subsequently diagnosed.

Research on ASD screening has been conducted pri-

marily in English-speaking countries such as the U.S.,

Canada, and England (Chlebowski et al. 2013; Daniels and

Mandell 2013; Eaves et al. 2006; Snow and Lecavalier

2008; Wiggins et al. 2014). Recently, the number of ASD

screening studies from non-English-speaking countries has

increased, including China (Wong and Hui 2008), Japan

(Honda et al. 2005; Kamio et al. 2013), and Korea (Hong

et al. 1999; Kim et al. 2011; Kim et al. 2013). This increase

may be due to the availability of English screening tests as

well as the translation of such tests into other languages.

There have been only a few ASD screening studies

conducted with Korean children. Hong et al. (1999)

screened 295 children using a 23-item questionnaire, which

included questions regarding interpersonal relationships,

communication, and repetitive behaviors among other

topics. Subsequently, 137 of the children were further

screened using the Korean version of the ASD Behavior

Checklist and the Denver Developmental Screening Test.

Following this administration, 34 of the children were re-

ferred for an evaluation by a team of pediatric psychiatrists.

It was reported that the prevalence of ASD in the region

was 0.092 %. Kim et al. (2002) examined the validity of

the Korean CHAT (K-CHAT) in 16- to 20-month-old

children. They reported that parents’ responses on the

K-CHAT differentiated typically developing children (TD;

n = 68) and children with an intellectual disability (ID;

n = 49) from those with ASD spectrum disorder (ASD;

n = 49). Kim et al. (2002) evaluated the utility of the

K-CHAT in a relatively small sample with a narrow age

range of children who were previously diagnosed with ID

or ASD.

The M-CHAT was translated into Korean by Kim

(2009). However, its implementation in Korean children

has not been systematically examined. A slight modifica-

tion of the Korean M-CHAT was completed by the first

author and her associates and was referred to as the K-M-

CHAT Version 2 (referred to as K-M-CHAT-2 henceforth)

on the official M-CHAT website. Kim et al. (2013) con-

ducted pilot testing of the K-M-CHAT-2 in Cheonan,

South Korea on a small sample of 203 children aged 16–

36 months. The high screen-positive rate at the time of

initial screening was decreased substantially after a follow-

up phone interview. Therefore, it was important to examine

the clinical utility of the K-M-CHAT-2 in Korean children

with a relatively large sample size in order to provide

scientific evidence supporting the utility of this screening

instrument.

The most recently published large-scale study (55,266

children ages 7–12 years old) investigating the prevalence

of ASD in South Korea, reported the prevalence of ASD in

South Korea to be an estimated 2.64 % (Kim et al. 2011).

This finding clearly shows that ASD is a very important

public health concern and underscores the necessity of

rigorous screening. Unfortunately, ASD screening studies

of young children in the general Korean population are still

lacking. Research on the development, implementation,

and evaluation of ASD screening instruments is warranted

to establish standard clinical practice of early screening and

the provision of subsequent intervention for children with

ASD in South Korea.

Recently, more statistically appropriate and advanced

models of psychometric analyses have become available (e.

g., confirmatory factor analysis and item response theory).

In particular, the value of item response theory in clinical

assessment has been addressed (Thomas 2011), but its

contribution to ASD screening literature has been less ex-

tensive. Latent variable models based on item response

theory provide comprehensive assessment of a measure-

ment instrument as a whole and each item individually.

Item response theory has been identified as an especially

strong tool for assessing an instrument constructed with

binary items that determine symptoms’ presence or ab-

sence, such as the M-CHAT. To the best of our knowledge,

no existing research has examined the psychometric

properties of the M-CHAT using item response theory.

Purpose of the Current Study

The current study expands on research by Kim et al. (2013)

by drawing on a substantially increased sample size and

data collection over a broader region. The present study

was intended to evaluate the screen-positive rates in

Korean children using the K-M-CHAT-2 in South Korea.

The study aimed to evaluate the clinical utility and the

psychometric properties of the K-M-CHAT-2 as a screen-

ing test in Korean children aged between 16 and

36 months. The focus of the study was to examine the

percentage of children who screened positive on the K-M-

CHAT-2 at the initial screening and after a follow-up in-

terview. The research questions of the study were: (a) to

examine the rate of children who screened positive on the

K-M-CHAT-2 at the initial screening using the defined

criteria (i.e., fail on two critical items or fail on three total

items), (b) to examine the rate of children who continued to

J Autism Dev Disord

123

screen positive after a phone follow-up using the same

criteria, and (c) to investigate the psychometric properties

of 23 items of the K-M-CHAT-2 using item response

theory.

Method

Participants

A sample of 2300 parents with children between the ages

of 16 and 36 months old was recruited across South Korea.

Thus, the age range of the participants was expanded be-

yond the 16- to 30-month range that Robins et al. (2001)

implemented. Parents were recruited from daycare centers,

public health centers, hospitals, and a private pediatric

clinic. The study was announced through daycare center

directors’ meetings in Seoul and Ulsan. The director of the

Preschool Inclusion Advocacy Group in Seoul also posted

a recruitment notice on their website and assisted with in-

person participant recruitment. The first author also con-

tacted individual preschool directors and solicited their

assistance in recruiting parents at their centers. Due to the

Individual Information Protection Law in Korea, it was not

feasible to obtain birth records of all children born in the

country or a mass mailing address list. Therefore, recruit-

ment relied heavily on personal connections with directors

of daycare centers, public health centers, and hospital who

were blind to the purpose of the study.

Demographic information is summarized in Table 1. It

includes children in four age groups, their gender, the five

regions from which data were collected, and mothers’ and

fathers’ education levels. In terms of the age groupings

used for the current study, 21.6 % of the sample was aged

16–20 months, 22.3 % aged 21–24 months, 34.0 % aged

25–30 months, and 22.1 % aged 31–36 months. Ap-

proximately half of the children were boys. Data were

collected across ten provinces, which were clustered into

five regions (Northeast, Central, Southeast, Northeast, and

Southwest region) to keep the sample size within each

group relatively similar. Of the data, 35.7 % were collected

in Seoul (Regional Group 1), which is the mostly densely

populated region in South Korea. Proportional to the gen-

eral population, 12.4 % of the sample were from Regional

Group 2, 11.9 % were from Regional Group 3, 15.7 % were

from Regional Group 4, and 2.1 % were from Regional

Group 5 (East and South-west regions and a southern is-

land). Parental education groupings included a high school

diploma or lower, a 2- to 4-year college degree, or a

graduate degree including a master’s or doctorate degree. It

should be noted that parental education information was

missing for approximately 30 % of mothers and 34 % of

fathers in the sample. Questions asking parental education

were added after data collection had begun and the first 200

copies of the questionnaire (approximately, 8.7 % of 2300

total) were distributed. In addition, approximately 20 % of

mothers’ educational information and 25 % of fathers’

educational information was missing due to participants’

preferences. Parents who did not report their educational

level are likely to be in the high school or lower group,

owing in large part to the paramount cultural value placed

on achieving formal education in South Korea. The Korean

Children Panel 2009 data based on a randomly selected

representative sample of 1904 participants (Korea Institute

of Child Care and Education; KICCE 2010) reported that

30.4 % of mothers and 27.8 % of fathers achieved an

education level of high school graduate or lower, figures

that were higher than the rates in our sample. The per-

centage of participants who attained a college degree in our

sample is similar to the KICCE (2010) data. In the KICCE

Table 1 Demographic information of children and parents

(N = 2048)

Variables % (n)

Children

Age

16–20 months 21.6 (441)

21–24 months 22.3 (457)

25–30 months 34.0 (697)

31–36 months 22.1(453)

Gendera

Male 50.6 (1036)

Female 48.3 (990)

Regionb

Group 1 (Seoul) 35.7 (731)

Group 2 (Kyunggi-Do) 12.4 (254)

Group 3 (Chungchung-Do) 11.9 (244)

Group 4 (Kyungsang-Do) 15.7 (322)

Group 5 (otherc) 2.1 (43)

Parents’ education

Motherd

High school or lower 19.7 (283)

College degree 69.5 (999)

Graduate degree 10.8 (155)

Fathere

High school or lower 16.0 (217)

College degree 67.1 (911)

Graduate degree 16.9 (229)

a 1.1 % missing informationb 22.2 % missing informationc Included Gangwon-Do, Junla-Do, and Jeju islandd 29.8 % missing informatione 33.7 % missing information

J Autism Dev Disord

123

study, 64 % of mothers and 61.3 % of fathers were college

graduates, and in the present study, 69.5 % of mothers and

67.1 % of fathers had a college-level education. The per-

centage of graduate degrees in our sample (10.8 % of

mothers and 16.9 % of fathers) was higher than the rates

found in the KICCE data (5.7 % of mothers and 10.7 % of

fathers). Considering that the parents who did not report

their educational level are more likely to be in a lower

educational group, the parental education level outlined in

the table is a fair reflection of the current South Korean

population.

Instrument: The Korean Modified Checklistfor Autism in Toddlers (K-M-CHAT-2)

The K-M-CHAT-2 is based on the M-CHAT (Robins et al.

2001) and a slight modification of the initially translated

Korean version by Kim (2009). Prior to initiating data col-

lection in Kim et al. (2013), the translation of the K-M-

CHAT-2 was reviewed by the first author and community

collaborators (a developmental pediatrician and staff mem-

bers of the Korean-American Special Education Center) in

Southern California. The review yielded a slightly modified

translation of eight items one word change per item (pri-

marily verbs, pronouns, and one noun). Subsequently, the

revised translation was tested with ten Korean mothers in

Southern California by asking them which of the two ver-

sions sounded more natural and was easier to understand.

Based on their responses, the K-M-CHAT-2 as well as an

accordingly modified follow-up interview were used in the

current study (see “Appendix” for the K-M-CHAT-2).

Data Collection Procedures

The study protocol was approved by the California State

University, Fullerton Institutional Review Board (IRB)

committee for the first author to conduct the study in South

Korea. The IRB approved Korean translation of the in-

formed consent form was used for the study. Each

participant received a two-page questionnaire. The first

page contained an informed consent form as well as items

regarding demographic information (e.g., age of the child,

date of response, mother and father’s education, region of

residency, and phone number and email address for follow-

up contact and sharing of screening results). The second

page included the K-M-CHAT-2 items. The K-M-CHAT-2

was distributed to parents who agreed to participate

through teachers at daycare centers. At hospitals, nurses

administered the K-M-CHAT-2 to parents while they were

waiting for their pediatrician’s visit. Teachers and admin-

istrative staff at hospitals and daycare centers collected the

completed responses, including the signed informed con-

sent form.

Data were collected in Seoul and the five provinces of

Kyunggi-Do (Northwest region), Chungchung-Do (Central

region), Kyungsang-Do (Southeast region), Ganwon-Do

(Northeast region), and Junla-Do (Southwest region) across

South Korea. The collection of the completed K-M-CHAT-

2 forms was carried out primarily through postal mail

service (or through email in only a few cases) to the first

author. Additionally, she picked up some of the completed

responses in person when notified by the daycare centers in

Seoul.

Scoring

Collected K-M-CHAT-2 forms were scored by a trained

research assistant who had completed a graduate degree in

Communicative Disorders in South Korea. She was blind

to the purpose of the study and utilized an automated

scoring protocol with Microsoft Excel as provided on the

official M-CHAT website. After scoring was completed

using the program, the research assistant marked any failed

items on the K-M-CHAT-2 response form. She then

identified the participants who failed the screening using

two criteria outlined by Robins et al. (2001): either failing

two or more of six critical items, or failing three or more

out of 23 total items.

Follow-up Interview

In an attempt to decrease the false-positive rate, a follow-

up interview was conducted with parents of children whose

initial screen results were positive for ASD (as suggested

by Robins and Dumont-Mathieu 2006). There was a vari-

able time delay between data collection and review of the

scoring, which ranged from a few weeks to 3 months be-

tween the collection of the K-M-CHAT-2 and the follow-

up interview. The unforeseen delay was due to obtaining

collected data from distant sites by mail. When individual

scoring was completed, the first author reviewed the failed

items. The follow-up interviews on the failed items were

conducted over the phone by the first author.

Sharing the Screening Results

Children did not receive evaluations or diagnoses as part of

this study. However, screening results were briefly reported

to the parents using a template created for the study. The

following two options were on the template: (a) “The

screening results indicate that your child may be at risk for

ASD. We suggest that you consult a child development

expert and rule out the risk,” or (b) “The screening results

indicate that your child is not at risk of ASD now. How-

ever, because some children can show concerning

behaviors at 3 or 4 years of age, if you become concerned

J Autism Dev Disord

123

later, please consult a child development expert at that

time.” Those who were followed-up by phone were pro-

vided with the screening results immediately after the

follow-up, or a brief report was sent as an attachment to

those parents who provided their email address.

Data Entry and Verification

Completed individual scoring files were converted into one

data file using a Microsoft Excel merge program developed

for this research. The first author verified that each par-

ticipant’s responses were correctly incorporated into this

master data file. When scoring errors were detected, the

error was corrected; there were 77 errors out of 2300 re-

sponses, yielding an error rate of 3.3 %. Data for 252

participants were excluded during the data verification

process. The data were excluded if: (1) the child was out of

the age range (either younger than 16 months or older than

36 months, n = 232), (2) respondents were not native

speakers of Korean (n = 4), or (3) there were no responses

on two or more items (n = 16). Among these 16 excluded

participants who did not respond to two or more items, the

most frequently missed items were items 1, 6, 11, 17, 18,

19, 22, and 23. The net result was a total sample of 2048

participants.

Results

A Priori Analysis

A priori analyses and descriptive statistics were analyzed

using IBM Statistical Package for Social Science Version

20. A series of one-way analysis of variance was conducted

to examine any differences in the K-M-CHAT-2 total

scores after the follow-up interview between and among

age, gender, and regional groups. There were no regional

[F (4, 1589) = 2.29, p = 0.06], gender [F (1, 2024) = 3.48,

p = 0.06], or age [F (3, 2044) = 1.43, p = 0.23] group

difference in the total score. Mean and standard deviations

(SD) of total scores by regional, gender, and age groups are

summarized in Table 2.

Descriptive Statistics

Screen-Positive Rate

Screening outcomes are summarized in Fig. 1. A total of

541 out of 2048 participants (26.4 %) were classified as

screen-positives on initial screening. Children who either

failed on two or more out of the six critical items, or three

or more of the 23 total items were identified as screen

positive. Despite the use of both total score and critical

item score criteria, it was found that all children who

screened positive based on the critical item score were also

screened positive on the total item score criteria.

Follow-up was not required for 25 participants who had

a initial total score of seven or higher (following the rec-

ommendation by Chlebowski et al. 2013). Therefore,

parents of those 516 who failed on the initial screening

were potentially eligible for follow-up phone interviews.

Of these, 312 participated in a phone follow-up, yielding a

60.4 % retention rate. This was similar to the response rate

obtained on the M-CHAT Japanese version reported by

Kamio et al. (2013). Follow-up interviews were not able to

be completed for 204 participants (39.5 %) for various

reasons, including the parents’ refusal to participate, inac-

curate contact information, or an inability to reach the

parents after multiple attempts.

Screening procedures and outcomes were summarized

by younger (2-year-old) and older (3-year-old) age groups

(see Table 3). The two groups were similar in the per-

centage of no follow-up, phone-follow-up, unavailable for

follow-up, screen positive and no follow-up required, and

screen positive at initial screening. However, the screen

positive rate after the follow-up in the younger age group

was lower than the rate in the 3-year-old group (3.6 vs.

5.1 % respectively). The final estimated screen positive

rates in the 2-year-old and 3-year-old groups were 2.1 and

2.5 % respectively, which was proportional to the sample

size in each group.

After the follow-up phone interview with a clinician,

participants’ responses on many items changed sig-

nificantly; percentages of changed response for each item

ranged between 1.0 and 65.7 % (refer to Table 4). Among

312 participants who received the follow-up interview,

65.7 % changed their initial response on item 11 and

Table 2 Total item scores by regional, gender and age groups

n Total score M (SD)

Region group

Group 1 (Seoul) 731 1.88 (1.45)

Group 2 (Kyunggi-Do) 254 2.11 (2.01)

Group 3 (Chungchung-Do) 244 1.99 (1.31)

Group 4 (Kyungsang-Do) 322 2.06 (1.69)

Group 5 (other) 43 2.30 (2.19)

Gender

Male 1036 1.66 (1.67)

Female 90 1.53 (1.35)

Age group

16–20 months 441 1.64 (1.68)

21–24 months 457 1.53 (1.50)

25–30 months 697 1.55 (1.36)

31–36 months 453 1.60 (1.52)

J Autism Dev Disord

123

45.5 % changed their initial response on item 18. The

percentages of changed responses for 6 items (7, 8, 13, 19,

22, and 23) were between 10 and 20 %. The percentages of

changed responses for the remaining 15 items were below

10 % (refer to Table 4). As a result of these changes, only

14 out of 312 children remained as screen-positives; thus,

the percentage of screen-positives among the follow-up

group was estimated as 4.5 %. When the rate of screen-

positives after the phone follow-up was extrapolated to

those who did not complete the phone follow-up (n = 204),

an additional 9 participants (4.5 % of 204 children) were

estimated to be potentially screen-positive. As a result, a

total of 48 (including 14 screen-positive after the follow-

up, 9 estimated screen positive, and 25 whose total item

score was 7 or higher at the initial screening) was identified

as screen-positive. Thus, the screen positive rate for all

2048 participants was 2.3 %.

A summary of the percentage of children who failed on

each item on the initial screening (Time 1) and after the

phone-follow-up (Time 2) is presented in Table 5. This

table allows further examination of the response distribu-

tion for each item. It should be noted that at both Time 1

and 2, the fail rates on items 11 (66.5 and 56.4 %, re-

spectively) and 18 (34.1 and 27.1 %, respectively) were

remarkably higher than those for other items. The fail rates

on items 11 and 18 after the phone follow-up were de-

creased by 10 and 7 percentage points respectively.

Item Response Theory

To examine the psychometric properties of the K-M-

CHAT-2, item factor analysis within an item-response

theory (IRT) framework was selected as the best analytic

method. IRT is based on the assumption that the

Fig. 1 A summary of screening

procedure and outcomes. Note:Shaded boxes refer to initial

screen positive (n = 541).

Screen positive rate at initial

screening = 26.4 %

[(312 + 204 + 25)/2048].

Screen positive rate found after

the follow-up interview with

312 participants = 4.5 %

(14/312). Estimated number of

screen positive children among

204 participants who did not

participated in the follow-up

using the screen positive rate of

4.5 % = 9 children

(204 9 0.045). Final estimated

screen positive rate after the

follow-up = 2.3 %

[(25 + 14 + 9)/2048]. 1Total

score ≥7

Table 3 Screening procedure and outcomes by age group

16–24 months

old (n = 898)

25–36 months

old (n = 1150)

No follow-up 666 (74 %) 841 (73 %)

Phone-follow up 136 (15 %) 176 (15 %)

Screen negative 131 (96 %) 167 (95 %)

Screen positive 5 (4 %) 9 (5 %)

Unavailable for follow-up 85 (10 %) 119 (11 %)

Screen positive, no follow-up required 11 (1 %) 14 (1 %)

Screen positive rate at initial screening (136 + 85 + 11)/898 = 25.8 % (176 + 119 + 14)/1150 = 26.8 %

Screen positive rate found among the follow-up participants 5/136 = 3.6 % 9/176 = 5.1 %

Estimated number of screen positive children among the

unavailable participants for the follow-up

85 * 0.036 = 3 children 119 * 0.051 = 6 children

Final estimated screen positive rate (5 + 11 + 3)/898 = 2.1 % (9 + 14 + 6)/1150 = 2.5 %

J Autism Dev Disord

123

probability of a respondent’s passing or failing an item is a

joint function of the person parameter (a person’s standing

on a latent trait) and the item parameters (characteristics of

the item itself). IRT models gain a number of potential

advantages over classical test theory (CTT) by providing

more comprehensive information on items as well as scale

(see Hays et al. 2000). Similar to CTT, item factor analysis

within an IRT framework examines the factor loadings and

establishes that a single underlying latent trait (θ) (e.g.,

ASD) is present. The most commonly used criteria to ac-

cept the factor loading of items is a result higher than 0.4

(Cabrera-Nguyen 2010). Different from CTT, IRT esti-

mates two item parameters: (a) item discrimination and

(b) item difficulty. An item discrimination parameter indi-

cates how good an item is at differentiating individuals

with high trait levels (high level of ASD symptoms) from

individuals who have low trait levels (low level of ASD

symptoms). Thus, high discrimination is desirable as a

screening test. An item difficulty parameter is the level of a

trait where a respondent has a 50 % chance of obtaining the

higher response (failing = 1) (Hays et al. 2000). From a

clinical standpoint, “difficulty” can be translated into

“severity” of clinical symptoms (Thomas 2011). More

specifically, in this study item difficulty parameters indi-

cate the level of severity of ASD symptoms required to fail

an item statement. In Figs. 2 and 3, the dotted line indicates

the item difficulty parameter presenting the level of the

trait where a respondent has a 50 % chance of failing each

item. An item with low difficulty is more likely to be failed

even when a respondent has very low levels of autistic

symptoms. An item with high difficulty is generally less

likely to be failed unless a respondent has very severe

levels of autistic symptoms. Therefore, in general, items

with overly high or low difficulty may not be desirable in a

screening instrument for the general population. The pre-

sent study also examined the item characteristics curve(ICC). ICC illustrates the relationship between a respon-

dent’s trait level (θ) and the probability of obtaining a

higher response (in this study, failing an item). The shape

of ICC is determined by item discrimination and difficulty

parameters. Items with higher item discrimination yield a

steeper S-shaped curve. The ICCs of items with lower item

difficulty tend to be located farther to the left on the trait

scale, and those of items with higher item difficulty tend to

be located farther to the right. The examination of ICCs can

provide more concrete information about item character-

istics. Lastly, the study investigated test informationfunction, which presents the sum of all item information

functions of the K-M-CHAT-2 items. This test information

parameter can be thought of as the reliability of a mea-

surement. A test information result of 4 can be interpreted

as a Cronbach alpha of 0.70, and test information of 9 can

be interpreted as a Cronbach alpha of 0.90 (Thissen 2000).

The IRT analyses were conducted using the Weighted

Least Square Means and Variances estimator in Mplus 7.11

(Muthen and Muthen 2013). A two-parameter logistic

model was selected because it is most applicable for a

unidimensional measurement constructed with binary re-

sponse items (Hays et al. 2000) and is most relevant to the

situation in which clinical symptoms are queried dichoto-

mously—symptoms are either present or absent (Thomas

2011). Item factor loadings, item discrimination, item dif-

ficulty parameters, item characteristics curves, and test

information curves were examined to assess the psycho-

metric properties of the K-M-CHAT-2. Fit indices

including the root mean square error of approximation

(RMSEA) and comparative fit index (CFI) were used to

evaluate the fit of the model to the data. An RMSEA of

0.05 or less indicates good model fit (MacCallum et al.

1996). The 90 % confidence interval and probability that

the RMSEA is less than 0.05 (PCLOSE) provide additional

information on how closely a model fits the data. PCLOSE

should be nonsignificant at the 0.05 level to indicate a close

fit to the data. A CFI value above 0.9 suggests an accept-

able fit of the data to the model (Bentler 1990).

Table 4 Percentage of changed response after follow-up interview

(n = 312)

Items Percentage (%) Frequency (n)

Item 1 8.3 26

Item 2 7.1 22

Item 3 2.9 9

Item 4 2.6 8

Item 5 5.4 17

Item 6 9.3 29

Item 7 10.9 34

Item 8 11.5 36

Item 9 6.1 19

Item 10 1.3 4

Item 11 65.7 205

Item 12 2.2 7

Item 13 16.3 51

Item 14 1.3 4

Item 15 6.1 19

Item 16 1.0 3

Item 17 8.0 25

Item 18 45.5 142

Item 19 14.7 46

Item 20 5.8 18

Item 21 9.0 28

Item 22 17.3 54

Item 23 12.2 38

Bold indicates the two highest response change after follow-up

J Autism Dev Disord

123

Table 6 summarizes the results of the item factor ana-

lysis of 23 items of the K-M-CHAT-2. The tested factor

model provided unsatisfactory fit to the data. While

RMSEA showed excellent fit to the data (RMSEA = 0.026,

PCLOSE = 1.0), CFI was lower than 0.90 (CFI = 0.85)

and did not meet the criteria for acceptable fit. Examination

of factor loadings of each item identified that factor load-

ings of items 1, 3, 11, 18, and 22 were less than 0.40

(indicated in bold on Table 6). In the examination of item

discrimination parameters, items 11 and 18 were the two

Table 5 Comparison of % fail by items between time 1 and 2 in Korean children (N = 2048)

Item T11 T22

1. Enjoy being swung 5.7 4.4

2. Take an interest in other children 4.0 2.9

3. Like climbing on things 3.3 2.8

4. Enjoy playing peek-a-boo 1.5 1.1

5. Ever pretend to talk on the phone/take care of a doll 3.7 2.9

6. Ever use index finger to point to ask for something 6.3 4.9

7. Ever use index finger to indicate interest in something 7.2 5.5

8. Can play properly with small toys 6.9 5.1

9. Ever bring objects over to you to show you 3.3 2.3

10. Look in the eye for more than a second or two 1.3 1.1

11. Ever seem oversensitive to noise 66.5 56.4

12. Smile in response to your face or your smile 0.9 0.5

13. Imitate you 6.8 4.3

14. Respond to his/her name when you call 1.1 0.9

15. If you point at a toy across the room, does your child look at it? 2.1 1.2

16. Does your child walk? 0.6 0.4

17. Does your child look at things you are looking at? 5.7 4.5

18. Does your child make unusual finger movements near his/her face? 34.1 27.1

19. Does your child try to attract your attention to his/her own activity? 8.7 6.5

20. Have you ever wondered if your child is deaf? 3.0 2.1

21. Does your child understand what people say? 4.3 2.9

22. Does your child sometimes stare at nothing or wander with no purpose? 10 7.4

23. Does your child look at your face to check your reaction when faced with something unfamiliar? 13.9 12

Item descriptions are the main points of each item, instead of verbatim due to space limitation. Item 2, 7, 9, 13, 14, and 15 are critical 6 items1 Initial paper screening2 Follow-up telephone interview

Fig. 2 Item characteristics curves of item 1, 3, 11, 18, 22 Fig. 3 Item characteristics curves of remaining 18 Items

J Autism Dev Disord

123

items with the lowest discrimination. This is consistent

with results that items 11 and 18 were the items with the

highest failing rate, and failing responses on these two

items significantly changed after the follow-up interview.

Values of item discrimination parameters of items 1, 3, and

22 were also relatively lower than other items. Regarding

the item difficulty parameter, items 11 and 18 showed the

lowest levels of item difficulty, which indicates that these

two items are most likely to be failed even when the target

child has a very low level of autistic symptoms. In contrast,

items 1, 3, and 22 were the items with the highest item

difficulty parameters, which means these items are the least

likely to be failed unless a respondent has a very high level

of autistic symptoms. Figure 2 presents item characteristics

curves of items 1, 3, 11, 18, and 22, while Fig. 3 illustrates

item characteristics curves for the remaining 18 items.

Comparison of the shape of ICC in Figs. 2 and 3 clearly

shows that item characteristics and functions of items 1, 3,

11, 18, and 22 are very different from the other 18 items.

ICCs of items 1, 3, 11, 18, and 22 were neither steep nor S-shaped due to low discrimination, indicating there is no

clear discrimination between autistic and non-autistic in-

dividuals. In addition, ICCs for items 11 and 18 with the

lowest difficulty were located farther left, and ICCs for

items 1, 3, and 22 with the highest difficulty were located

farther right. ICCs of the 18 items showed a very similar

pattern of S-shape and very steep curve. The results of the

analysis indicate the psychometric properties of items 1, 3,

11, 18, and 22 of K-M-CHAT-2 may not be sound. Lastly,

the total value of test information of the 23 items was over

14. As discussed previously, a test information value

greater than 9 can be interpreted to the Cronbach alpha

coefficient of 0.90. Thus, this result suggests that the items

in K-MCHAT-2 do have good reliability.

Table 6 Summary of item

factory analysis with 23 items

(N = 2048)

Item Factor loadings Item discrimination Item difficulty

Item 1 0.253 0.262 6.721 (highest)

Item 2 0.620 0.790 3.051

Item 3 0.328 0.347 5.811 (third highest)

Item 4 0.562 0.679 4.064

Item 5 0.705 0.993 2.683

Item 6 0.901 2.077 1.833

Item 7 0.856 1.653 1.866

Item 8 0.558 0.673 2.933

Item 9 0.734 1.081 2.708

Item 10 0.779 1.241 2.931

Item 11 −0.325 −0.344 (lowest) 0.495 (second lowest)

Item 12 0.763 1.180 3.344

Item 13 0.708 1.003 2.418

Item 14 0.834 1.513 2.822

Item 15 0.843 1.565 2.689

Item 16 0.648 0.851 4.044

Item 17 0.655 0.868 2.588

Item 18 −0.178 −0.181 (second lowest) −3.428 (lowest)

Item 19 0.595 0.740 2.546

Item 20 0.444 0.495 4.560

Item 21 0.719 1.035 2.629

Item 22 0.231 0.238 6.259 (second highest)

Item 23 0.479 0.546 2.450

Model fit indices χ2/df = 549/230, CFIa = 0.85

RMSEAb = 0.026, 90 % CI of RMSEAc = 0.023–0.029, PCLOSEd = 1.0

a Comparative fit indexb Root mean square error of approximationc 90 % confidence interval of RMSEAd Probability that RMSEA is less than 0.05

J Autism Dev Disord

123

Discussion

Screen Positive Rate Before and After PhoneFollow-up Interview

The results of the current study yielded a screen-positive

rate of 26.4 % on the initial screening. Final estimated

screening-positive rates after the phone follow-up was

2.3 %. Overall, the screen positive rate at initial screening

was much higher than other studies, but the rate after fol-

low-up was similar to the rate found in previous research

utilizing the M-CHAT. In the study by Robins et al. (2001),

the initial screen-positive rate was approximately 10.0 %,

and the rate after the follow-up was 3.0 %. A large

population screening study (N = 18,989) using the

M-CHAT by Chlebowski et al. (2013) yielded 9.1 %

screen-positives on the initial screening, and 1.4 % of

18,989 children (18–24 months old) screened positive after

the follow-up. The screen positive rates at initial screening

and after follow-up in the current study were substantially

higher than the findings of the study by Chlebowski et al.

(2013). Kim et al. (2011) reported a prevalence rate of

2.64 % in Korean children in a study that screened 55,266

children aged between 7 and 12 years old. Their ASD

prevalence estimate is similar to our screen-positive rate

after follow-up.

The results of the current study strongly support the

importance of a two-step screening procedure; the imple-

mentation of a follow-up interview is recommended

because initial screening results yielded relatively high

screen-positive rates, which were decreased substantially

after the phone follow-up. Unfortunately, about 38 % of

those who screened positive on initial screening did not

participate in the phone follow-up interview for various

reasons. An on-site follow-up could have contributed to

higher participation in the follow-up interview. Conducting

an immediate follow-up interview on-site would have

many advantages. Items could be explained in person if

needed, and it would be more personal. Additionally, such

a procedure would decrease the time interval between the

initial screening and a follow-up interview. An average of

3 months passed between the initial screening and follow-

up interviews in the current study for procedural reasons.

Providing immediate results facilitates timely referrals to

community professionals as necessary to address the

child’s developmental needs.

Item Response Theory

The results of the item factor analysis indicated that items

11, 18, and 22 had the lowest item discrimination among

all 23 items, which suggests these three items do not

adequately differentiate autistic children from non-autistic

children. In addition, analyses revealed that items 11 and

18 had the lowest item difficulty, suggesting that the

chances of failing these two items were high even for

children who are not autistic. Further, the results also

indicated that items 11 and 18 were the items with the

highest failing rate, and more than half of the participants

changed their responses on these two items after the fol-

low-up interview. The percentage of changed responses for

item 22 was also relatively higher than most items.

The converging results of item discrimination, item

difficulty and high failing rate on items 11, 18, and 22 in

the current study—as well as evidence emerging from the

review of the previous literature—suggest that these items

may not sufficiently convey their intended meaning. In the

original study by Robins et al. (2001), items 11, 18, and 22

were three out of the five with the highest failing rate

among non-autistic children (19.9, 6.1, and 10.8 % re-

spectively). Similar findings were reported in previous

literature, particularly for the studies that translated the

M-CHAT into a foreign language and implemented it with

a non-English speaking population. For example, the study

by Canal-Bedia et al. (2011) reported the relatively higher

failing rate for items 11, 18, 22, and 23, and explained that

it was due to misunderstandings about the meaning of these

items. Similarly, in the present study, the follow-up inter-

view revealed that items 11 and 18 were often

misunderstood by Korean parents. Several Korean parents

misconstrued the wording of item 11 (“Does your child

ever seem oversensitive to noise?”) to include ideas of

“hears sounds well,” “wakes up from a nap due to small

sounds,” and “likes listening to music.” The follow-up

interview suggested that many parents’ responses to item

18 (“Does your child make unusual finger movements near

his/her face?”) may be subject to more culturally-based

misunderstandings of the item’s meaning. Some parents

responded “yes” to this item when their children point to

their cheek with the index finger, which is a frequently

observed playful behavior in Korean play dyads. Regarding

item 22 (“Does your child sometimes stare at nothing or

wander with no purpose?), some Korean parents shared

that they could not understand the intended meaning of the

question and simply answered it to the best of their ability.

We carefully reexamined the translation of these three

items to see whether there was any significant translational

issue that could explain the misunderstanding. However,

there was only a very minor translational difference from

the K-M-CHAT-2. The review of item 22 indicated that the

frequency of occurrence (“sometimes”) was eliminated on

the Korean translation of the item. There is no clear evi-

dence whether the observed misunderstanding on item 22

was due to this minor translation issue, but it should be

J Autism Dev Disord

123

noted that item 22 was excluded in the publication of a

revised M-CHAT (M-CHAT-R; Robins et al. 2014).

Moreover, literature on the M-CHAT suggests one more

important explanation of the poor performance of items 11,

18, and 22. Kimple et al. (2014) studied a sample of

Spanish speaking patients and reported that 11, 18, and 22

were the most frequently failed items. Specifically, this

study pointed to possible problems with items requiring

reverse coding and thus excluded all reverse-coded items.

Items 11, 18, and 22 comprise three out of the total of four

reverse-coded items. However, Kimple et al. (2014) dis-

puted the notion that reverse coding is the primary issue;

they argued that reverse coding is a common procedure in

social science research and that this may not be enough to

explain the highest failing rate found on items 11, 18, and

22. Taken altogether, evidence from the literature and

findings of the current study suggest that items 11, 18, and

22 should be used with caution when working with Korean

parents and children.

The results of this study also revealed that items 1, 3,

and 22 had the highest item difficulty parameters among all

23 items. The K-M-CHAT-2 is intended to be an ASD

screening instrument for the general population. The utility

of items with high item difficulty in a general screening

tool should be carefully examined. High item difficulty

means that these items are unlikely to be failed unless a

child has a very severe level of ASD symptoms. An in-

teresting finding observed in Robins et al. (2001) is that

failing rates among children diagnosed with ASD were

very low on items 1 and 3 (2.6 and 5.1 % respectively). In

other words, at least 95 % of autistic children passed items

1 and 3. The failing rate on most items of the M-CHAT

ranged between 17.9 and 82 % among autistic children.

This finding by Robins et al. (2001) is consistent with the

high item difficulty of item 1 and 3 found in the current

study and suggests that ASD detectability and thus

screening utility of these two items is very low. However,

the failing rate on item 22 among autistic children in their

study was 61.5 % and thus was similar to other items’

failing rate. Similarly, Ventola et al. (2007) studied the

M-CHAT items that differentiated autistic spectrum dis-

order from developmental delay and developmental

language disorder and also found a much lower failing rate

of items 1 and 3 (1.9 and 6.3 %, respectively) compared to

the other M-CHAT items. Items 1, 3, and 16 are foil items,

intentionally included to allow parents of children with

severe ASD an opportunity to respond affirmatively to

something positive. Thus, the high difficulty parameters

found in the current study are consistent with the intended

purpose of the items.

In sum, issues regarding low item discrimination and

low or high item difficulty may have contributed to the

higher failing rate on certain items and substantially higher

screen-positive rate at initial screening found in this study.

The results of item factor analysis suggest that items 1, 3,

11, 18, and 22 should be used with caution, particularly

when they are implemented with a Korean population. As

discussed earlier, items 1 and 3 are foil items. It is un-

known how parents would respond to the discriminating

items if these foil items were not included. In the imple-

mentation of this measure, practitioners should examine

these items very carefully during a follow-up interview. So

far, little research exists on item-level psychometric prop-

erties of the M-CHAT. Therefore, future research should

further evaluate item-level psychometric properties of the

M-CHAT in English and other languages to provide strong

empirical evidence of the utility of the M-CHAT.

Limitations and Future Directions

The current study was the first attempt to examine the

clinical utility of the Korean version of the M-CHAT in a

relatively large sample across diverse regions in South

Korea. As the first empirical attempt to screen 16- to

36-month-old children across South Korea, this study

contributes to the literature by demonstrating the feasibility

of implementing ASD screening in South Korean children.

In spite of this contribution, there were some procedural

limitations. The current study did not utilize a final diag-

nostic evaluation process when the phone follow-up

screening yielded a screen-positive result. Those parents

were provided with a brief screening result report either by

email or phone. They were counseled to seek further

assessment for potential ASD by having their child

evaluated by community professionals. Therefore, the

current study was unable to obtain sensitivity, specificity

and positive predictive value, which should be evaluated in

future studies. It is important to examine the relationship

between sensitivity, specificity, and positive predictive

value and item characteristics evaluated by item response

theory. In addition, future research should further explore

how elimination of the items identified in the current study

would influence sensitivity, specificity, and positive pre-

dictive value.

Those 25 children who failed on seven or more items did

not receive a phone follow-up on the assumption that it is

highly unlikely the screening results would be changed

after the follow-up interview. If an email address was

provided, a brief report was sent to the parent suggesting

their child be evaluated by a professional. However, they

should have been followed-up for clinical reasons even

though the current study did not have follow-up diagnostic

evaluation plan procedures. In future studies, follow-ups

should be delivered to children who failed more than seven

items, and they should be referred to diagnostic evaluations

as indicated.

J Autism Dev Disord

123

Data collection was implemented by individuals in

various disciplines (teachers, nurses, etc.). Approximately

23 % of the sample was recruited from hospitals where

nurses or clinic staff distributed the K-M-CHAT-2 to par-

ents. Remaining participants were recruited from daycare

centers, preschools, or through personal outreach by stu-

dents in the Communicative Disorders program in South

Korea. This could have created potential confounds of re-

liability of administration. However, the influence from the

teachers was considered to be none to minimal because

their responsibility was simply to distribute the survey to

parents of 16- to 36-month-old children. The majority of

parents completed their responses at home and returned

them to the teachers who distributed the M-CHAT to them.

If the K-M-CHAT-2 were more systematically and con-

sistently administered by trained research assistants or

clinicians in all settings, high screen positive rates at the

initial screening could have been decreased.

The majority of participants in the present study were

parents, with the exception of a few cases where the

grandparents who served as primary caregivers responded.

When the M-CHAT-2 was distributed at daycare centers or

preschools, a disproportionate number of those who re-

sponded could have been caregivers who were concerned

about their child’s development; they also could have been

more educated than the population as a whole. In addition,

several daycare centers have partnered with parents in the

past for other research projects. Therefore, there could have

been potential biases in recruitment.

In addition, 204 participants who were screen positive at

initial screening did not participate in the follow-up inter-

view, and the final screen positive rate was estimated based

on extrapolation of follow-up interviews from 308 par-

ticipants. Thus, findings of the present study might have

been influenced by this attrition. Data were collected across

South Korea, but no regional differences on total scores

were obtained in the current study. The samples were fairly

well-represented, reflecting the population sizes of the

various regions. It should be noted that samples obtained

from the Southwest and Northeast regions (Region Group

5) were somewhat smaller than those from other regions,

which should be improved in future studies. This study is

based on the Modified Checklist for Autism in Toddlers

(M-CHAT), but a revised English M-CHAT-R (Robins

et al. 2014) is now available. Features of the revised

M-CHAT are: (1) exclusion of three items—Item 4 “Enjoy

playing peek-a-boo;” Item 8 “Play properly with small

toys;” and Item 22 “Stare at nothing or wander with no

purpose”—bringing the total number of items down to 20;

(2) more explicit wording in many items; (3) inclusion of

examples for each item; and (4) re-wording of items 3

(item 5 on the original M-CHAT) and 11. Recently, a

Korean translation of the M-CHAT-R was completed and

is now available at the official M-CHAT website. The

clinical utility and psychometric characteristics of the

K-M-CHAT-R should be examined in future studies.

Acknowledgments This study was supported by a Fulbright

Scholar Program 2012–2013 U.S. Scholar Award (Korean American

Educational Commission) and California State University, Fullerton

sabbatical award 2012. We are extremely grateful to the parents who

participated in the study. We thank the Jungrang-gu public health

center, as well as the many daycare centers, preschools, and many

other individuals who assisted in recruiting the participants. An ex-

tended gratitude goes out to Korean colleagues (Moonja Shin, Sunah

Chang, YoonKyung Lee, Sunghee Choi, DaeYoung Won, Mina

Whang, JaeOk Kim) and Eunyoung Park, SuEun Chang, Joanne Min.

Special thanks to Prof. Jiyoung Shin at the Korea University who

provided with an office and scholarly inspiration during the first au-

thor’s sabbatical.

Appendix: K-M-CHAT Version 2

아래 항목에 답해 주시기 바랍니다. 반드시 전 문항에 응답해주십시오. 만약 아래 질문 중 그 행동의 횟수가 빈번하지 않으면

(예: 한 두 번 본 경우) 그 문항은 ‘아니오’ 라고 답해 주십시오

1. 당신이 아이를 안고 그네처럼 흔들어 주거

나 무릎에 앉혀 흔들어 주는 것을 아이가 좋아하나요?

예 아니오

2. 아이가 다른 아이들에게 관심을 가지나요? 예 아니오

3. 아이가 물건이나 가구에 올라가기나 계단

오르기를 좋아합니까?

예 아니오

4. 아이가 까꿍 놀이나 숨바꼭질을 좋아합니

까?

예 아니오

5. 아이가 흉내 놀이를 하나요? (예: 인형 돌보

기, 전화 걸기 놀기 등)예 아니오

6. 아이가 뭔가를 요구할 때 검지 손가락으로

가리킵니까?

예 아니오

7. 아이가 흥미로운 것을 가리키기 위해 검지

손가락을 사용합니까?

예 아니오

8. 아이가 자동차나 블록과 같은 작은 장난감

을 입으로 빨거나 반복적으로 떨어뜨리지

않고 용도에 맞게 갖고 노나요?

예 아니오

9. 아이가 당신에게 보여주기 위해 장난감이나 기타 물건을 가지고 오나요?

예 아니오

10. 아이가 1초에서 2초 이상 당신의 눈을 응시하나요?

예 아니오

11. 아이가 소리에 민감하다고 느끼시나요?(예: 시끄러워 귀를 막는 행동)

예 아니오

12.당신의 얼굴을 보면 미소를 짓거나 당신이웃으면 따라 웃나요?

예 아니오

13. 아이가 당신을 따라 하나요? (예: 찡그리

면 따라서 찡그림)예 아니오

14.아이가 이름을 부르면 아이가 반응을하나

요?

예 아니오

15.당신이 멀리 있는 물건을 손으로 가리키면

아이가 그 쪽을 보나요?

예 아니오

16. 아이가 걸어 다니나요? 예 아니오

17. 아이가 당신이 보고 있는 것을 따라 보나

요?

예 아니오

J Autism Dev Disord

123

18. 아이가 자신의 얼굴 주위에서 손가락으로

특이한 행동을 하나요?

예 아니오

19.아이가 자신의 행동으로 당신의 관심을 끌

려고 하나요?

예 아니오

20.아이의 청각에 문제가 있다고 생각한적이있나요?

예 아니오

21. 아이가 다른 사람의 말을 이해하나요? 예 아니오

22.아이가 허공을 응시하거나 목적 없이돌아다니나요?

예 아니오

23. 아이가 낯선 것을 봤을 때 당신의 반응을보기 위해 당신의 얼굴을 보나요?

예 아니오

Copyright by Diana Robins, Deborah Fein, and Marianne Barton.

Translated into Korean by Hyun Uk Kim, Fairfield University

References

American Psychiatric Association. (2013). Diagnostic and statisticalmanual of mental disorders (5th ed.). Arlington, VA: American

Psychiatric Association.

Anderson, D., Liang, J. W., & Lord, C. (2014). Predicting young adult

outcome among more and less cognitively able individuals with

ASD spectrum disorders. Journal of Child Psychology andPsychiatry, 55, 485–494. doi:10.1111/jcpp.12178

Baron-Cohen, B., Allen, J., & Gillberg, C. (1992). Can ASD be

detected at 18 months? The needle, the haystack, and the CHAT.

The British Journal of Psychiatry, 161, 839–843. doi:10.1192/bjp.161.6.839

Bentler, P. M. (1990). Comparative fit indexes in structural models.

Psychological Bulletin, 107, 238–246.Cabrera-Nguyen, P. (2010). Author guidelines for reporting scale

development and validation results. Journal of the Society forSocial Work and Research, 1(2), 99–103. doi:10.5243/jsswr.

2010.8

Canal-Bedia, R., Garcia-Primo, P., Martin-Cilleron, M. V., Santos-

Borbujo, J., Guisuraga-Fernandez, Z., Herraez-Garcia, L., &

Posada-de la Paz, M. (2011). Modified Checklist for ASD in

Toddlers: Cross-cultural adaptation and validation in Spain.

Journal of ASD and Developmental Disorders, 41, 1342–1351.doi:10.1007/s10803-010-1163-z

Centers for Disease Control and Prevention (CDC) (2014). Prevalence

of ASD spectrum disorder among children aged 8 years-ASD

and developmental disabilities monitoring network, 11 sites,

United States, 2010. Morbidity and Mortality Weekly Report, 63(2), 1–21. Retrieved from http://www.cdc.gov/mmwr/preview/

mmwrhtml/ss6302a1.htm?s_cid=ss6302a1_w

Chlebowski, C., Robins, D., Barton, M. L., & Fein, D. (2013). Large-

scale use of the modified checklist for ASD in low-risk toddlers.

Pediatrics, 131, e1121–e1127. doi:10.1542/peds.2012-1525Daniels, A. M., & Mandell, D. S. (2013). Children’s compliance with

American Academy of Pediatrics’ well-child care visit guideli-

nes and the early detection of ASD. Journal of ASD anddevelopmental Disorders, 43(12), 2844–2854. doi:10.1007/

s10803-013-x

Dietz, C., Swinkels, S., van Daalen, E., van Engeland, H., &

Buitelaar, J. K. (2006). Screening for autistic spectrum disorder

in children aged 14–15 months. II: Population screening with the

Early Screening of Autistic Traits Questionnaire (ESAT). Design

and general findings. Journal of ASD and DevelopmentalDisorders, 36, 713–722. doi:10.1007/s10803-006-0114-1

Eaves, L. C., Wingert, H., & Ho, H. H. (2006). Screening for ASD:

Agreement with diagnosis. ASD, 10(3), 229–242. doi:10.1177/1362361306063288

Filipeck, P. A., Accardo, P. J., Ashwal, S., Baranek, G. T., Cook, E.

H., Dawson, G., et al. (2000). Practice parameter: Screening and

diagnosis of ASD: Report of the quality standards subcommittee

of the American Academy of Neurology and Child Neurology

Society. Neurology, 55, 468–481. doi:10.1212/WNL.55.4.468

Fombonne, E. (2009). Epidemiology of pervasive developmental

disorders. Pediatrics Research, 65, 591–598. doi:10.1203/PDR.0b013e31819e7203

Hays, R., Morales, L. S., & Reise, S. (2000). Item response theory and

health outcomes measurement in the 21st century. Medical Care,38, 1128–1142.

Honda, H., Shimuzu, Y., Imai, M., & Nitto, Y. (2005). Cumulative

incidence of childhood ASD; A total population study of better

accuracy and precision. Developmental Medicine and ChildNeurology, 47, 10–18. Retrieved from http://onlinelibrary.wiley.

com; doi:10.1111/j.1469-8749.2005.tb01034.x/pdf

Hong, K., Lee, S., Jung, B., Kim, B., & An, Y. (1999). Prevalence of

ASD in a city (Gumi): Population study in 3–6 years old

children. Journal of the Korean Association for Persons withASD, 1(1), 1–25.

Kamio, Y., Inada, N., Koyama, T., Inokuchi, E., Tsuchiya, K., &

Kuroda, M. (2013). Effectiveness of using the Modified Checklist

for ASD in Toddlers in two-stage screening of ASD spectrum

disorder at the 18-month health check-up in Japan. Journal of ASDand Developmental Disorders,. doi:10.1007/s10803-013-1864-1

Kim, H. U. (2009). Korean translation: Modified Checklist of ASD in

Toddlers. http://www.mchatscreen.com/Official_M-CHAT_Web

site.html

Kim, A. Y., Kim, H. J., & Lee, S. H. (2002). Development of

K-CHAT. The Korean Journal of Developmental Psychology, 15(3), 17–32.

Kim, Y. S., Leventhal, B. L., Koo, Y., Fombonne, E., Laska, E., Lim,

E., et al. (2011). Prevalence of ASD spectrum disorders in a total

population sample. The American Journal of Psychiatry, 168,904–912. doi:10.1176/appi.ajp.2011.10101532

Kim, S. J., Seung, H. K., & Hong, G. H. (2013). Preliminary

examination of clinical use of the Korean modified checklist of

ASD in toddlers. Communication Sciences and Disorders, 18(2),172–182. doi:10.12963/csd.13017

Kimple, K. S., Bartelt, E. A., Wysocki, K. L., & Stenier, M. J. (2014).

Performance of the Modified Checklist for ASD in Toddlers in

Spanish-speaking patients. Clinical Pediatrics,. doi:10.1177/

0009922814522346

Kleinman, J. M., Robins, D. L., Ventola, P. E., Pandey, J., Boorstein,

H. C., Esser, E. L., et al. (2008). The Modified Checklist for

ASD in Toddlers: A follow-up study investigating the early

detection of ASD spectrum disorders. Journal of ASD andDevelopmental Disorders, 38, 827–839. doi:10.1007/s10803-

007-0450-9

Kogan, M. D., Strictland, B. B., Blumberg, S. J., Singh, G. K., Perrin,

J. M., & van Dyck, P. C. (2008). A national profile of the health

care experiences and family impact of ASD spectrum disorder

among children in the United States, 2005–2006. Pediatrics, 122,e1149–e1158. doi:10.1542/peds.2008-1057

Korea Institute of Child Care and Education (2010). Korean children

panel 2009 data. Korea Institute of Child Care and Education

http://www.kicce.re.kr

MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996).

Power analysis and determination of sample size for covariance

structure modeling. Psychological Methods, 1, 130–149.Muthen, B., & Muthen, L. (2013). Mplus statistical software version

7.11. https://www.statmodel.com

Pandey, J., Verbalis, A., Robins, D., Boorstein, H., Klin, A., Babitz,

T., et al. (2008). Screening for ASD in older and younger

toddlers with the Modified Checklist for ASD in Toddlers. ASD,12, 513–535. doi:10.1177/1362361308094503

J Autism Dev Disord

123

Rice, C., Nicholas, J., Baio, J., Pettygrove, S., Lee, L.-C., Braun, K.

V. N., et al. (2010). Changes in ASD spectrum disorder

prevalence in 4 areas of the United States. Disability and HealthJournal, 3, 186–201. doi:10.1016/j.dhjo.2009.10.008

Robins, D., Casagrande, K., Barton, M., Chen, C. M., Dumont-Mathieu,

T., & Fein, D. (2014). Validation of the Modified Checklist for

ASD in Toddlers, revised with follow-up (M-CHAT-R/F). Pedi-atrics, 133(1), 37–45. doi:10.1542/peds.2013-1813

Robins,D.L.,&Dumont-Mathieu,T.M. (2006). Early screening forASD

spectrum disorders: Update on the Modified Checklist for ASD in

Toddlers and other measures. Developmental and BehavioralPediatrics, 27(2), S111–S117. doi:10.1177/1362361308094503

Robins, D. L., Fein, D., Barton, M. L., & Green, J. A. (2001). The

Modified Checklist for ASD in Toddlers: An initial study

investigating the early detection of ASD and pervasive devel-

opmental disorders. Journal of ASD and DevelopmentalDisorders, 31(2), 131–144. doi:10.1007/s10803-007-0450-9

Snow, A. V., & Lecavalier, L. (2008). Sensitivity and specificity of

the Modified Checklist for ASD in Toddlers and the Social

Communciation Questionnnaire in preschoolers suspected of

having pervasive developmental disorders. ASD, 12, 627–644.doi:10.1177/1362361308097116

Stone, W. L., Coonrod, E. E., & Ousley, O. Y. (2000). Brief report:

Screening tool for ASD in two-year-olds (STAT): Development

and preliminary data. Journal of ASD and Developmental Disor-ders, 30(6), 607–612. Retrieved from http://download.springer.

com/static/pdf/57/art%253A10.1023%252FA%253A10056476290

02.pdf?auth66=1388709592_3f75704da7740879905f12a5802495b

f&ext=.pdf

Thissen, D. (2000). Reliability and measurement precision. In H.

Wainer (Ed.), Computerized adaptive testing: A primer (2nd ed.,

pp. 159–184). Hillsdale, NJ: Lawrence Erlbaum Associates

Publishers.

Thomas, M. L. (2011). The value of item response theory in clinical

assessment: A review. Assessment, 18, 291–307. doi:10.1177/1073191110374797

Ventola, P., Kleinman, J., Pandey, J., Wilson, L., Esser, E., Boorstein,

H., et al. (2007). Differentiating between autism spectrum

disorders and other developmental disabilities in children who

failed a screening instrument for ASD. Journal of Autism andDevelopmental Disorders, 37, 425–436. doi:10.1007/s10803-

006-0177-z

Wiggins, L. D., Piazza, V., & Robins, D. L. (2014). Comparison of a

broad-based screen versus disorder-specific screen in detecting

young children with an ASD spectrum disorder. ASD, 18, 76–84.doi:10.1177/1362361312466962

Wong, V. C. N., & Hui, S. L. H. (2008). Epidemiological study of

ASD spectrum disorder in China. Journal of Child Neurology,23, 67–72. doi:10.1177/0883073807308702

Yama, B., Freeman, T., Graves, E., Yuan, S., & Campbell, M. K.

(2012). Examination of the properties of the Modified Checklist

for ASD in Toddlers (M-CHAT) in a population sample. Journalof ASD and Developmental Disorders, 42, 23–34. doi:10.1007/s10803-011-1211-3

Zaroff, C. M., & Uhm, S. Y. (2012). Prevalence of ASD spectrum

disorders and influence of country of measurement and ethnicity.

Social Psychiatry and Psychiatric Epidemiology, 47(3), 395–398.doi:10.1007/s00127-011

J Autism Dev Disord

123