Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Draft
Development, Reliability and Validity Testing of Toddler
NutriSTEP®: A Nutrition Risk Screening Questionnaire for Children 18-35 Months of Age
Journal: Applied Physiology, Nutrition, and Metabolism
Manuscript ID: apnm-2015-0048.R1
Manuscript Type: Article
Date Submitted by the Author: 06-Apr-2015
Complete List of Authors: Randall Simpson, Janis; University of Guelph
Gumbley, Jillian; University of Guelph, Family Relations and Applied Nutrition Whyte, Kylie; University of Guelph, Family Relations and Applied Nutrition Lac, Jane; University of Guelph, Family Relations and Applied Nutrition Morra, Crystal; Noojmowin Teg Health Centre, Rysdale, Lee; University of Guelph, Family Relations and Applied Nutrition Turfryer, Mary; York Region Health Services, McGibbon, Kim; Thunder Bay District Health Unit, Beyers, Joanne; Sudbury & District Health Unit, Keller, Heather; University of Waterloo, Kinesiology
Keyword: nutrition < nutrition, feeding behavior < behavior, behavioural nutrition < nutrition, growth, pediatrics < medicine
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
1
Development, Reliability and Validity Testing of Toddler NutriSTEP®: A Nutrition Risk
Screening Questionnaire for Children 18-35 Months of Age
Authors:
Randall Simpson, Janis, University of Guelph, Guelph, Ontario, N1G 2W1, Canada
Gumbley, Jillian, University of Guelph, Guelph, Ontario, N1G 2W1, Canada
Whyte, Kylie, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]
Lac, Jane, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]
Morra, Crystal, Noojmowin Teg Health Centre, Little Current, Ontario, P0P 1K0, Canada
Rysdale, Lee, University of Guelph, Guelph, Ontario, N1G 2W1, Canada [email protected]
Turfryer, Mary, York Region Public Health, Newmarket, Ontario, L3Y 6Z1, Canada
McGibbon, Kim, Thunder Bay District Health Unit, Ontario, P7B 6E7, Canada
Beyers, Joanne, Sudbury & District Health Unit, Sudbury, Ontario, P3E 6H3 Canada
Keller, Heather, Schlegel-University of Waterloo Research Institute for Aging, University of
Waterloo, Waterloo, Ontario, N2L 3G1 Canada [email protected]
Corresponding Author:
Janis Randall Simpson, Department of Family Relations and Applied Nutrition, University of Guelph,
50 Stone Road East, Guelph, Ontario, Canada N1G 2W1. Phone: 519 924-4120, ext 53843. FAX:
519 766-0691. E-mail: [email protected].
Page 1 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
2
Funding:
Canadian Institutes of Health Research, University of Guelph, University of Guelph-Humber
Conflict of Interest:
Janis Randall Simpson, Lee Rysdale and Heather Keller receive royalties from the sale of the version
of Toddler NutriSTEP® that has scoring options for use by implementers of the NutriSTEP® screening
program at: www.Flintbox.com.
A version without scoring options is available free of charge for parents at: www.nutritionscreen.ca.
Page 2 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
3
Abstract
Background /Objectives
Nutrition is vital for optimal growth and development of young children. Nutrition risk screening can
facilitate early intervention when followed by nutritional assessment and treatment. NutriSTEP® is a
valid and reliable nutrition risk screening questionnaire for preschoolers (3-5 years). A need was
identified for a similar questionnaire for toddlers (18-35 months). The purpose was to develop a
reliable and valid Toddler NutriSTEP®.
Subjects/Methods
Toddler NutriSTEP® was developed in four phases. Content and face validity were determined with a
literature review, parent focus groups (n=6, 48 participants) and experts (n=13) (Phase A). A draft
questionnaire was refined with key intercept interviews of 107 parents/caregivers (Phase B). Test-
retest reliability (Phase C), based on intra-class correlations (ICC), Kappa (ĸ) statistics, and Wilcoxon
tests was assessed with 133 parents/caregivers. Criterion validity (Phase D) was assessed using
Receiver Operating Characteristic (ROC) curves by comparing scores on the Toddler NutriSTEP® to a
comprehensive nutritional assessment of 200 toddlers with a registered dietitian (RD).
Results
The Toddler NutriSTEP® was reliable between two administrations (ICC=0.951, F=20.53, p< 0.001);
most questions had moderate (ĸ≥0.6) or excellent (ĸ≥0.8) agreement. Scores on the RD nutrition risk
rating and the Toddler NutriSTEP® were correlated (r=0.67, p<0.000). The area under the ROC curve
for moderate and high RD risk ratings were 84.6% and 82.7%, respectively. Cut-points of ≥21
(sensitivity 86%; specificity 61%) (moderate risk) and ≥26 (sensitivity 95%; specificity 63%) (high
risk) were determined.
Conclusions
Page 3 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
4
The Toddler NutriSTEP® questionnaire is both reliable and valid for screening for nutritional risk in
toddlers.
Running Title:
Development of Toddler NutriSTEP®
Key Words:
Nutrition risk screening, toddlers, validity, reliability, NutriSTEP®
Page 4 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
5
Introduction
Nutrition is vital for optimal growth and development. Unhealthy feeding/eating environments (e.g.,
inappropriate foods served or lack of division of responsibility) put children at risk for poor nutrition
(Satter 1995; Savage et al. 2007). Young children with poor nutrition are at risk for growth,
behavioural and developmental problems, including overweight/obesity (Thompson et al. 2014).
Delayed, inadequate or inappropriate nutrition intervention and management has both short- and long-
term health consequences for growing children, including iron deficiency (Christofides et al. 2005;
Paoletti et al. 2014), and lay the foundation for chronic diseases such as diabetes, heart disease, obesity,
and osteoporosis (Halfon et al. 2012).
Nutrition screening identifies those at nutritional risk who may be asymptomatic, using
characteristics of known nutrition problems for the purpose of assessment and treatment ("ethical
screening") (American Dietetic Association 1994; Kondrup et al. 2003; Rush 1997). Because
eating habits and patterns are established at an early age, it is important to address the
foundations of healthy eating and a healthy lifestyle during the toddler and preschool periods.
Thus, screening has this secondary purpose of increasing awareness of potential nutritional
problems for parents and care providers. Screening to facilitate early action, where intervention
may be less intensive and costly than assessment, is crucial in effective primary health care
models (Kondrup et al. 2003).
NutriSTEP® (Nutrition Screening Tool for Every Preschooler) is a valid and reliable 17-item,
community-based, parent-completed questionnaire for identifying nutritional risk in preschoolers (3-5
years) (Randall Simpson et al. 2008). NutriSTEP® can be used to address nutrition concerns with
parents; further, the use of NutriSTEP® in public health settings can provide data for monitoring and
surveillance. NutriSTEP® has been implemented in Canadian public health jurisdictions including
Page 5 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
6
Ontario as an accountability indicator (Ontario 2014) and New Brunswick (New Brunswick Public
Health 2012), and is used in research and primary care (Persaud et al. 2013; Watson-Jarvis et al. 2011).
Since the release of the preschool NutriSTEP®, other pediatric nutrition risk screening tools have been
developed, but are for use in acute/hospital settings (Gerasimidis et al. 2010; Hulst et al. 2010;
McCarthy et al. 2012; Secker and Jeejeebhoy 2007). A gap continues to exist for identifying nutrition
risk in community-living toddlers. A recently-developed Australian questionnaire focuses on toddlers’
food group intake (Bell et al. 2014); however, it does not include other diverse nutrition risk factors,
such as physical growth and development, physical activity and sedentary behaviour, and family
factors related to nutrition. Other toddler questionnaires address only caregiver feeding behaviours and
practices (Chaidez et al. 2011; Hurley et al. 2013) and not other nutrition risk factors.
During the implementation of the preschool version of NutriSTEP®, exploring the validation of
NutriSTEP® for children of 18 months of age was suggested (Watson-Jarvis et. 2011), as this age group
has unique nutritional issues and risks that are different from preschoolers. Additionally, a senior
public health official in Ontario (S. Basrur (personal communication, 2006)) suggested that a nutrition
risk questionnaire for this age group was needed to complement universal screening for 18 month old
children in Ontario that began in 2009 (Ontario Ministry of Children and Youth Services 2011).
The preschool NutriSTEP® has four risk factor areas (attributes): food and fluid intake; physical
growth; physical activity and sedentary behaviour; and, factors affecting food intake (Randall Simpson
et al. 2008). All of these attributes are also relevant to nutrition risk in toddlers and thus the preschool
NutriSTEP® could be used as an initial starting point for the toddler version. While some questions, or
modifications of questions, on the preschool NutriSTEP® are appropriate for toddlers, there are
developmental issues for toddlers that are not addressed in the preschool version. Based on the
literature and on expert and parent/caregiver opinions, additional concerns, unique to toddlers, such as
Page 6 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
7
developmental milestones related to food and fluid consumption, and feeding relationships were
considered for inclusion.
The overall purpose of this article is to describe the development, reliability and validity testing of
Toddler NutriSTEP® for children 18-35 months of age. The ultimate goals of this screening tool are
three-fold: to identify toddlers who require a nutrition assessment to diagnose and treat their impaired
state; to provide referrals for needed services; and, to track population-level nutrition risk in toddlers.
Subjects and Methods
Design
Toddler NutriSTEP® was developed in four phases (Figure 1) (Phase A: Face and Content Validity;
Phase B: Refinement; Phase C: Reliability; and, Phase D: Validation), using a combined
clinimetric (reliance on intuition and experience to create clinically useful instruments),
psychometric (reliance on the generation of many question items to address attributes of areas of
interest) and criterion (validation compared to a 'gold standard’) method (Keszei et al. 2010; Streiner
and Norman 2008). The project was modeled on the development of the original preschool NutriSTEP®
(Randall Simpson et al. 2008) and adapted from Keller et al. (2000; 2001) and Jones (2004a).
Subjects
For all phases, convenience samples of parents/caregivers of toddlers from ethnically, geographically,
and socio-demographically diverse backgrounds in rural and urban areas throughout Southern and
Northern Ontario, Canada were recruited by research assistants at Ontario Early Years Centres and at
Peer Nutrition (programs through Toronto Public Health). To ensure representativeness, and thus
applicability, to potentially vulnerable groups, populations with possible low and high nutrition risk
(including European, Middle Eastern, Asian, African and Aboriginal Canadians) were specifically
recruited. Inclusion criteria for participants were that they were parents/primary caregivers of toddlers
and able to read and write English at a grade six level. Residence in Canada for at least five years was
Page 7 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
8
also an inclusion criteria for participants in the first three phases. As demographic and socioeconomic
diversity was desired for validation, this inclusion criterion was not used in Phase D and sites in
multiethnic Toronto were used for recruitment. Exclusion criteria included hospitalized and
institutionalized children. Ethics clearance was provided by the University of Guelph Research Ethics
Board. Participants provided informed signed consent and completed a demographic form used
previously (Randall Simpson et al. 2008; Rysdale et al. 2011). Incentives included grocery vouchers
(Phases A, B, C: $20 CDN; Phase D: $40 CDN) and parent education materials. Focus group
participants in Phase A received a healthy snack.
Methods
Phase A: Face and Content Validity
The objective of this phase was to determine and confirm what constitutes nutritional risk in toddlers
and to draft the Toddler NutriSTEP®.
A literature review was conducted to assess the current expert understanding of what constitutes toddler
nutrition risk and to determine if there were any existing nutrition risk questionnaires for this age group
(Streiner and Norman 2008). Parents/caregivers were then recruited for focus groups to confirm their
concerns and risk behaviours for this age group (Kreuger and Casey 2009; Randall Simpson et al.
2008). Six one-hour focus groups were conducted by a trained moderator and two trained note takers;
a pilot-tested standardized script about the suitability of the NutriSTEP® preschool questions for
toddlers was used, considering possible deletions or additions for the toddler age group. Debriefing
amongst researchers after each focus group identified if changes were required to the script to focus the
discussion and to develop further probes. A single transcriptionist transferred audio to text; transcripts
were checked for errors and were reviewed by core members of the research team (JRS, JG, LR, HK,
JB). Key findings from these focus groups and from the literature review became the basis for the draft
Toddler NutriSTEP®.
Page 8 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
9
The content of the draft Toddler NutriSTEP® was reviewed by 13 (an appropriate number (Streiner and
Norman 2008)) registered dietitians (RDs), with expertise in pediatrics and with a variety of work
experience from across Canada. These experts not only confirmed the content of the questions but also
verified language to promote clarity (Keszei et al. 2010). An online survey (pre-tested with nutrition
graduate students) was used to rank draft questions based on their relative importance to further
confirm content and identify items less relevant for potential exclusion. Further, a group
teleconference with the same RDs, using a discussion guide, was used to address each proposed
Toddler NutriSTEP® question, clarifying concerns or language, to reach consensus on final questions.
Phase B. Refinement
The objective of this phase was to refine and finalize the draft Toddler NutriSTEP®.
Key intercept interviews were conducted by six trained research assistants to ensure that wording of
question stems and response options was understandable, user friendly, and culturally-appropriate to
diverse parents. The goal was to recruit approximately 100 participants (Streiner and Norman 2008).
These parents first completed the draft Toddler NutriSTEP® and then were asked a series of open-
ended questions using a standardized interview guide about each question stem to confirm the expected
cognitive processing (Randall Simpson et al. 2008). Participants' understanding, thoughts, and
opinions of the question stems, possible examples, and question response options were recorded.
Using qualitative content analysis (Harris et al. 2009), the feedback was reviewed and consideration
was given to any potential changes to item stems, examples and response categories.1
Phase C: Test-retest reliability
The objective of this phase was to determine the test-retest reliability of Toddler NutriSTEP®.
1 The question stems are given in Supplementary Material (Table S1).
Page 9 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
10
A sample size of approximately140 participants was desired and considered more than adequate based
on a desired Kappa (ĸ ) value (0.8) and anticipated prevalence of risk (30-40%) (Jones 2004b; Randall
Simpson et al. 2008; Streiner and Norman 2008). At recruitment, participants completed the draft
Toddler NutriSTEP®; approximately 2-4 weeks later (Jones 2004b), participants completed the draft
questionnaire for the second time.
Phase D: Validation
The primary objective of this phase was to determine criterion validity of Toddler NutriSTEP® relative
to a nutrition risk assessment by an RD, considered to be the 'gold standard' (American Dietetic
Association 1994; Jones 2004c; Streiner and Norman 2008).
Sample size for criterion validity was estimated at between 110-200 participants based on: 1) estimates
for a Receiver Operating Characteristic (ROC) curve analysis (~110) with area under the curve (AUC)
of 85% (Streiner and Cairney 2007); and, 2) estimates of specificity of 75%, risk of 26%, error of 10%
and 95% confidence interval (~215) (Jones 2004c). Our goal was set at 200 participants.
The validation phase involved two visits with the parents/caregivers and toddlers by one RD with
expertise in pediatrics who conducted a comprehensive nutritional assessment of the toddlers. At the
first visit, parents/caregivers completed the draft Toddler NutriSTEP® which was then placed into a
sealed envelope so that the RD was blinded to responses. Toddlers’ weights were measured (to 50 g)
in triplicate using a calibrated scale, with a removable weighing tray for younger toddlers (Tanita
Digital Baby Scale (1584), Arlington Heights, IL, USA). Height (to 0.1 cm) for toddlers ≥24 months
of age was measured using a portable stadiometer (Road Rod (214), SECA, Chino, CA, USA); length
(to 0.1 cm) was measured using a portable measuring mat for toddlers <24 months of age (Starters
Pediatric Measure Mat, Slater & Frith, Norwich, UK). Weight for length (<2 years), and BMI for age
(≥2 years) was assessed using the 2010 World Health Organization growth standards adapted for
Page 10 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
11
Canada (Dietitians of Canada 2014). Finally, parents/caregivers were given instructions to complete a
three-day food record.
Within one month, the second visit was completed. The food record was reviewed and consumption
amounts and patterns were compared with Canada's Food Guide (Health Canada 2012). The RD
completed a comprehensive content-valid assessment interview using a risk-rating guide2, based on the
literature review (Phase A), and adapted from the preschool version (Randall Simpson et al. 2008). The
RD risk rating was a 10-point scale used in previous research (low risk (1-4); moderate risk (5-7); high
risk (8-10)) (Keller et al. 2001; Keller et al. 2005; Randall Simpson et al. 2008). Construct validity was
also determined through comparison of the researchers' theoretical expectations of the questionnaire
across demographic groups with the performance of the tool itself (Jones 2004c).
Data analysis
All quantitative data were entered in duplicate to check for accuracy. Data analysis was conducted
with SPSS (version 20, IBM Inc., Chicago, IL, USA) or SAS (version 9.3, SAS Systems Inc., Cary,
NC, USA). Demographic information was analyzed using descriptive statistics. Test-retest reliability
was assessed via intra-class correlations (ICC), Wilcoxon signed rank tests, Pearson's correlation, and a
paired sample t-test (Jones 2004b; Streiner and Norman 2008). Kappa statistics, ranging from 0-1, with
higher values indicating greater reliability, were also calculated for individual questions (dichotomized
into 'risk' and 'no risk') to determine if they were significantly different across the two occasions (Jones
2004b). To assess validity, scores (0-68) on the Toddler NutriSTEP® were compared to the RD risk
rating using ROC curves that were created using moderate (≥5) and high risk (≥8) cut-points on the 10-
ten point RD rating scale (Streiner and Cairney 2007; Streiner and Norman 2008). A high AUC
indicates that the measured variable is consistent with scoring of the criterion (Streiner and Norman
2008). Appropriate risk cut-points for Toddler NutriSTEP® were identified by comparing trade-offs of
2 Supplementary Material Table S2
Page 11 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
12
sensitivity (Se) and specificity (Sp) for various scores using the ROC curves output, considering as well
the original preschool NutriSTEP® scoring. To assess construct validity, t-tests and analysis of
variance were used to compare the average response on the questionnaire across varying risk groups to
see if hypothesized trends existed (Jones 2004c; Streiner and Norman 2008).
Results
Subjects
As shown in Table 1, most parents/caregivers were in their early 30s, female, married, with post-
secondary education, and with household incomes >$60,000 per year. Many parents/caregivers and
children in Phases A, B and D did not speak English as their first language. Many parents/caregivers,
particularly in Phases A and D, were not born in Canada, whereas most of the toddlers in all phases
were born in Canada. There was no attrition of participants in Phases A and B.
Phases A and B
Literature regarding NutriSTEP® nutrition risk issues in toddlers was limited, with most North
American data collected from just two studies: Continuing Survey of Food Intakes by Individuals
(CSFII) (Kranz et al. 2004) and Feeding Infants and Toddlers Study (Devaney et al. 2003). These
surveys provided information on issues specific to toddlers that were not already considered in the
preschool NutriSTEP®. A number of nutrition risk issues, specifically for toddlers, were identified:
changes in milk (including breastmilk and formula) consumption (Hawkins and Law, 2005; Siega-Riz
et al. 2010; Skinner et al. 2004); introduction of inappropriate table foods (Briefel et al. 2004; Butte et
al. 2010; Fox et al. 2004); excess intake of fruit juices and sweetened beverages (Fox et al. 2004; Fox et
al. 2006; Nicklas and Hayes 2008; Siega-Riz et al. 2010); picky eating (Carruth et al. 2004);
development of inappropriate parent-toddler feeding relationships (Satter 1990); and, prolonged bottle
and/or “sippy-cup” use (Bonuck et al. 2004). As well, no existing nutrition risk screening tool for
community-living toddlers was identified.
Page 12 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
13
Forty-eight parents/caregivers participated in the six focus groups. Participants confirmed the
importance of almost all preschool NutriSTEP® questions, with some modifications. They also
identified non-nutrition issues unique to toddlers that can impact a child’s weight such as too much
time spent in car seats/strollers and poor sleep routines. Based on the literature review and the focus
group feedback, a 21-item Toddler NutriSTEP® was drafted incorporating the developmental issues of
toddler nutrition. After expert consultation, this was reduced to 19 items for the next phase (Phase B).
Key intercept interviews were conducted with 107 parents/caregivers of toddlers. Major suggestions
were the inclusion of more examples and clarification of some terms (e.g., fast food). Consideration of
these comments and suggestions resulted in a final 17-item draft Toddler NutriSTEP® used in Phases C
and D. The final draft included the following: four new toddler-specific questions (e.g., drinking from
a baby bottle; child feeding his/her self); six questions modified from the original preschool
NutriSTEP® for toddlers (e.g., including breast milk as an example for the question on consumption of
milk products); and, seven original preschool NutriSTEP® questions (e.g., questions on: frequency of
consumption of food groups from Canada's Food Guide; food security; frequency of consumption of
fast food; amount of "screen time").
Phase C
Of the 158 parents/caregivers recruited, 141 completed the draft Toddler NutriSTEP® on two visits, on
average 17±1 days apart; 133 had complete data that were used for this analysis. Total mean scores
(±SD) on the Toddler NutriSTEP® (15±7 for both; range 3-34 of a possible 68) were found to be highly
reliable (ICC=0.951, F=20.53, p<0.001). The Pearson's correlation on the total Toddler NutriSTEP®
score between administrations was significant (r=0.91, p<0.001) and the paired sample t-test was not
significantly different over time (t-statistic=0.882, p>0.05).
All dichotomized responses to individual questions had significant ĸ statistics (p<0.001), with most
showing at least fair reliability (ĸ>0.4) (Table 2). Further, most questions had non-significant
Page 13 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
14
Wilcoxon tests, indicating reliability on all response options, with only three questions having
significant differences: grain product consumption, milk product consumption, and the child deciding
how much to eat.
Phase D
Of the 218 families recruited for the validation phase, 200 completed both visits. The mean (±SD)
score on Toddler NutriSTEP® was 19.8±8.7 with a range of 3-46 (of a possible 68). Items that placed
toddlers in this phase at risk were: low consumption of grains, fruits and vegetables and meat and
alternatives; finding food expensive; drinking from a bottle with a nipple; not being hungry at meals;
and, weight concerns.
The ROC curve for Toddler NutriSTEP® scores compared to the RD high risk scores (score ≥8) is
shown in Figure 2 with an AUC of 82.7%. At the moderate risk cut-point for the RD rating (score ≥5),
the AUC was 84.6% (ROC curve not shown). Data for Se and Sp for the moderate and high risk RD
cut points (scores ≥5 and ≥8, respectively), as well as the prevalence of toddlers at various Toddler
NutriSTEP® scores, are shown in Table 3. Cut-points were determined for three risk classification
categories as follows: low risk (score ≤20); moderate risk (score 21-25); and, high risk (score ≥26).
Based on these data, a cut point of ≥21 has Se of 86% and Sp of 61% compared to the RD moderate
risk rating and Se of 71% and Sp of 80% compared to the RD high risk rating. A cut point of ≥26 has
Se of 95% and Sp of 43% compared to the RD moderate risk rating and Se of 83% and Sp of 63%
compared to the RD high risk rating.
The mean (±SD) RD risk score was 4.8±2.5. The Spearman's rho between the scores on Toddler
NutriSTEP® and the RD risk rating scores was 0.67 (p<0.000). Further, the mean scores for the RD-
rated risk categories for Toddler NutriSTEP®, based on cut-points in Table 3, were significantly
different (F=349.3, p<0.000), indicating that NutriSTEP® risk classifications were reflective of the
RD risk ratings.
Page 14 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
15
Results of the tests for construct validity are shown in Table 4. As hypothesized, NutriSTEP® scores
were significantly lower (lower risk) for parents born in Canada, with higher household income, and
higher levels of education.
Discussion
The major outcome of this research is the successful development of a valid and reliable nutrition risk
questionnaire for toddlers. The validation results (i.e., AUC 83% and 85% for moderate and high risk,
respectively) are slightly better than for the original NutriSTEP® (i.e., AUC 82% and 74% for moderate
and high risk rating, respectively) (Randall Simpson et al. 2008). Furthermore, the Se and Sp at the
final chosen cut-points for the toddler version are comparable to the preschooler version. The
sensitivity of Toddler NutriSTEP® (>86%) is higher than for malnutrition screening tools (59-70%)
designed for hospitalized children (Gerasimidis et al. 2010; McCarthy et al. 2012).
The validation was conducted on a diverse sample, providing a wide range of nutrition risk scores.
There was a considerable prevalence of high nutritional risk (27%) for children in Phase D. This was
considerably higher than for Phase C (7%) and in comparison to the preschool NutriSTEP® validation
sample (14%) (Randall Simpson et al. 2008), and other NutriSTEP® research based on convenience
samples (7%) (Rysdale et al. 2011; Watson-Jarvis et al. 2011). Differences in the prevalence of
problematic issues by item for Phases C and D are also different and are likely attributed to successful
recruitment of a very diverse sample from urban Toronto for Phase D. For example, significantly more
mothers in Phase C versus Phase D were born in Canada, had English as their first language, had a
higher level of education and higher household income (all p<0.002). As expected, based on
previously- documented associations between socioeconomic status and nutrition (Bradley and Corwyn
2002; Hendricks et al. 2006), NutriSTEP® scores were lower for those toddlers from families with
higher income and higher level of education. Children of immigrant parents have been reported to be
at nutritional risk because of many issues, including food insecurity and consumption of foods high in
Page 15 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
16
energy and fat (Flaskerud and Kim 1999; Satia-Abouta et al. 2002). Our findings reflect the
relationships between education, income, immigration status and nutritional risk of children seen in the
literature, and therefore demonstrate construct validity of the Toddler NutriSTEP®. There was a higher
prevalence of toddlers classified as overweight and/or obese (43%) in the validation sample relative to
the preschool validation sample (21%) which was less diverse (Randall Simpson et al. 2008), and to
Canadian preschool children in the CCHS (26%) (Shields 2006). Demographic differences were also
reflected in the greater number of questions with a high prevalence of risk for the validation sample
(Phase D) versus the reliability sample (Phase C).
Common areas of concern included low frequency of consumption of grain products (47-64%,
for Phases C and D, respectively), also reported in a national sample of Canadian toddlers (Kirkpatrick
and Tarasuk 2008). More than 30% drank from a baby bottle with a nipple, even though the transition
from bottle feeding to an open cup is recommended to be complete by 18 months of age (Bonuck et al.
2004; Health Canada 2014). Over 33% watched TV or played video games more than two hours per
day whereas recent Canadian recommendations are for less than one hour per day (Canadian Society
for Exercise Physiology 2012). These results outline areas of modifiable nutrition risk and provide
evidence to support public health and primary health care interventions directed at parents/caregivers of
toddlers.
The Toddler NutriSTEP® is considered reliable based on analyses of total score and on individual
questions. These findings are consistent with the original preschool NutriSTEP® (Randall Simpson et
al. 2008). Reliability is slightly higher (ICC=0.95) than the Australian food group questionnaire
(ICC=0.90) (Bell et al. 2014) and higher than other toddler risk tools for hospitalized children
(Gerasimidis et al. 2010; McCarthy et al. 2012). Based on the ĸ values for the dichotomized individual
questions, and on the Wilcoxon tests, the question on frequency of grain consumption was less reliable
Page 16 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
17
than the others. This question may require future refinement to promote reliability, if a true variability
in intake was not the reason for this difference.
Two other questions (frequency of milk consumption; child decides how much to eat) also
demonstrated some variability in repeat administration as analyzed with Wilcoxon tests. However,
both questions had significant and adequate ĸ statistics (0.70 and 0.53, respectively) indicating that,
although the absolute score on the question from time one to time two differed, the risk classification
for that question did not. Therefore, these two questions are considered to have adequate reliability, as
they are still able to reliably classify a child into a particular risk-rating category across administrations.
Strengths of this work include the rigorous development, reliability and criterion validation testing of
Toddler NutriSTEP® and inclusion of a diverse sample, including Aboriginal Peoples. Other recent
dietary/feeding practice questionnaires for toddlers have been developed in more specialized groups
such as Latino mothers (n=94) in the United States (Chaidez et al. 2011), low-income African
American mothers (n=297) (Hurley et al. 2013), and relatively advantaged samples in Australia
(n=111) (Bell et al. 2014). Toddler NutriSTEP® is novel in that it is comprehensive and includes
attributes (e.g., physical growth and development, physical activity and sedentary behaviour, and
family factors related to nutrition) in addition to dietary intake that differentiates it from the Australian
food-group-based dietary questionnaire, that was validated relative to a food frequency questionnaire
(Bell et al. 2014). Further, the Toddler NutriSTEP® assesses comprehensive nutrition risk compared to
other toddler questionnaires that focus only on caregiver feeding practices and behaviour. For
example, Chaidez et al. (2011) developed a 34-item questionnaire with construct validation of
indulgent, authoritative and environmental feeding practices of caregivers; and, Hurley et al. (2013)
developed a 27-item questionnaire with construct validation of five caregiver feeding behaviours
(responsive, forceful/pressuring, restrictive, indulgent, uninvolved). A further strength is that a single
RD conducted all 200 validation assessments thereby minimizing variability in the criterion
Page 17 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
18
assessment. The sample size for all phases met or exceeded recommendations and was sufficient to
answer the research questions. This study is not, however, without limitations. As convenience and
purposive samples were recruited, it is not surprising that demographics were not consistent with the
Canadian population; the reliability sample had higher levels of education and the validation sample
had a higher representation of vulnerable groups compared to the Canadian population (Statistics
Canada 2011). As such, prevalence estimates of nutrition risk from these studies need confirmation in
a population-level representative sample of Canadian toddlers; however, the validity of the tool is not
affected by this sampling bias. Key intercept interviews, although completed by highly-trained
interviewers, were likely completed by engaged parents who may be atypical. Nevertheless, quality
insights into comprehension of the questions were provided from this group and the reliability and
validation samples did not identify any further challenges with the questionnaire itself.
Conclusions
Toddler NutriSTEP® is a rigorously-developed, valid and reliable nutrition risk screening tool for use
by parents of children aged 18-35 months. This tool fills an important research and practice gap as well
as offers potential for population health surveillance and monitoring of nutrition risk. Well-baby visits
become less frequent after the second year of life. Toddlers continue to transition fully to table foods
and to develop self-feeding practices. Without on-going monitoring, potential unhealthy behaviours can
develop with increased nutritional and subsequent health problems. The prevalence of nutrition risk in
this group suggests a need for systematic screening programs in community settings. Toddler
NutriSTEP® is an easy-to-use tool for this purpose. Further research will include predictive validation
and evaluation of the effectiveness of Toddler NutriSTEP®.
Page 18 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
19
Acknowledgements
The contributions of the following are gratefully acknowledged by the authors. Michelle Edwards
(Statistician, University of Guelph), Elizabeth Shaver Heeney (City of Hamilton Public Health
Services), Heather St. Bernard John (Toronto Public Health), University of Guelph students (Kimberly
Booker, Lauren Baker, Alison Weber), University of Guelph-Humber student (Kimberlee Rizun-
Glynn), Northern Ontario Dietetic Internship Program students (Stephanie Hill, Riley Fulkerson, Jenna
Campbell, Tammy Vachon), and the many Ontario Early Years Centres across Ontario that participated
in this research.
Conflict of Interest
Janis Randall Simpson, Lee Rysdale and Heather Keller receive royalties from the sale of the version
of Toddler NutriSTEP® that has scoring options for use by implementers of the NutriSTEP® screening
program at: www.Flintbox.com. However, an online version without scoring options is available free
of charge for parents at: www.nutritionscreen.ca.
Page 19 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
20
References
ADA (American Dietetic Association). 1994. Identifying patients at risk: ADA’s definitions for
nutrition screening and nutrition assessment. Council on Practice (COP) Quality Management
Committee. J. Am. Diet. Assoc. 94: 838-839. PMID: 8046173.
Bell, L.K., Golley, R.K., and Magarey, A.M. 2013. A short food-group-based dietary questionnaire
is reliable and valid for assessing toddlers' dietary risk in relatively advantaged samples. Br. J.
Nutr. J2: 1-11. doi: 10.1038/ejcn.2013.23.
Bonuck, K.A., Huang, V., and Fletcher, J. 2010. Inappropriate bottle use: An early risk for
overweight? Literature review and pilot data for a bottle-weaning trial. Matern. Child Nutr. 6(1):
38-52. doi: 10.1111/j.1740-8709.2009.00186.x.
Bradley, R.H., and Corwyn, R.F. 2002. Socioeconomic status and child development. Ann. Rev.
Psychol. 53: 371-399. PMID:11752490.
Briefel, R.R., Reidy, K., Karwe, V., Jankowski, L., and, Hendricks, K. 2004. Toddlers’ transition to
table foods: Impact on nutrient intakes and food patterns. J. Am. Diet. Assoc. 104(1): S38-S44.
PMID:14702016.
Butte, N.F., Fox, M.K., Briefel, R.R., Siega-Riz, A.M., Dwyer, J.T., Deming, D.M., et al. 2010.
Nutrient intakes of US infants, toddlers, and preschoolers meet or exceed dietary reference intakes.
J. Am. Diet. Assoc. 110(12): S27-S37. doi: 10.1016/j.jada.2010.09.004.
Page 20 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
21
Canada. Statistics Canada. 2011. Employment and Social Development Canada. Ottawa: Statistics
Canada; Table 202-0605 - Median after-tax income, by economic family type, constant dollars,
annual (dollars). Indicators of Well-Being in Canada [online]. Available from
http://www40.statcan.ca/z01/cs0002-eng.htm [accessed 3 February 2015].
Canadian Society for Exercise Physiology. 2012. Canadian Physical Activity Guidelines -
Background Information [online]. Available from http://www.csep.ca/english/view.asp?x=587
[accessed 2 February 2015].
Carruth, B.R., Ziegler, P.J., Gordon, A., and Barr, S.I. 2004. Prevalence of picky eaters among
infants and toddlers and their caregivers' decisions about offering a new food. J. Am. Diet. Assoc.
104(1 Suppl 1): s57-64. PMID:14702019.
Chaidez, V., and Kaiser, L.L. 2011. Validation of an instrument to assess toddler feeding practices
of Latino mothers. Appetite. 57:229-236. PMID:21600943.
Christofides, A., Schauer, C., and Zlotkin, S.H. 2005. Iron deficiency and anemia prevalence and
associated etiologic risk factors in First Nations and Inuit communities in northern Ontario and
Nunavut. Can. J. Public Health 96: 304-307. PMID:16625803.
Devaney, B., Kalb, L., Briefel, R., Zavitshy-Novak, T., Clusen, N., and Ziegler, P. 2004. Feeding
infants and toddlers study: overview of the study design. J. Am. Diet. Assoc. 104, Suppl 1: s8-13.
PMID:14702012.
Page 21 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
22
Dietitians of Canada. 2014. WHO Growth Charts Adapted for Canada. Summary of Changes -
March 2014. [online]. Available from
http://www.dietitians.ca/Downloads/Public/WHO-Growth-Charts-Summary-of-Change-March-
2014.aspx [accessed 31 March 2015].
Edington, J. Problems of nutritional assessment in the community. 1999. Proc. Nutr. Soc. 58; 47-
51. PMID:10343339.
Flaskerud, J.H., and Kim, S. 1999. Health problems of Asian and Latino immigrants. Nurs. Clin.
North Am. 34(2): 359-380. PMID:10318729.
Fox, M.K., Pac, S., Devaney, B., and Jankowski, L. 2004. Feeding infants and toddlers study: What
foods are infants and toddlers eating? J. Am. Diet. Assoc. 104(1): S22-S30. PMID:14702014.
Fox, M.K., Reidy, K., Novak, T., and Ziegler, P. 2006. Sources of energy and nutrients in the diets
of infants and toddlers. J. Am. Diet. Assoc. 106(1 Suppl 1): S28-S42. PMID:16376628.
Gerasimidis, K., Keane, O., Macleod, I., Flynn, D.M., and Wright, C.M. 2010. A four stage
evaluation of the Paediatric Yorkhill Malnutrition Score in a tertiary paediatric and district
general hospital. Br. J. Nutr. 104: 751-756. doi: 10.1017/S0007114510001121.
Halfon, N., Verhoef, P.A., and Kuo, A.A. 2012. Childhood antecedents to adult cardiovascular
disease. Pediatr. Rev. 33(2): 51-60. doi: 10.1542/pir.33-2-51.
Page 22 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
23
Harris, J.E., Gleason, P.M., Sheean, P.M., Boushey, C., Beto, J.A., and Bruemmer, B. 2009. An
introduction to qualitative research for food and nutrition professionals. J. Am. Diet. Assoc. 109:
80-89. doi: 10.1016/j.jada.2008.10.018.
Hawkins, S.S., and Law, C. 2005. A review of risk factors for overweight in preschool children: A
policy prospective. Int. J. Pediatr. Obes. 1: 195-209. PMID:17907326.
Health Canada. 2012. Ottawa: Canada's Food Guide [online]. Available from http://hc-sc.gc.ca/fn-
an/food-guide-aliment/index-eng.php [accessed 2 February 2015].
Health Canada. 2014. Food and Nutrition; p.540-541. Nutrition for Healthy Term Infants:
Recommendations from Birth to Six Months. A joint statement of Health Canada, Canadian
Paediatric Society, Dietitians of Canada, and Breastfeeding Committee for Canada; 2014 May 27
[online]. Available from http://www.hc-sc.gc.ca/fn-an/nutrition/infant-nourisson/recom/index-
eng.php [accessed 3 February 2015].
Hendricks, K., Briefel, R., Novak, T., and Ziegler, P. 2006. Maternal and child characteristics
associated with infant and toddler feeding practices. J. Am. Diet. Assoc. 106(1 Suppl 1): S135-148.
PMID:16376637.
Hulst, J.M., Zwart, H., Hop, W.C., and Joosten, K.F.M. 2010. Dutch national survey to test
the STRONGkids nutritional risk screening tool in hospitalized children. Clin. Nutr. 29:
106-111. doi: 10.1016/j.clnu.2009.07.006.
Page 23 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
24
Hurley, K.M., Pepper, M.R., Candelaria, M., Wang, Y., Caulfield, L.E., Latta, L., et al. 2013.
Systematic development and validation of a theory-based questionnaire to assess toddler feeding. J.
Nutr. 143:2014-2049. PMID:24068792.
Jones, J.M. 2004a. Development of a nutritional screening or assessment tool using a multivariate
technique. Nutrition 20: 298-306. PMID:14990272.
Jones, J.M. 2004b. Reliability of nutritional screening and assessment tools. Nutrition 20: 307-311.
PMID:14990273.
Jones, J.M. 2004c. Validity of nutritional screening and assessment tools. Nutrition 20: 312-317.
PMID:14990274.
Keller, H.H., Hedley, M.R., and Wong Brownley, S. 2000. The development of seniors in the
community: Risk evaluation for eating and nutrition (SCREEN). Can. J. Diet. Pract. Res. 61(2): 67-
72. PMID: 11551350.
Keller, H.H., McKenzie, J.D., and Goy, R.E. 2001. Construct validation and test-retest reliability of
the seniors in the community: Risk evaluation for eating and nutrition questionnaire. J. Gerontol.
56A(9): M552-M558. PMID:11524447.
Keller, H.H., Goy, R., and Kane, S.L. 2005. Validity and reliability of SCREEN II (Seniors in the
community: Risk evaluation for eating and nutrition, version II). Eur. J. Clin. Nutr. 59: 1149-1157.
PMID:16015256.
Page 24 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
25
Kirkpatrick, S.I., and Tarasuk, V. 2008. Food insecurity is associated with nutrient inadequacies
among Canadin Adults and Adolescents. (Online supporting material) J. Nutr. 138: 604-612.
PMID:18287374.
Kondrup, J., Allison, S.P., Elia, M., Vellas, B., and Plauth, M. 2003. ESPEN Guidelines for
Nutrition Screening 2002. Clin. Nutr. 22(4): 415-421. PMID:12880610.
Kranz, S., Siega-Riz, A.M., and Herring, A.H. 2004. Changes in diet quality of American
preschoolers between 1977 and 1998. Am. J. Public Health 94: 1525-1530. PMID:15333309.
Kreuger, R.A., and Casey, M.A. 2009. Focus groups. A practical guide for applied
research. 4th edition. Sage Publications, Thousand Oaks, CA.
McCarthy, H., Dixon, M., Crabtree, I., Eaton-Evans, M.J., and McNulty, H. 2012. The
development and evaluation of the Screening Tool for the Assessment of Malnutrition in
Paediatrics (STAMP) for use by healthcare staff. J. Hum. Nutr. Diet. 25: 311-318. doi:
10.1111/j.1365-277X.2012.01234.x.
New Brunswick Public Health. 2012. New Brunswick Public Health Nutrition Framework for
Action 2012-2016. 2012 [online]. Available from
http://www2.gnb.ca/content/dam/gnb/Departments/h-
s/pdf/en/Publications/PublicHealthNutritionFrameworkforAction.pdf [accessed 2 February 2015].
Page 25 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
26
Nicklas, T.A., and Hayes, D. 2008. Position of the American Dietetic Association: nutrition
guidance for healthy children ages 2 to 11 years. J. Am. Diet. Assoc. 108(6): 1038-44, 1046-7.
PMID:18564454.
Ontario. 2014. Technical Document: Health Promotion Division 2014. Public Health Funding
and Accountability Agreement Indicators. Toronto: Ministry of Health and Long-Term Care Health
Promotion Division, Toronto, Canada.
Ontario Ministry of Children and Youth Services. 2011. Ontario's Enhanced 18-month Well Baby
Visit [online]. Available from
http://www.children.gov.on.ca/htdocs/English/topics/earlychildhood/health/enhanced_18-
month.aspx [accessed 3 February 2015].
Paoletti, G., Bogen, D.L., and Ritchey, A.K. 2014. Severe iron-deficiency anemia still an issue in
toddlers. Clin. Pediatr. 53(14): 1352-1358. doi: 10.1177/0009922814540990.
Persaud, N., Maguire, J.L., Lebovic, G., Carsley, S., Khovratovich, M., Randall Simpson J.A., et al.
2013. Cholesterol associated with eating behaviors during early childhood. CMAJ 185(11):E531-
536. doi: 10.1503/cmaj.121834.
Randall Simpson, J.A., Keller, H.H., Rysdale, L.A., Beyers, J.E. 2008. Nutrition Screening Tool for
Every Preschooler (NutriSTEP™): validation and test-retest reliability of a parent-administered
questionnaire assessing nutrition risk of preschoolers. Eur. J. Clin. Nutr. 62: 770-780.
PMID:17554250.
Page 26 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
27
Rush, D. 1997. Nutrition screening in old people: its place in the coherent practice of preventive
health care. Ann. Rev. Nutr. 17: 101-125. PMID:9240921.
Rysdale, L., Witchell, E., Keller, H., Randall Simpson, J., Turfryer, M., McGibbon, K., et al. 2011.
Parent perceptions of NutriSTEP and nutrition risk screening in a sample of Ontario preschool
children. Infant, Child and Adolescent Nutrition 3; 197-204. PMID: 21815309.
Satia-Abouta, J., Patterson, R.E., Newhouser, M.L., and Elder, J. 2002. Dietary acculturation:
applications to nutrition research and dietetics. J. Am. Diet. Assoc. 1002(8): 1105-1118.
PMID:12171455.
Satter, E. 1990. The feeding relationship: problems and interventions. J. Pediatr. 117: S181-S189.
PMID:2199651.
Satter E. Feeding dynamics: helping children to eat well. 1995. J. Pediatr. Health Care 9: 178-184.
PMID:7629684.
Savage, J.S., Fisher, J.O., and Birch, L.L. 2007. Parental influence on eating behavior: conception
to adolescence. J. Law Med. Ethics 35: 22-34. PMID:17341215.
Secker, D.J., and Jeejeebhoy, K.N. 2007. Subjective Global Nutritional Assessment for children.
Am. J. Clin. Nutr. 85: 1083-1089. PMID:17413109.
Shields, M. 2006. Overweight and obesity among children and youth. Health Rep. 17(3), 27-42.
Page 27 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
28
Siega-Riz, A.M., Deming, D.M., Reidy, K.C., Fox, M.K., Condon, E., and Briefel, R.R. 2010.
Food consumption patterns of infants and toddlers: Where are we now? J. Am. Diet. Assoc.
110(12): S38-S51. doi: 10.1016/j.jada.2010.09.001.
Skinner, J.D., Ziegler, P., and Ponza, M. 2004. Transitions in infants’ and toddlers’ beverage
patterns. J. Am. Diet. Assoc. 104(1): S45-S50. PMID:14702017.
Streiner, D.L., and Cairney, J. 2007. What's under the ROC? An introduction to receiver operating
characteristic curves. Can. J. Psychiatry 52: 121-128. PMID:17375868.
Streiner, D., and Norman, G.R. 2008. Health measurement scales: A practical guide to their
development and use (4th ed.). Oxford University Press, New York, NY.
Thompson, J., Manore, M., and Sheeshka, J. 2014. Nutrition for preschoolers, age 4-5 years. In
Nutrition: A functional approach. 3rd Canadian ed.. Pearson Education Canada, Toronto, Canada.
pp. 540-541.
Watson-Jarvis, K., Fenton, T.S., McNeil, D., and Campbell, K. 2011. Preschool nutrition risk in
Calgary. Can. J. Diet. Pract. Res. 72: 33101-e3106. PMID:21382227.
Page 28 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Table 1. Demographic characteristics of parents/caregivers and toddlers in the
development of Toddler NutriSTEP®*.
______________________________________________________________________________
Parent/Caregiver Characteristics Phase A Phase B Phase C Phase D
Focus
Groups
Key-
Intercept
Interviews
Reliability Reliability
n = 48 n = 107 n =133 n =200
mean ± SD mean ± SD mean ± SD mean ± SD
Parent age (years) 34±12 32±7 34±9 32±6
Number of household members 4±1 NA 4±1 4±1
Number of children in household 2±1 2±1 2±1 2±1
n (%) n (%) n (%) n (%)
Gender
Male 6 (13) 6 (6) 11 (8) 15 (8)
Female 42 (87) 101 (94) 122 (92) 182 (91)
Marital Status
Married/partnered 45 (96) 84 (79) 118 (89) 168 (84)
Single/separated/divorced/widowed 2 (4) 22 (21) 15 (11) 30 (15)
Education†
< High school 6 (13) 10 (9) 11 (8) 30 (15)
Graduated high school 3 (6) 12 (11) 5 (4) 16 (8)
Some post-secondary 8 (17) 15 (14) 19 (14) 38 (19)
Page 29 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Graduated post-secondary 30 (64) 70 65) 98 )74) 116 (58)
Income (CDN) †
<$30,000 15 (34) 0 21 (16) 84 (42)
$30,000 - 59,000 10 (23) 38 (36) 20 (15) 39 (20)
$60,000 - 89,000 19 (43) 21 (20 ) 35 (26) 29 (15)
>$90,000 0 25 (33) 41 (31) 31 (16)
Mother's first language†
English 25 (54) 73 (68) 122 (92) 104 (52)
French 5 (10) 16 (15) 3 (2) 0
Other 17 (35) 18 (17) 8 (6) 96 (48)
Mother born in Canada† 28 (58) 87 (81) 121 ( 91) 100 (50)
Child Characteristics Phase A Phase B Phase C Phase D
mean ± SD mean ± SD mean ± SD mean ± SD
Age (months) 27.2±5.6 25.8±4.8 25.1±5.1 25.4±5.1
Body mass index‡
(for toddlers ≥2 years)
n (%)
Normal 114 (57)
<3rd percentile 0
>85th - 97th %ile 58 (29)
> 97th %ile 28 (14)
Gender n (%) n (%) n (%) n (%)
Male 25 (52) 61 (57) 68 (51) 99 (50)
Page 30 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Female 23 (48) 46 (43) 64 (48 101 (51)
Age (months)
18 - 23 14 (29) 38 (36) 60 (45) 77 (39)
24-29 12 (25) 42 (39) 61 (38) 67 (34)
30 - 35 22 (46) 27 (25) 26 (20) 56 (28)
First language
English 23 (48) 80 (75) 122 (92) 108 (54)
French 3 (6) 9 (8) 1 (1) 0
Other 15 (31) 9 (8) 9 (7) 92 (46)
Born in Canada 42 (88) 105 (98) 123 (92) 169 (85)
______________________________________________________________________________
* Percentages may not add up to 100% due to rounding errors and/or missing data
† Significantly different (p<0.05) between Phases C & D
‡ BMI based on 2010 WHO growth curves adapted for Canada (Dietitians of Canada 2014)
Page 31 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Table 2. Proportion of respondents with individual Toddler NutriSTEP® items indicating
risk for reliability (Phase C) and validity (Phase D) Studies and reliability statistics.
______________________________________________________________________________
Phase C Phase C Phase C PhaseD
NutriSTEP© Item Reliability Kappa*
statistic
Wilcoxcon
Signed
Rank
Validity
(n=133) (n=200)
% %
Grains
>3 x per daya 53 0.39 -2.161† 36
≤3 x per dayb 47 64
Milk/Substitute Products
≥3 x per daya 88 0.70 -2.185† 76
≤2 x per dayb 12 24
Vegetables & Fruit
≥3 x per daya 77 0.68 -0.448 62
≤2 x per dayb 23 38
Meat & Alternatives
≥2 x per daya 74 0.44 0.00 61
≤1 per dayb 26 39
Fast Food
≤1 x per weeka 71 0.63 -0.622 80
Page 32 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
≥2 x per weekb 29 20
Flavoured Beverages
≤1 x per daya 75 0.67 -0.011 69
≥2 x per dayb 52 31
Food is Expensive
Never to rarelya 73 0.74 -0.698 55
Sometimes to most of the timeb 27 45
Chewing/swallowing difficulty
Never to rarelya 95 0.85 -0.962 90
Sometimes to alwaysb 5 10
Feeding his/her self
Most of the time to alwaysa 90 n/a‡ -0.333 67
Sometimes to neverb 10 33
Drinks from a baby bottle
with a nipple
Never to rarelya 71 0.89 -0.447 63
Sometimes to alwaysb 29 37
Hungry at mealtimes
Sometimes to alwaysa 27 0.49 -0.836 56
Never to rarelyb 73 44
Eating frequency
≥3-≤6 x per daya 89 0.46 -1.043 77
≤2 x per day or >6 x per dayb 11 23
Page 33 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Child controls amount
consumed
Always/most of the timea 89 0.53 -2.532 69
Sometimes to neverb 11 31
Watches television at meals
Never to sometimesa 86 0.70 -0.870 75
Most of the time to alwaysb 14 25
Sedentary activity during day
≤1 hour per daya 64 36 -1.253 67
≥2 hours per dayb 36 33
Growth adequate§
Yesa 90 10 -1.134 76
Nob 10 24
Weight§
Appropriatea 81 0.82 -1.098 65
Too much/littleb 19 35
_____________________________________________________________________________________________________________________
a Indicates no risk
b Indicates risk
* All kappa statistics were significant at p<0.001. Kappa statistics are: fair (≥0.4); moderate
(≥0.6); excellent (≥0.8)
† Significant Wilcoxcon test at p=0.05
Page 34 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
‡Unable to calculate a kappa statistic because risk/no risk was consistent across administrations
§ Based on parental perception of adequate growth/weight
Page 35 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Table 3. Cumulative prevalence of participants at/above Toddler NutriSTEP®
score and
comparison of sensitivity (Se) and specificity (Sp) using the moderate and high risk
dietitian cut-points as a comparison for individual Toddler NutriSTEP®
score values.
______________________________________________________________________________
Toddler
NutriSTEP®
Score
Cumulative
Prevalence of
Participants at/above
Toddler NutriSTEP®
Score
Moderate Risk ≥5 High Risk ≥8
Phase D Phase D
Validation Study Validation Study
Reliability
Study*
Validation
Study
Se Sp Se Sp
n=133 n=200
(%) (%) (%) (%) (%) (%)
3 97 99.5 1 100 1 100
4 96.2 98.5 3 100 2 100
5 94 97 6 100 4 100
6 91.7 96.5 7 100 4 100
7 88.7 94.5 11 100 6 100
8 84.2 93 14 100 8 100
Page 36 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
9 78.1 92 16 100 9 100
10 72.9 90 20 100 12 100
11 66.2 84 32 100 19 100
12 57.9 78 42 98 26 100
13 48.1 72.5 50 96 32 100
14 40.6 70.5 54 94 35 97
15 36.1 65 56 97 41 97
16 30.8 59.5 63 84 45 90
17 28.6 54 71 81 52 90
18 24.8 50 73 75 56 87
19 19.5 44.5 77 68 61 80
20 15.8 40 82 65 66 80
21 37 86 61 71 80
22 14.3 36 88 61 72 80
23 12.8 34 90 59 74 80
24 9 31 92 56 76 77
25 6 27 95 51 80 70
26 24 95 43 83 63
27 22 95 40 85 63
28 5.3 20.5 95 32 88 53
29 4.5 15 98 29 91 50
30 3 21.5 98 22 93 40
31 2.3 20.5 99 20 94 37
Page 37 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
32 9 100 18 96 37
33 0 7.5 100 15 97 34
34 0 6.5 100 13 98 30
35 0 6 100 12 98 27
36 0 5 100 10 98 20
37 0 3.5 100 7 98 13
38 0 1.5 100 3 99 6
39 0 0.5 100 2 100 6
43 0 0.5 100 1 100 3
46 0 0.5 100 0 100 0
____________________________________________________________________________
* Values are for the first administration of the questionnaire
Page 38 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Table 4. Construct validation statistics for Toddler NutriSTEP® .
______________________________________________________________________________
Construct Toddler
NutriSTEP®
score
Overall Model
(mean±SD)
Household Income
<$30,000 (n=78) 23±8
$30,000-$59,999 (n=39 19±8 F=27.68, p<0.001
>$60,000 (n=60) 14±6c
Level of Education
< Graduated high school (n=29) 24±7a
Graduated high school (n=15) 21±6b F=3.96, p=0.009
Graduated post-secondary (n=114) 18±9c
Immigration Status (Mother)
Born in Canada (n=96) 15±6a t=-7.67, p<0.001
Immigrated to Canada (n=95) 24±8b
Immigration Status (Father)
Born in Canada (n=93 15±6a t=-7.00, p<0.001
Immigrated to Canada (n=85) 23±9b
Immigration Status (both parents)
Both parents born in Canada (n=85) 15±6a
One parent born in Canada (n=16) 15±6a F=35.42, p<0.001
Neither parent born in Canada (n=77) 24±8b
_____________________________________________________________________________________________________________________
Page 39 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
a,b,c Values in the same column with different superscripts are significantly different
Page 40 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
1
Figure Captions
Figure 1. Phases in the development of Toddler NutriSTEP®
Figure 2. Receiver operator characteristic (ROC) curves to Toddler NutriSTEP®
score relative to
nutritional risk rating (risk ≥8 on a scale of 1 to 10) by a registered dietitian
Page 41 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Figure 1. Phases in the Development of Toddler NutriSTEP® 204x164mm (300 x 300 DPI)
Page 42 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism
Draft
Figure 2. Receiver operator characteristic (ROC) curves to Toddler NutriSTEP® score relative to nutritional risk rating (risk ≥ 8 on a scale of 1 to 10) by a registered dietitian
161x158mm (300 x 300 DPI)
Page 43 of 43
https://mc06.manuscriptcentral.com/apnm-pubs
Applied Physiology, Nutrition, and Metabolism