Final Item Inclusion List - University of Kentucky · Renae Mantooth (B.A.), & Ellen L. Usher...

A HOW -TO GUIDE !

+ Ensure another instrument does not exist

+ Select appropriate survey response format

+ Identify limits of the construct

+ Base items on previous instruments or standards found in literature+ Consider multidimensionality of the construct

+ Consider clarity and level of difficulty (Bloom’s Taxonomy; Bandura, 2006)

+ Build a large initial pool of items

ConsultationWith

Experts

Final ItemInclusion List

Document every stage,every item removed, and justify each step.

+ Are items grounded in theory?

+ Is the population of interest represented appropriately? + Transcribe interviews

to identify key issues.

+ Opportunity to use verbal probing to determine comprehension, retrieval, decision, and response processes of participants in relation to each item.

+ Internal Structure

+ Observed Correlation Analyses+ Latent Variable Analyses

Evidence for Test Content Evidence for Response Processes

Evidence for Internal Structure

Abigail M. A. Love (M.S.), Christen M. Dillon (M.M.),

Renae Mantooth (B.A.), & Ellen L. Usher (PhD)

+ Item Removal Criteria

(Bensen & Clark, 1982; DeVellis, 2012)

+ Create items to reflect underlying latent construct(s)

CognitiveInterviews

w/ SamplePopulation

Don’t skip steps!

(AERA, APA, & NCME, 2014)

(AERA, APA, & NCME, 2014) (AERA, APA, & NCME, 2014)

Evidence for Test Content(AERA, APA, & NCME, 2014)

Writing socially desirable items should be avoided.

Items should be age appropriate and differentiated.

Items should not be redundant, ambiguous, vague, or carry multiple meanings.

Relations to Other Variables(AERA, APA, & NCME, 2014)

p20motivationlab.org/research