The effect of differential item functioning in anchor items on
population invariance of equating Anne Corinne Huggins University
of Florida Slide 2 Introduction Population invariance of equating
Test score level Differential item functioning Item level Need to
connect two facets of invariance Certain types of DIF might cause
equating dependence across forms of a test. Due to (1)a
differential difficulty, particularly for anchor items (2)a second
factor(s) (3)Item parameter drift Slide 3 IRT true score equating A
three-step process: Specify a true score on one form Find the
corresponding IRT ability estimate Find the IRT ability estimates
corresponding true score on the other form Slide 4 Place the latent
ability scores on each test on the same scale We can use the
probability formula to get the relations. Slide 5 Scaling
Mean-sigma Mean-mean 5 Here, measurement errors are ignored. If a
and b were biased, A and B will be biased. Q: How to estimate A
under Rasch model? Slide 6 The Stocking and Lord method Minimize
the difference in test characteristic curves (TCCs) for anchor
items The Haebara method Minimize the difference in item
characteristics curves (ICCs) 6 Slide 7 Equating invariance Compare
the whole group to multiple subgroups Group to overall (compare one
subgroup with the whole group) 7 Sigma Q is the unconditional SD of
scores in population. Slide 8 The factors that might influence
population invariance of IRT true score equating Differential
anchor form DIF The magnitude of DIF in anchor items Number of DIF
anchor items Direction of DIF in anchor items Ability differences
between subpopulations 8 Slide 9 Method Uniform DIF R, BILOG-MG,
SPSS Design Nonequivalent groups with anchor test (NEAT) design Two
forms contain 50 dichotomous items, respectively, with 20% of
common items Ability ~ N (0,1) 4500 participants (3000 in the
references group) 3PL model 100 replications 9 Slide 10
Manipulations on DIF in anchor items Magnitude (.3,.6,.9) % of DIF
items (20%, 49%, 60%) Directionality of DIF Mean ability
differences Differential anchor form DIF Null conditions No DIF in
anchor items and no mean ability differences No DIF in anchor items
but had mean ability differences 10 Slide 11 ResultsNo DIF (RMSD
(x) ) 11 Slide 12 No DIF, Equal mean abilities (RSD (x) ) 12 Slide
13 No DIF, Unequal mean abilities (RSD (x) ) 13 Slide 14 14 Why
didnt DIF show the impact here? Slide 15 15 Slide 16 16 Slide 17 17
Slide 18 18 Slide 19 19 Slide 20 20 Slide 21 21 Slide 22 22 Slide
23 23 Slide 24 24 Slide 25 Conclusions Differential anchor form DIF
occurs, the magnitude of equating dependence increases. IPD occurs
for different subpopulations. When differential anchor from DIF
occurs: Direction of DIF in anchor items shows the largest effect
on equating dependence. The magnitude of DIF in anchor items Number
of DIF anchor items Ability differences between subpopulations 25
Slide 26 Comments It is a good writing sample. Errors: P.9, since
the study manipulated the direction of DIF, the statement of
magnitude of DIF should be revised properly. The author intended to
design the sample size as the true subpopulations in the US, but
the actual sample sizes did not reflect its intends. Slide 27 When
DIF or item parameter drift occurs in a specific anchor item, this
item should not be used as an anchor. We usually do not talk about
DIF in anchor items because DIF detections should be done before
any equating or linking. When all anchor items are DIF items, the
results should be very similar with the ones without DIF anchor
items. Slide 28 Future studies Sample size differences across score
levels Test length, anchor length, ratio of anchor to test length,
mixed item formats 28
