View
214
Download
0
Category
Preview:
Citation preview
3 MASSACHUSETTS LNIV AMHERST LAB OF PSYCHOMETRIC AND -- ETC F/G 12/1BAYESIAN ESTIMATION IN THE ONE-PARA1ETER LATENT TRAIT MODEL. (U)MAR 80 H SWAMINATHAN, J A GIFFORD N0001- -79C0039
UNCLASSIFIED LR-106N L
'EhlhlhElhEEllIIIIIIIIIIIIIIIfll..
llllllll~ENl
IIIIIII
COD BAYESIAN ESTIMATION IN THE ONE-PARAMETER
LATENT TRAIT MODEL
0
wtHAR IHARAN SWA I NATHAN
AND
JANICE A. GIFFORD
Research Report 80-1March 1980
Laboratory of Psychometric andEvaluative Research (LR 106)
School of EducationUniversity of Massachusetts
Amherst, MA 01003
Prepared under contract No. NOOO14-79-C-0039, NR150-427with the personnel and Training Research Programs
Psychological Sciences DivisionOffice of Naval Research
Approved for public release; distribution unlimited.Reproduction in whole or in part is permitted for
any.purpose of the United States Government.L84
80 *6 24 008
Unclassified'9ECUAITY CLASSIFICATION OF THIS PAGE (tm Da d) Data900"00
REPOR DOCUENTAION PGEEAD INSTRUCTIONSREPORT DOCUMENTATION PAGE BEFORE COMPLETING FORM
I. REPORT NUMBER 12. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER
Technical Report 80-1=. " % ~~~TITLE rf,d S..h,,I. ..... *I ,T~ r r .. _.=--'EE
BAYESIAXSTIHATION IN THE O-ZAA ER Trp--
!.ATENTjIT MOECID S ,. RFORMING ORG. REPORT NUMBER
Lab. Report 106
h a.n a*., .~J i e A / f~ro rd",-tA C O N T R A C T O R G R A N T N U M ' E ()nathan J ce ifford4-79-
9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT, TASK
Laboratory of Psychometric and Ev/luative Researih AREA I WORK UNIT NUMBERS
S~hgl Lnf ... /U.ixe. -ty of/assachusettwA t'NR150-427-- e r s , 4 A 0 1 0 0 3 I
11. CONTROLLING OFFICE NAME AND ADDRESS
Personnel and Training Research Programs MarOffice of Naval Research (Code 458) 2. iS
Arlington, VA 22217 3614. MONITORING AGENCY NAME A ADDRESS(if dllleet from Controlllng Office) IS. SECURITY CL*ISS- (ol this report)
UnclassifiedIS&. DECLASSIFICATION/DOWNGRADING
SCHEDULE
16. DISTRIBUTION STATEMENT (of
Approved for public release; distribution unlimited.
17. DISTRIBUTION STATEMENT (o the abetrct entered In Block 20. It different irom Repot)
'S.1+ L-Ij ~R~-A--
IS. SUPPLEMENTARYeWS'
This research was jointly supported by the Air Force Human Resources
Laboratory and by the Office of Naval Research.
IS. KEY WORDS (Continue an reverse aide if neseewar od Identify by block numbef)
latent trait theoryBayesain estimation
20. ABSTRACT (Continue on reveso aide If neceaar and identlfy by Nock mambe)
,-When several parameters are to be estimated simultaneously, and when both
structural and incidental parameters have to be estimated, a Bayesian solutionto the estimation problem may be appropriate. This is the case in latent traitmodels, where the "structural" parameters are item parameters, while the"incidental parameters" are ability parameters since these increase withoutbound as the numbers of examinees is increased to provide stable estimates ofthe item parameters. Bayesian estimates for the parameters in the one-paramete
DD I,=1 1473 EDITION OF I NOV S IS OBSOLETEUS/ 0102-LF.014401 Unclassified
SCITVCL AIFICATION OF THIS P AGE
UnclassifiedSCURITY CLASSIICATION OF THIS PASS 1M& SnO
latent trait model were obtained for two cases) -
/1 ,Conditionalestiuation of ability (for those situations when itemsare previously calibrated), and.
(2 ,doint estimation of item and ability parameters.
For each of the two cases, a simulation study was carried out to studythe efficacy of the two Bayesian procedures described and to compare theBayesian estimates with the comparable maximum likelihood estimates. <The..
Bayesian and maximum likelihood estimates were compared with respect to:(a).the mean value of the estimates, as compared with the mean values of thetrue values; (b) the mean squared error difference between true values andestimated values; and (c) the regression of the true value on the estimatedvalue. Overall, the results favored the Bayesian estimates; the means ofthe estimates are closer to the means of the true values; the slopes andintercepts are in general closer to one and zero respectively; and the meansquare deviations are dramatically smaller (in some cases, one-tenth thesize of those for ML estimates).
UnclassifiedSECURITY CLASIICATION OF ThIS PAGSI(Whbe Diee RfmQ j
TABLE OF CONTENTS
Page
INTRODUCTION . 1
Bayesian Procedures .. ..................... 3
Bayesian Estimation in the One-ParameterLogistic Model .. ................... .... 7
The Model. .. ........................ 7Conditional Estimation of Ability...............8Joint Estimation of Item and Ability Parameters. .. ..... 13
Large Sample Properties of thePosterior Distribution. .. ................. 18
COMPARISON STUDIES .. .. ................... 23
DISCUSSION .. .. ........................ 31
References .. .. ........................ 34
iT
L:nnG
Bayesian Estimation in the One-ParameterLatent Trait Model1'2
INTRODUCTION
In recent years there has been considerable interest among measurement
theorists and practitioners in latent trait theory since it offers the
potential for improving educational and psychological measurement practices.
However, before latent trait theory can be successfully applied to solve
existing measurement problems, the problem of estimating parameters in
latent trait models has to be addressed.
The literature in latent trait theory abounds with procedures for the
estimation of parameters. The estimation procedures that have been devel-
oped over the past thirty years range from heuristic procedures such as
those given by Urry (1974) and Jensema (1976) to conditional as well as
unconditional maximum likelihood procedures (Andersen, 1970, 1972, 1973a,
1973b; Bock, 1972; Lord, 1968, 1974; Samejima, 1969, 1972; Wright &
Panchapakesan, 1969; Wright & Douglas, 1977). With the exception of the
"conditional" maximum likelihood procedure provided by Andersen (1970)
for the one-parameter model, the maximum likelihood estimators of the
parameters in the latent trait models are less than optional as a result
1The research reported here was performed pursuant to Grant No.N0014-79-C-0039 with the Office of Naval Research and to Grant No.FQ 7624-79-0014 with the Air Force Human Resources Laboratory. Theopinions expressed here, however, do not reflect the positions or poli-cies of these agencies.
2The author is grateful to the encouragement and support provided
by Dr. Malcolm Res of the Air Force Human Resources Laboratory, and toDr. Charles Davis of the Office of Naval Research.
__ Z.a
-2-
of the problem of estimating "structural parameters" in the presence of
"incidental parameters" (Andersen, 1970; Zellner, 1971, pp. 114-154).
The "structural parameters" in latent trait models are the item parameters
while the "incidental parameters" are the ability parameters since these
increase without bound as the number of examinees is increased to pro-
vide stable estimates of the parameters. Furthermore, as Novick, Lewis,
and Jackson (1973) have remarked, "in the estimation of many parameters
some, by chance, can be expected to be substantially overestimated and the
others substantially underestimated."
When several parameters have to be estimated simultaneously, and
when, as in the present case, both structural and incidental parameters
have to be estimated, a Bayesian solution to the estimation problem may
be appropriate (Zellner, 1971, pp. 114-119). This is particularly true
if prior information orbelief about the parameters is available, since
in this case, the incorporation of this information will certainly increase
the "accuracy" or the meaningfulness of the estimates. An example of
this was encountered by Lord (1968), where in order to prevent estimates of
the item discrimination parameter from drifting out of bounds, it was
necessary to impose limits on the range of values the parameter could
take. Although the estimation procedure employed by Lord (1968) was not
Bayesian, this illustrates the role of prior information in obtaining
meaningful estimates. A further argument that can be advanced in
favor of a Bayesian approach is that the logic of the Bayesian infer-
ential procedure is more appealing than the classical, sampling theoretic,
inferential procedure. As Zellner (1971, p. 362) has pointed out,
"...there is no need to justify inference procedures in terms of their
-3-
behavior in repeated, as yet nobez'ved,1 samples as is usually done in the
sampling theory approach." Consequently, it is possible to make proba-
bilistic statements about the parameters themselves, based on the infor-
mation that is available.
Bayesian Procedures
It may be instructive to review briefly the Bayesian estimation
procedure. Let p(y, 8) denote the joint probability density function (pdf)
for a random observation vector y and a parameter vector 0, also random.
Then,
p(Y-, e) - p(Yjl) p( 8)
p (..Y-) p (Y)where
p(eIY) - p(_) p(ZI)/p(y)or,
[(1] P(_) P(D) p(Y-1o
since p(y) # 0 is a constant. Equation [1] is the essence of Bayes'
Theorem and is of primary importance in the estimation of parameters
and for drawing inferences concerning the parameters. The probability
density function p(61y) is the posterior pdf for the parameter vector e,
given the sample information or data, and p(S) is the prior pdf for thevector e. The quantity p(iI.) is a proper pdf as long as is a random
variable. However, the moment the vector y is realized, p(yje) ceases
IThe italics have been provided by the authors.
-4-
to have the interpretation as a pdf. In this case, p(lj_) is strictly a
mathematical function of 6, well known as the likeZihood function.
Since the notation p(y]8) can be mistaken for a pdf, the likelihood
function is often written as L(QJ 8), and sometimes, to emphasize the
fact that it is a function of 6, as L(81I). Thus, the expression given
in [1] can be written as
[2] p(.1I) -L(B1) pC()
It is interesting to note that if p(e) is assumed to be a constant,
i.e., the prior belief about 8 is summarized via a uniform distribution,
the posterior pdf of 8 is proportional to the likelihood function. In a
sense, this interpretation constitutes a Bayesian justification of
maximum likelihood principle.
Once the prior belief about the parameter e is specified, the joint
posterior pdf of the vector 8 given the data can be readily obtained. The
posterior pdf of 8 contains all the information necessary for drawing
inferences concerning 8 (Jointly or individually) and for obtaining esti-
mtes of 6 once a "loss function" is prescribed. For instance, if a squared-
error loss function is deemed appropriate, then the mean of the posterior
pdf of 6 can be taken as the estimator of e. On the other hand, if a
zero-one loss function is appropriate, then, the mode of the posterior
pdf of 8 is the estimator of 8. Similarly, for the absolute deviation
loss function, the median of the posterior pdf of 6 is the appropriate
estimator.
The Bayesian procedure described above has been successfully applied
in a variety of situations. For a sampling of these applications the
reader is referred to Novick and Jackson (1974), and Zellner (1971).
~-5-
However, Bayesian methods have found only a limited application in the
area of latent trait theory. Birnbaum (1969) obtained Bayes estimates of
the ability parameter in the one- and two-parameter logistic models under
the assumption that the item parameters were known. He chose, for mathe-
matical tractability, the prior pdf of 0i, the ability of the ith
examinee, to be the logistic density function, i.e.
p(Oi) - exp (-DOi)/[l+exp(-DOi)]2
where D=1.7 is a scaling factor. Owen (1975), in applying the latent
trait model in an adaptive testing context, obtained Bayes estimates of
ability, 01, under the assumption that the prior pdf of ei was normal
with mean, zero, and variance, unity.
The Bayesian procedure suggested by Birnbaum (1969) and Owen (1975)
require rather exact specification of prior belief.1 An alternative and
a more powerful procedure has been suggested by Lindley (1971). He has
shown that if tte information that is available can be considered exchange-
able, then a hierarchical Bayesian model can be effectively employed for
the estimation of parameters.
In order to illustrate the hierarchical model, let us consider
the problem of estimating, say, the ability 61 of an individual (i-1, ... , N).
If it can be assumed, apriori, that exchangeability holds, i.e., the
information about ei is no different from the information about any other
ej, observed or yet to be observed, then, ei can be assumed to be a random
sample from some distribution, p(e). For convenience, if p(e) is taken
IMeredith and Kearnes (1973) and Sanathanan and Blumenthal (1978)
have obtained empirical Bayes estimators of the ability parameters forthe one-parameter model. In these procedures the prior pdf is estimated
k from the data.
I L W l ,. . . . a . e"-,. .... .
-6-
to be normal with mean U and variance a2, then this would constitute
specification of the first stage of the hierarchical model. Since v
and Y2 are unknown, specifying prior beliefs on these "hyperparameters"
would constitute the second stage of the hierarchical model. Usually,
the hyperparameter distributions are specified in such a way that they
depend upon constants which can be determined from the prior belief the
investigator has about the parameters, and hence the hierarchical model
terminates at the second stage. With this two stage model, it is possible
to estimate 0i (i=l, ..., N) without any reference to the nuisance param-
eters, p and a2 .
Novick (1971) has described this hierarchical model as an analog of
the empirical Bayes procedure advocated by Robbins (1955) and the simul-
taneous estimation procedure provided by Stein (1962). Furthermore, as
Novick, Lewis, and Jackson (1973) have pointed out, this procedure not
only employs the direct information gained through the observation of
an individual, but also the collateral information contained in observations
from other individuals. They further note that, "In effect, this collateral
information is used to provide 'prior' information for the estimation....
Thus to some extent, the problem of selecting prior distributions for
Bayesian analyses is neutralized, and this is effected from a strictly
Bayesian approach."
The hierarchical Bayesian model has been successfully employed by
Lindley and Smith (1972), Novick et al. (1973), and Zellner (1971), to
name a few. However, this approach has not been employed for estimating
parameters in latent trait models. The purpose of this paper, hence, is
to provide a Bayesian estimation procedure, in the sense of Lindley, for
estimating parameters in the one-parameter latent trait model.
-7-
Bayesian Estimation in the One-Parameter Logistic Model
The Model
Let Xij denote a random variable that represents the binary response
of an examinee i (i-1, ..., N) on item J (J-l, ..., n). If the examinee
responds correctly to the itemXij-l, while for an incorrect response,
Xij=O. We assume that the complete latent space is unidimensional, and
that the probability, P[Xij-l], that an individual with ability parameter
ei will correctly respond to an item with difficulty parameter, bj, is
given by the logistic model,
[3] P[Xijlli] ff= exp(6i-bj)/{l+exp(Oi-bj)}.
On the other hand, the probability that the individual will respond
incorrectly is given by
- [4] P[Xij=Oit] = i - P[Xij-llei
= i/{l+exp(Bi-bi)}
The probabilities given in Equations [3] and [4] can be combined to yield
[5] P[XiJ - xijj L] - exp{xij(ei-bj)}/{l+exp(Gi-bj)}
where xii-1 for a correct response and xij=0 for an incorrect response.
The above model, since it depends only on one item parameter,
difficulty, is commonly known as the Rasch model or the one-parameter
logistic model. For a detailed description of this model and its prop-
erties, the reader is referred to Wright (1977).
-8-
Conditional Estimation of Ability
In some situations it may be of interest to estimate the ability Oi
of an examinee who takes a test which has been calibrated, i.e., the
difficulty parameters are known. Moreover, since the problem of esti-
mating ability when the item parameters are known is simpler to deal
with and provides an illustration of the basic ideas involved, this case
will be dealt with in detail first.
The model given by Equation [5], should in the strict sense be
expressed as
[6] P[Xij = xijlei,bjl = exp{xij(ei-bj)}/{1+exp(ei-bj)}
Although there are several ways to write the model, the expression given
by [6] is the most convenient for the present situation.
It follows, from the principle of local independence, that the
joint probability of responses of the N examinees on n items is given by
[7] P[XlfXl1,Xl2=x12,... ,Xij=xij,... ,XNn=xNnl8,e 2 , .... 9N;bl,b 2 ... ,bn]
N n=fil 11 exp{xij(6i-bj)}/{l+exp(ei-bj)}
i=l jil
Once the responses of the N examinees on the n itemsare observed, the
above expression ceases to have the probability interpretation and becomes
the likelihood function, L(X - xle, _). Upon simplification,
[8] L(X=x,b__) = exp{J I xij(6i-bj)}/[ E{(l+e ' 1 -hI)}i j
W exp{i riei - I qjbj}/H n{(l+exp(ei-bj)}i j
where ri = I x i j and qj = xii. Since the item parameters are
. ..
V
-9-
known constants, the likelihood function is strictly a function of 6
and, hence, can be expressed as
[9] L(xl, b) - exp{j ri~i}/ f{l+exp(Oi-bj))
i
Returning to Equation [1], we see that in order to obtain the
posterior density function of 8 given the observations and the item
parameters, it is necessary to specify the prior distribution of 6. To
this end, in the first stage of the hierarchical model, we assume that,
apriori, the ability parameters, 61, are independently and identically
normally distributed, i.e.,
S[101 eiju,o n p,.
The assumption that the thetas are independently and identically distri-
buted follows from the assumption of exchangeable prior information about
the thetas. The assumption of normality also appears to be reasonable
and has been made by numerous authors, e.g., Lord and Novick (1968).
In order to complete the hierarchical Bayesian model, we have to
specify prior distributions for p and 0. This is the second stage. At
this level, we assume that, apriori, p and * are independently distributed,
and that p has the uniform distribution. Thus,
[1h p6(,) - p(O).
The uniform distribution is not a proper distribution, although this
choice can be justified to some extent (Zellner, 1971, pp. 41-43). It
may, however, be more appropriate to specify a "non-diffuse" prior and
this possibility will be explored further in a later paper.
-10-
It now remains to specify the form of p(f). Since * is thevariance of 01, f can be assumed to have the inverse chi-square, X-2
distribution, i.e.,
[121 p(#IV, S2) 2 exp(-VS2/2f).
The quantities v and s2 are parameters of the inverse chi-square dis-
tribution, and have to be specified apriori. The inverse chi-square
distribution can be expressed in different ways. Novick and Jackson
(1974) prefer the form
p(4 V, X) f 2 exp(-X/20).
For this form, the mean of the distribution is X/(v-2) and the mode is
A/(v+2). For the form given by Equation [12] the mean is s2v/(v-2) and
the mode is s2v/(v+2), with both mean and mode approaching s2 as v
increases. These two forms are clearly equivalent, but the form given
by Equation [12] is employed in the sequel because it provides a direct
interpretation of the parameter v and s2 . The quantity s2 thus represents
the investigator's belief about the "typical" value of the parameter
while v represents his/her degree of confidence.
The joint posterior distribution of 0' - [ 1 , e2, ..., ON] given
b and the item responses is given by
[13] p(61bL, _) - L(xj, b_) p(Ojjj,¢)p(u, ).
The likelihood function L(xj2,b) is given by Equation [9], p(p,*) by
Equation [12], and
-11-
N 1[14] p(el,*) - in1 - exp{- (ei-U)2/01
Nexp{- (ei-v) 2/20}
i-i
Combining these expressions, we have,
[15] S(bx,o,v, s2) c [exp{. ri}/IT II {1+exp(ei-bj)1]i it
[ 'Z exp {- e C( i- 20}][O exp(-vs2/20)]
i
The above expression depends upon the "nuisance" parameters p and 0 and
hence these have to be integrated out. Since .(ei-p)2 = I(ete.)2 + N(e.-U)2,
and
_CD exp -{N(G.-)2/20}d 0 if
integration with respect to v yields
[16] p(_ebx,0,vs 2) L(xJ_,b_) *-(N+V+l)/2 exp[-{vs2+.(6i_.) 2}/20]
Noting that
f m exp(-k/O)do k
and integrating with respect to *, we obtain
[17] p(_b,x,v,s2) - L(x0,_b) (vs2 + (6-e.)2}-(N+v-l)/2
ii
(18] = [exp{ reil l{l+exp(ei-bj)}]i i
.{*v2 + I (e )2 -(N+--e)/2
i i-f~s -
-12-
The Joint posterior modes are obtained by differentiating log
p(elk,xE) with respect to e, setting these derivatives equal to zero,
and solving the resulting equations:
[19] P pj -= ei y (i=l, .. ,N)
where
ij exp(Oi-bj)/{l+exp(ei-bj)}
andN
a2 [ vs8+ (6i6.)2 }/(V+N-l)
Since this system of equations is non-linear, numerical procedures have
to be employed. The Newton-Raphson iterative procedure is ideally suited
for this situation. Let
n[20] W) P + (0 _e )/ar2
-ri
Then
n[21] f'(e6 P (1-P )+ {02(1_ 1) -2(0 -e)/(v+N-l)}/(aY2)2.
i j=l ij i
if 8'6 is the value of e at the kth iteration, then 6(kl is given byi i i
[22] e (k+l)= e (k) _ f( 8 (k) )/fI( 6 (k))
i i i i
with e(o) the starting value being given by (Wright & Douglas, 1977),
[23] 6(o) -b + {l+s2/2.89 }log (ri/n-ri)ib
where
-~ ~ b/n an s - Cjb) 2/(n-1)
bil n n ab IbJb
Zr
-13-
Although the iterative scheme given in [22] is for estimating the
ability e for each individual, in reality, only the ability correspond-
ing to each raw score r (r-1, ... , n-l) need be estimated. The ability
corresponding to raw score r=O and r-n cannot be estimated by virtue of
[23]. Hence, individuals who obtain perfect score or zero score are
eliminated from the analysis. It should also be pointed out the Newton-
Raphson scheme given above is not the vector version of the procedure
since for this procedure the matrix of derivatives {af/9 1aO3} has to
be computed and inverted. The procedure described here worked sufficiently
well, converging in as few as three to four iterations.
Joint Estimation of Item and Ability Parameters
The case considered above, where the item parameters were assumed
to be known, provides the necessary background for the Bayesian estima-
tion procedure. However, this situation may not be realistic and,
hence, it is necessary to develop a procedure for the joint estimation
of the item and ability parameters.
We proceed in the manner indicated for the case of known item
parameters. Hence, in addision to making the assumptions about the
ability parameters, we have to make assumptions regarding the item param-
eters. Again, as in the previous case, we specify prior beliefs about
the parameters in two stages. In the first stage, for the model given
in [5], we assume:
[24a] eili e *o ' N(1Iee), (il, ... , N)
[24b] b j1"bO b NO (b, ob). (J-1,., n)
-14-
In addition, we assume that, apriori, ei and b are independent, ek and
61 (k#1) are independent, and bk and bt are independent.
As for the ability parameters, the specification of prior belief
about b seems reasonable, especially if an item bank is available. This
assumption has been made by several authors (Lord & Novick, 1968; Wright
& Douglas, 1977). Furthermore, as a result of the hierarchical Bayesian
model, departures from this assumption appear to have a negligible
effect on the estimates of b .
For the second stage, we assume that
[25a] p(pS,¢oe) p()
-(ve/ 2+1)c exp(-s 2 v /20 )
and
[25b] P(vb,Ob) POO
-(\%/2+1)
exp(-SVb/20b)"
We have thus assumed that, apriori, the hyperparameters are independent,
and that the prior information about the parameters, 11, and 1b' is
"vague".
The joint posterior pdf of 6, and b, is given by
[26] p(e,bx,g, €,Ab,$b,Ve,s 2 , s2)
N nL(O,bjx){ H p(61) H p(b )}P(O0) p(¢b)
i-l j=l
w1' re L(6,blx) is the likelihood function given by [8]. Now
2 -(N+v e+2)2 1 2/20 )2/20[271 R T P(e t)} p(00) =€exp(-VeS,/2 e) exp{-(e t-Ve)22e)
...
-15-
Upon integrating with respect to and u., we have, from [17]
C N(28] f f { H p(i)1 p(eo)dle doe-" 0 j=1
Nc [V S2 + (e e.)-(N+v-l)/2.
Similarly,
n
[29] fO f0 {j- p(bj)} p(Ob)db debn
[vbS 2+ I (b - b.)2]-(n+vb-l)/2
J-
Combining [26], [28] and [29], we obtain the joint posterior density of
e and b:
[301 p(8,blx,vgSV , Vb,S2)
N N[{exp( I rg6i)){vgS + I (e -8)2)-(N+ve-l)/2]
i-l ii .
n n(exp(- I q b )}{fv bS2 + E (bj-b.) -(nv-I l,,
N nII I {i + exP(ei-b)}]Mi-l j=1
The quantity given as L(8,bx),
exp(Iri~i) exp(lqjbj)/ f{1+exp(ei-bj)} = H n exp(Oi-bj)/{l+exp(6i-bj)} ,
and, hence, is bounded. In fact,
ThefL(_e,bJ) i 1 th.
i Therefore, it follows that
1I
-16-
N -(+ve-1)/ 2f I Ip( _, ) ...) I dedb < f [V s82 + N (e .e )2 ] de_
ii-2 n -(n+vb-)/2
"f[ s a + I (b -b)21 db1" I
The integralson the right of the inequality clearly exist since the kernels
are those of multivariate t densities. Hence, the posterior pdf,
p(,b , 6e 2vb,Sb), is a proper pdf although the normalizing constant
cannot be evaluated explicitly.
The joint posterior modes may be taken as estimatesof e i and b
(i-1, ..., N; J-l, ..., n). These are obtained by setting equal to zero
the derivatives of log p(8,b ...), and solving the resulting equations:
n[31] Pij ri - (i-e.)/C (i-1 ..., N),
j=1
N[321 P qj + (b -b.)/a2 (J-1, ... , n),
i=l b
where
Pij = exp(ei-bj)/{l + exp(ei-bj)}
ri M [ xij
Iq M x i i
02 M {V s2 + [ (e-0)21/(ve+N-1)
and
_ {Vb82 + (b-b.))2/(Vb+n-1)
Since the systems of equations is non-linear, the Newton-Raphson procedure
is employed to solve the equations iteratively. In order to accomplish
this, we let
ni 11 i[33] f~ t J- Pt (et-e.)/o - rj-
-17-
andN[34] h(b) - ij (b qj
i-i
Thenn
[35] f'(ei) = ij(-Pij) + { - 2(e-e)/(v+N-1/(a ) 2jz
and
[36) h'(bj) - - [ Pj(1-Pij) - {a(1 - - 2(bb )/(vb+n-l)}/(cb)2i-i
As before, if e(k) and b(k) denote the values of 8 and bj at the kth
iteration, then
[37] 0( k+l ) - 6(k) f(e(k))/ft(e(k))
and
(k+)= (k) - (k) (k)
(o) (0)Starting with initial values 6i (i1, ... , N), and bj (J-1, ... ,
(0)where ei is given by (23], and
b (0) log [(N-q i )/q i ]
6 is estimated. These values of e are then used to obtain revised estimates
of b. This process is repeated with the revised estimates of b being used
to obtain revised estimates of e. The process is terminated when the
convergence criterion is reached. This procedure is not the full Newton-
Raphson procedure and, in this case, is preferred to the full Newton-
Raphson procedure since the latter requires obtaining an inverse of the
-18-
matrix of second derivatives at each stage of the iteration. In practice,
the procedure outlined here converges rather rapidly.
As pointed out earlier, although the equations provided are for
estimating eB(i=1, ..., N), only er(r-l, ..., n-l) need be estimated.
In order to carry this out, the quantities given in Equations [34] and
[35] have to be computed as follows:
N n-lPiJ I Nr PJ (e)
i r=l r
N n-lPIJUl-Pij) Nr P j(er )11-P (er))i~l r=l
where N denotes the number of examinees who obtained raw score r and
Pj( 0 r) exp(er-bj)/ {1 + exp(Or-bj)}
Large Sample Properties of the Posterior Distributions2 s 2
The posterior pdf, p(e,bveVb,sesbx), given by Equation [30] is
a product of the likelihood function and a multivariate "double-t"
distribution. The "double-t" distribution is a product of two multi-
variate t densities (Tiao & Zellner, 1964; Zellner, 1971, p. 101). As
a result of its complex form, properties of the posterior pdf cannot be
obtained. However, it is possible to obtain the asymptotic properties
of the posterior pdf, and this will suffice, in most cases, for inferences
to be drawn regarding the parameters.
Let t be a vector of parameters, and y a vector of observations.
Then, the posterior pdf of t, p(dy), is
-19-
p(tl) -p t) L (yI
where p(t) is the prior distribution of t and L(yjIt), the likelihood
function. Then, for large samples,
[39] p(t) - L(yjt_)
and, in turn, L(yI.t) is approximately multivariate normal centered at
t, the maximum likelihood estimate, with dispersion matrix
[40] E _ [-a2 log L(yjt)/atiatj]-1-
Thus, for large samples,
For a detailed discussion of this result we refer the reader to Jeffreys
(1961, p. 193) and Zellner (1971, p. 32).
This result clearly applies in the present situation when both n
and N, the number of items and the number of examinees, are large.
Denoting the [(n+N)xl] vector [e,b] as
' [ , b']
[411 Llx N N(t, E)
In order to evaluate E, we write
G8 Ge[42] r
Gbe Gb
where
-20-
[431 Ge -3- 2109 L(xL~)3~O~
-t P (1-P )1 6i
where 6mis the Kronecker delta,
[44] Gb {-a 2log L(xiq,b)/3b abrn1,
NI Pi 2 (l-Pim) 62mv
and
[451 G eb = {-32log L(x[-Q,.)/ aeiabj I
p Pi(1-Pij
ij ij
estimate of 01. and variance, a2 , given by the ith diagonal element ofei
[46] a2 = [Ge -GGe e Geb GO GbeP
Similarly, the marginal distribution of bj has me an ithe maximum like-
2lihood estimate of b, and variance, Obj' given the jth diagnonal element
of E, i.e.,
[4] 1 [Gb - Gbe G;1 G 1
-21-
This approximation to the posterior pdf of e and b can be improved
upon if we take into account the "double-t" distribution (see Equation
[30]). For a sufficiently large sample, the multivariate t density ap-
proaches the normal density. Thus, in the expression
N (ve+N-1)/2[Ves2 + I (e -e.)
i=l
if we write veuNke where 0<k 8 <l, for large N, we obtain
2 N ee)2]N(k +l)/2+ (k8+l) N(0 2k T I (ei-e.)21
exp{- _ A11 --
where
(ke+l) 1 iA11 = keS2 [N] N
with IN being the identity matrix and l' = [i 1 ... 11 Similarly,
for large n,
2 n -(vb+n-l)/2 2 n -n(kb+l)/2+4k[491 Ivbsb + I (bj-b.)2] = [nkbsb + I (b-b. )2]
e{-(kb+1) nexpf- -b I(b -b.) 212kbsb jul
= exp(- b A22 b}
where(kb+l) 1
A2 2 2 (1'n n 1
Thus,
&
-22-
[50] [vs 8 2 2 (v b+n-l)I2 2 N - 2-(BNI/
b y (b -b. )2 vese + (e i )2
exp f --1(9A 1 t+ b' A2 2
[51] exp f-t A ti
where
0 ' b'
and
A = A l l 0
0 A2 2
Combining [411 and [51], we have
[52] P_-h~~v~ s , 22b
[53] 2x {(t-.t)' E(t-t) + t' A t)
[54] -exp{- -1 (t -)'T (t - T)
where
[55] T E -
and
[561 T - (E+A)- (Et)*
*This result follows from the fact that
(x-A)'A(x-a) + (xi-bTE(x-b) -(xi-t)'T(x-t) + constant,
where
T -A+B,
andt- (A+B)1' {Aa +Bb)
-23-
If the off diagonal matrix Gab in [42] can be ignored, then
2 1 2[57] 06 Pij(l-Pij)]- + (N-l)(k+l)/NkesoJ-1
and
N[58] a bj [ P(I-Pij ) ] - + (n-i)(kb+l)/nkbsb
The expression [57] is useful when the item parameters are considered
known. Similarly, [58] is applicable when the ability parameters are
known. In general, however, when the item and ability parameters are
estimated simultaneously, the off diagonal matrix, Gab, cannot be
ignored, and hence, in this case, the complete expression given by either
[46] or [55] should be employed. With these results it is possible to
construct "credibility intervals" (Novick & Jackson, 1974) for the
parameters of interest.
COMPARISON STUDIES
In order to study the efficacy of the Bayesian procedure described
above and to compare the Bayesian estimates with the maximum likelihood
estimates, a simulation study was carried out. Although simulation studies
may not be realistic in some situations, they can be justified in the
present context since only through a simulation study can one estimation
procedure be compared with another.
Artificial data, representing the responses of N individuals on n
items, were generated using DATGEN (Hambleton & Rovinelli, 1973) accord-
ing to the one-parameter logistic model. In generating the values of
- -
* . e *°* * * • *..** *..** -l * 4. ** 9 40
-24-
0i and bj (i-l, ... , N; J-1, ... , n), it was assumed that ei and bj were
independently and identically normally distributed with mean, zero, and
variance, unity (we shall return to a discussion of this issue later).
The design of the comparison study was conceptualized in terms of the
following, completely crossed, factors: estimation procedure (Bayesian,
maximum likelihood); number of examinees, N (20, 50); number of items,
n (15, 25, 40, 50). This design was carried out for (i) conditional
estimation of e, and, (ii) joint estimation of 6 and b.
The size of the examinee population, N, and the test length, n,
were chosen to facilitate comparison of the maximum likelihood and the
Bayesian estimates for small sample sizes and short tests, since the large
sample behavior of the maximum likelihood estimates has been studied by
Swaminathan and Gifford (1979). These authors have found that maximum
likelihood estimates of ei and bj approach the true values for N as large
as 200 and n as large as 100. Since for these values of N and n, Bayesian
estimates can be expected to be the same as maximum likelihood estimates,
the study was focused on small values of N and n.
The Bayesian estimates and the maximum likelihood estimates were
compared with respect to accuracy. The two sets of estimates were com-
pared with respect to: (a) the mean value of the estimates, as compared
with the mean value of the true values; (b) the mean squared error
difference between the true values and the estimated values; and, (c) the
regression of the true value on the estimated value.
It may be argued that since the joint modes of the posterior distri-
bution were taken as estimates of the parameters, the criterion employed
to determine the accuracy of the estimates is incompatible with the loss
1- + . . ....+ " -+. .. + z ; . .+ .... X-, .. -.. .. ... ..... .. . .... , '
-25-
function employed to arrive at the estimates. This is a valid argument.
However, we are primarily interested in comparing the Bayesian estimates
with the maximum likelihood estimates. Since, in one sense, the maximum
likelihood estimates can be thought of as the modes of the posterior
distribution derived under the assumption that the prior information is
vague, comparison of two modal estimates using a different criterion
other than that involved in deriving the estimates may be justifiable;
particularly since this will not provide an "unfair" advantage to one
set of estimates.
Comparison of the two estimation procedures in terms of the regres-
sion of true values on the estimates needs some explanation. If T is
the true value of the parameter and E, the estimate, then E(TIE-e) 0 8+Ole.
If OoOand 6 1-1, then, it can be concluded that the estimates are unbiased,
and hence, the departure from the expected values of 00and B1 can be
taken as an indicator of bias. It should be pointed out here that the
classical notion of bias is not critical in Bayesian analyses. Neverthe-
less, comparison of the regression lines will provide a further assessment
of the accuracy of the two procedures.
The comparison of the maximum likelihood (ML) procedure and the
Bayesian procedure for the conditional estimation of ability 8 is provided
in Table 1. The first column contains the means of the true values of 6,
the ML estimates, and the Bayesian estimates. The second column provides
an assessment of accuracy in terms of the mean squared deviation between
the estimate 6 and the true value, O%. The correlations between 8t and
Sfor each estimation procedure is displayed in column four, while the
:regression of 6 t on 6is given in column five.
-26-
cn m- 0% P- r-4 co rU, -t t~0 % %D -4 %D
0
C" c 0 %0 -74 C1 %.7 W)03 0 0 0 0 0 -4 0 0 00. .
S..4 mO 04 %O 1- n.7
0~- 1-4 -4 1; 7 74 .4 -
W.4
N 0 U, 0 %0 0 V740 CN r- N1 %T C.) -74 -.7 N
0 41iI 0 0 7
0.J0 w C4 0 LM 174 co -T r- U,"74 ;P- -4 LM L Go N -T r- Go
0 -W 0 r 0% , a% m% m% 0% ON 0%w cc
w 0)
0ow co wl U, r- .74 In r. Go1-4 54 0 0% ON 0% 0% 0% m% 0%
47
-4
0.7 0 r, cn %.0 Go v. ON ON -4m -- n 00 V) -r r4 N1 0% ON
to 04 H 0 0 0 1-4 .74 0 0.7H 41
%- , '.0 0 9> c 0 N1 .-4 %D0
Aj 4) w r r 0 en~ .r OD c -Ch4 -T .74 V74 -4 IT7 N N N4
-4 "-4
-H 0. N4 0 Mn U, rw) -4 N 00 0% 74 r. U, Ul
10 0o *. *. ..
0
%* * en cn .q4 'Q in
4~~l 101.-N .7
en Go~-- 0% %0%c.-
cn~U , 0 0. -
Al
a)
0 t0
lk
z
-27-
An examination of the correlations between the true values and esti-
mates reveals that, in general, the difference between ML and Bayes
estimates, is negligible for relatively large values of N and n. However,
for small values of N and/or n, the Bayes estimates correlate better
with true values than the ML estimates.
The correlation coefficient, by itself, is not a sufficient
indicator of the accuracy of estimation. Clear differences between the
Bayesian and ML procedures emerge when we examine the other criteria.
In general, the means of the Bayesian estimates, in comparison with
the ML estimates, are closer to the means of the true values. This result
can be anticipated if we examine the estimating equations [19]. The
estimating equations for ML estimates are:
nJX Pij = ri (i-1, ... , N).j-l
The additional term in the Bayesian estimating equations, (6i-e )/o2
contributes to the regression of the estimates towards the mean, and
hence, the Bayesian estimates are closer to the means of the true values.
The only exception occurs with N-20 and n-l, 25. At this point, there
is no explanation for this anomolous result. Further replications are
clearly necessary to establish this point conclusively.
The most dramatic difference between the Bayesian estimates and the
ML estimates is with respect to the mean squared deviations of the esti-
mates from the true values. In general, the mean squared deviations are
much smaller for the Bayesian estimates than for the ML estimates. The
difference is particularly noticable with small N and n. In these cases,
the mean squared deviations for the ML estimates is almost four times as
iL
71 . 4 . 00 0 ' 0
-28-
large as that for the Bayesian estimates. This finding can again be
explained by the fact prior information is most helpful in these cases.
This, together with the regression effect described previously, results
in an increase in the accuracy of the estimation procedure.
An examination of the regressions of true values on estimated
vlues also provides some interesting results. In general, the intercepts
and the slopes of the Bayesian regressions are closer to zero and one
respectively, than the ML regressions. The trend for the intercepts is
reversed for large n. In these cases, the intercepts for the ML regressions
are closer to zero than the intercepts for the Bayesian regressions. This
latter result is interpretable, since the maximum likelihood estimates of
6, for large N and n, approach the true values. However, the trend for
small n and N is rather surprising since, as a result of regression towards
the mean, the Bayesian estimates can be expected to be "biased." The only
explanation for this finding is that the ML procedure is severely biased
for small n and N, even more so than the Bayesian procedure.
The above findings, for conditional estimation of 8, appear to be
valid for the joint estimation of 6 and b (Tables 2 and 3). In fact, the
results for the joint estimation of e and b favor the Bayesian estimates
on all counts for both 0 and b: the means of the estimates are closer
to the means of the true values; the mean squared deviations are much
smaller (in some cases, one-tenth the size of those for ML estimates);
the slopes and intercepts are closer to one and zero respectively (the
only exception occurs for large N and n, in which case, the intercepts
of the ML regression are closer to zero).
* * * *** ~ * * * . .. . e *... e. * ** * * off
-29-
en r4 -4 LM ''-4 0 0 -4 cn 0% 0n7 £
P.fl
00 0a 0. 0 - cn.0 0 0 C40 r
0
ca n
00 .4 M...Pw-4 4 -4 -4 r4 r-4 - .4
0
40 0 N o a~ r'. 00 .-4 r- 00 V4 en f'0 4rl c
.04-u 04.) *1
.0
0(D 0% It 0% 0% i 0% 0% 0% 0
(D0 0
00 0% *n in r4 %0N (a0% IT 0% 0 %0 0%
0 014 C14n
en P. Go N -Z 0% N 0~
a-) 0% 04 0% cn i o r-..
___ V- 4L - 4 4n
(a 93 t ' 7 7 iI i
00 ~ C~ N. N ' 0% ~D10 N -4 ' r4 0 N C'
* S 0 do .
-30-
- % - e~4 0 C4 4 cn4
- - v4 - r-4 9-4 .
&f~ ' U'4 0 -4 .0 '
c q t n r C 4 r- c% r-4 0nk 0q oo 0 v.4a 0% 0 0
V9.
0% 0)0 N 0 C4 4v.o A.0, O 0 r- at n 0% Ln r-4
0 ~~ co4 0% M. v-0 - . .r4 -I 0
r-. ON 0% I- irn C) ~0 1- a 0 0% ON CS % ON Nh m
V 0 LW. ~ v4 .
4JC4
00 CO n C c 0 ren % 0% 0% ON ON~ 0 s 4
41 -4 ~ U ' 0 0 ' -
CdUE-4 0
0v0 P.4 . 0 t*- ON1 O N -0 cc N 0- C)) CV0'0 r-
H00v4 *4 -
0 4) 1.0- 0 N IC:O "- 0 LI 0 Le) 0% m-
0 CU 00 v.4 as v.4 0 1 %t 4 o
01 14 ~ 0 4' f '
41 10 00 -4 "-4 r4 I- '0 ON %0
00
m) IT) 0 o 4 C4) fl- 10 0%' 0 0 0. v.4 m~ LI 0%
CU0 4 0 0 00 0- 0*v-4C . .
0I 0 I 0 0 C
4) 4 U) N " %0 cou0e
v0 Ln -I I - 1 4
o.10-4 0
0
4i 01 LM
'REM
-31-
DISCUSSION
The Bayesian procedure for estimating parameters in the one-parameter
latent trait model is an attractive alternative to the maximum likelihood
procedure. Bayesian procedures are conceptually more appealing since
direct interpretations of probability statements involving the parameters
are possible. Empirically, as the results of the comparison study indi-
cate, the Bayesian estimates of the parameters are superior to the maximum
likelihood estimates in terms of their accuracy.
Although the empirical results demonstrate the effectiveness of the
Bayesian procedure, it may be argued, and correctly, that the simulation of
the data favored the Bayesian procedure. The data were generated to meet
the strong distributional assumptions required by the Bayesian procedure.
In addition, in specifying prior belief about the distribution 00 and 0b*
s2 and s2 were set equal to one with the corresponding v6 and Vb being
specified as 15. In the simulation *e and *b were set at one, and the
specification of s2, s2 and the relatively large values for ye and vb
reproduced the true state of affairs. It is not surprising, therefore,
that the Bayesian procedure proved to be superior to the maximum likeli-
hood procedure.
In fairness to the study, it should be pointed out that the simula-
tion and the accurate specification of prior belief were deliberate in
order to determine the applicability of the Bayesian procedure, at least, under
ideal conditions. Preliminary investigations with non-normal data and
also with poor specification of priors indicate that the Bayesian proce-
dure, being based on a hierarchical model, is relatively robust and is
superior to the maximum likelihood procedure. A detailed study of the
effects of poor specification of priors and departures from underlying
- Mfib&MM"
-32-
assumptions is currently under way and we expect to report these results
in the near future.
Despite the encouraging results obtained, a theoretical problem
still remains with the estimation procedure. The procedure described in
this paper requires the joint estimation of n structural parameters and
N incidental parameters. If N- while n remains fixed, the joint posterior
pdf may not become concentrated about the estimated values. This trend
is evident from Tables 2 and 3; with increasing N, the intercept and
slope do not tend to zero and one respectively. This problem is similar
to the one that exists with maximum likelihood estimates. Although
from a Bayesian point of view asymptotic properties, such as consistency,
are not critical, the lack of them, at least to some degree, is discom-
forting. It appears that this situation can be remedied, if when esti-
mating the n structural, or item, parameters, the ability parameters
are considered nuisance parameters and can be integrated out to yield the
marginal posterior pdf of b. The marginal posterior pdf is currently
not available as a result of the exceedingly complex form of the joint
posterior pdf. 4pproximations, such as the one indicated (Equation [151)
may be employed to simplify the joint posterior pdf. Initial investiga-
tions reveal that this approximation is reasonably good, but further
research in this area is clearly needed.
In summary, we note that the Bayesian procedure developed in this
paper is relatively simple to implement, and computationally as efficient
as the maximum likelihood procedure. Despite the issues raised above,
the Bayesian procedure has the potential for greatly improving the accuracy
of the estimates. Moreover, the maximum improvement in accuracy occurs
-33-
for small values of N and n, a result that can be expected, and this
makes the Bayesian procedure more attractive than the maximum likelihood
procedure.
-34-
References
Andersen, E. B. Asymptotic properties of conditional maximum likelihoodestimates. The Journal of the Royal Statistical Society, Series B,1970, 32, 283-301.
Andersen, E. B. The numerical solution of a set of conditional equations.The Journal of the Royal Statistical Society, Series B, 1972, 34,42-54.
Andersen, E. B. Conditional inference in multiple choice questionnaire.British Journal of Mathematical and Statistical Psychology, 1973,26, 31-44. (a)
Andersen, E. B. A goodness of fit test for the Rasch model. Psychometrika,1973, 28, 123-140. (b)
Birnbaum, A. Statistical theory for logistic mental test models witha prior distribution of ability. Journal of Mathematical Psychology,1969, 6, 250-276.
Bock, R. D. Estimating item parameters and latent ability when re-sponses are scored in two or more nominal categories. Psychometrika,1972, 37, 29-51.
Bock, R. D., & Lieberman, M. Fitting a response model for n dichotomouslyscored items. Psychometrika, 1970, 35, 179-197.
Hambleton, R. K., & Rovineili, R. A FORTRAN IV program for generatingexaminee response data from logistic test models. BehavioralScience, 1973, 18, 74.
Hambleton, R. K., Swaminathan, H., Cook. L., Eignor, D., & Oifford, J. A.Developments in latent trait theory: Models, technical issues, andapplications. Review of Educational Research, 1978, 48, 467-510.
Jeffreys, H. Theory of probability (3rd ed.). Oxford: Clarendon, 1961.
Jensema, C. J. A simple technique for estimating latent trait mental testparameters. Educational and Psychological Measurement, 1976, 36,705-715.
Lindley, D. V. The estimation of many parameters. Foundations of Statis-tical Inference. (V. P. Godambe and D. A. Sprott, Eds.) Toronto:Holt, Rinehart, and Winston, 1971, 435-455.
Lindley, D. V., & Smith, A. F. Bayesian estimates for the linear model.Journal of the Royal Statistical Society, Series B, 1972, 34, 1-41.
-35-
Lord, F. M. An analysis of verbal Scholastic Aptitude Test using Birnbaum'sthree-parameter logistic model. Educational and PsychologicalMeasurement, 1968, 28, 989-1020.
Lord, F. M. Estimation of latent ability and item parameters when thereare omitted responses. Psychometrika, 1974, 39, 247-264.
Lord, F. M., & Novick, M. R. Statistical theories of mental test scores.Reading, MA: Addison-Wesley, 1968. -
Meredith, W., & Kearns, J. Empirical Bayes point estimates of latenttrait scores without the knowledge of the trait distribution.Psychometrika, 1973, 38, 533-554.
Novick, M. R. Bayesian considerations in educational information systems.Proceedings of the 1970 Invitational Conference on Testing Problems.Princeton, NJ: Educational Testing Service, 1971.
Novick, M. R., & Jackson, P. H. Statistical methods for educational andpsychological research. New York: McGraw-Hill, 1974.
Novick, M. R., Lewis, C., & Jackson, P. H. The estimation of proportionsin n groups. Psychometrika, 1973, 38, 19-46.
Owen, R. A. Bayesian sequential procedure for quantal response in thecontext of adaptive mental testing. Journal of the AmericanStatistical Association, 1975, 70, 351-356.
Robbins, H. An empirical Bayes approach to statistics. Proceedings ofthe Third Berkeley Symposium on Math, Statistics, and Probability1, 157. University of California Press, 1955.
Sanathanan, L., & Blumenthal, S. The logistic model and estimation oflatent structure. Journal of American Statistical Association,1978, 73, 794-799.
Samejima, F. Estimation of latent ability using a response pattern ofgraded scores. Psychometric Monograph, 1969, No. 17.
Samejima, F. A general model for free-response data. Psychometric Mono-
graph, 1972, No. 18.
Stein, C. M. Confidence sets for the mean of a multivariate normaldistribution. Journal of the Royal Statistical Society, Series B,1962, 24, 265-296.
Swaminathan, H., & Gifford, J. A. Precision of item parameter estimation.Paper presented at the 1979 Computerized Adaptive Testing Conference.Minneapolis, June 1979.
-36-
Tiao, G. C., & Zellner, A. Bayes' Theorem and the use of prior knowledgein regression analysis. Biometrika, 1964, 51, 219-230.
Urry, V. W. Approximations' to item parameters of mental test models andtheir uses. Educational and Psychological Measurement, 1976, 34,253-269.
Wright, B. D. Solving measurement problems with the Rasch model. Journalof Educational Measurement, 1977, 14, 97-116.
Wright, B. D., & Douglas, G. A. Best procedure for sample-free itemanalysis. Applied Psychological Measurement, 1977, 1, 281-295.
Wright, B. D., & Panchapakesan, N. A procedure for sample-free itemanalysis. Educational and Psychological Measurement, 1969, 29, 23-48.
Zellner, A. An introduction to Bayesian inference in econometrics.New York: Wiley, 1971.
IA
Page 1
DISTRIBUTION LIST
Navy Navy
Dr. Jack R. Borsting 1 Dr. Patrick R. HarrisonProvost & Academic Dean Psychology Course DirectorU.S. Naval Postgraduate School LEADERSHIP & LAW DEPT. (7b)Monterey, CA 93940 DIV. OF PROFESSIONAL DEVELOPMMENT
U.S. NAVAL ACADEMYDr. Robert Breaux ANNAPOLIS, MD 21402Code N-711NAVTRAEQUIPCEN 1 Dr. Norman J. KerrOrlando, FL 32813 Chief of Naval Technical Training
Naval Air Station Memphis (75)Chief of Naval Education and Training Millington, TN 38054
Liason OfficeAir Force Human Resource Laboratory 1 Dr. William L. MaloyFlying Training Division Principal Civilian Advisor forWILLIAMS AFB, AZ 85224 Education and Training
Naval Training Command, Code OOACOMNAVMILPERSCOH (N-6C) Pensacola, FL 32508Dept. of NavyW;ashington, DC '0370 1 Dr. Kneale Marshall
Scientific Advisor to DCNO(MPT)Deputy Assistant Secretary of the Navy OPOT
(Manpower) Washington DC 20370Office of the Assistant Secretary of
the Navy (1Wanpower, Reserve Affairs,1 CAPT Richard L. Martin, USNand Logistics) Prospective Commanding Officer
Washington. DC 20350 USS Carl Vinson (CVN-70)Newport News Shipbuilding and Drydock Co
Dr. Richard Elster Newport News, VA 23607Department of Administrative SciencesNaval Postgraduate School 1 Dr. James McBrideMonterey, CA 93940 Navy Personnel R&D Center
San Diego, CA 92152DR. PAT FEDERICONAVY PERSONNEL R&D CENTER 1 Dr. George MoellerSAN DIEGO, CA 92152 Head, Human Factors Dept.
Naval Submarine Medical Research Lab'Ir. Paul Foley Groton, CN 06340lavy Personnel R&D CenterSan Diego, CA 92152 1 Library
Naval Health Research Center
Dr. John Ford P. 0. Box 85122lavy Personnel R&D Center San Diego, CA 92138San Diego, CA 92152
1 Naval Medical R&D CommandCode 44National Naval Medical CenterBethesda, MD 20014
{I
Page 2
Navy Navy
Ted M. I. Yellen 1 Office of the Chief of Naval OperationsTechnical Information Office, Code 201 Research, Developnent, and Studies BrancNAVY PERSONNEL R&D CENTER (OP-102)SAN DIEGO, CA 92152 Washington, DC 20350
Library, Code P201L 1 Head, Manpower Training and ReservesNavy Personnel R&D Center Section (Op-964D)San Diego, CA 92152 Room 4A478, The Pentagon
Washington, DC 20350Technical DirectorNavy Personnel R&D Center 1 Captain Donald F. Parker, USNSan Diego, CA 92152 Commanding Officer
Navy Personnel R&D Center6 Commanding Officer San Diego, CA 92152
Naval Research LaboratoryCode 2627 1 LT Frank C. Petho, MSC, USN (Ph.D)Washington, DC 20390 Code L51
Naval Aerospace Medical Research LaboratPsychologist Pensacola, FL 32508ONR Branch OfficePldg 114, Section D 1 The Principal Deputy Assistant666 Summer Street Secretary of the Navy (MRA&L)Boston, MA 02211 4E780, The Pentagon
Washington, DC 20350PsychologistONR Branch Office 1 Director, Research & Analysis Division536 S. Clark Street Plans and Policy DepartmentChicago, IL 606.05 Navy Recruiting Command
4015 Wilson Boulevard')ffice of aval Research Arlington, VA 22203Code 437300 N. Quincy SStreet I Dr. Bernard Rimland (03B)Arlington, VA 22217 Navy Personnel R&D Center
San Diego, CA 921525 Personnel '. Training Research Programs
(Code 458) 1 Mr. Arnold RubensteinOffice of Naval Research Naval Personnel Support TechnologyArlington, VA 22217 Naval Material Command (08T244)
Room 104, Crystal Plaza #5Psychologist 2221 Jefferson Davis HighwayONR Branch Office Arlington, VA 203601030 East Green StreetPasadena, CA 91101 1 Dr. Worth Scanland
Chief of Naval Education and TrainingSpecial Asst. for Education and Code N-5
Training (OP-OIE) NAS, Pensacola, FL 32508Rm. 2705 Arlington AnnexWashington, DC 20370
Page 3
Navy Army
Dr. Fobert G. 3mith I Technical DirectorOffice of Chief of Naval Operations U. S. Army Research Institute for theOP-987H Behavioral and Social SciencesWashington, DC 20350 5001 Eisenhower Avenue
Alexandria, VA 22333Dr. Alfred F. SmodeTraining Analysis & Evaluation Group 1 HQ USAREUE & 7th Army
(TAEG) ODCSOPSDept. of the Navy USAAREUE Director of GEDOrlando, FL 32813 APO New York 09403
Dr. Richard Sorensen I DR. RALPH DUSEKNavy Personnel R&D Center U.S. ARMY RESEARCH INSTITUTESan Diego. CA 92152 5001 EISENHOWER AVENUE
ALEXANDRIA, VA 22333W. Gary ThomsonNaval Ocean Systems Center I Dr. Myron FischlCode 7132 U.S. Army Research Institute for theSan Diego, CA 92152 Social and Behavioral Sciences
5001 Eisenhower AvenueDr. Ronald Weitzman Alexandria, VA 22333Code 54 WZDepartment of Administrative Sciences 1 Dr. Michael KaplanU. S. Naval Postgraduate School U.S. ARMY RESEARCH INSTITUTEMonterey, CA 93940 5001 EISENHOWER AVENUE
ALEXANDRIA, VA 22333DR. MARTIN F. WISKOFFNAVY PERSONNEL R& D CENTER 1 Dr. Milton S. KatzSAN DIEGO, CA 92152 Training Technical Area
U.S. Army Research InstituteMr John H. Wolfe 5001 Eisenhower AvenueCode P310 Alexandria, VA 22333U. S. Navy Personnel Research and
Development Center 1 Dr. Harold F. O'Neil, Jr.San Diego, CA 92152 Attn: PERI-OK
Army Research Institute5001 Eisenhower AvenueAlexandria, VA 22333
1 DR. JAMES L. RANEY
U.S. ARMY RESEARCH INSTITUTE5001 EISENHOWER AVENUEALEXANDRIA, VA 22333
1 Mr. Robert RossU.S. Army Research Institute for the
Social and Behavioral Sciences5001 Eisenhower AvenueAlexandria, VA 22333
I
Page 4
Army Air Force
Dr. Robert Sasnor 1 Air Force Human Resources LabU. S. Army Research Institute for the AFHRL/MPD
Behavioral and Social Sciences Brooks AFB, TX 782355001 Eisenhower AvenueAlexandria, VA 22333 1 U.S. Air Force Office of Scientific
ResearchCommandant Life Sciences Directorate, NLU3 Army Institute of Administration Bolling Air Force BaseAttn: Dr. Sherrill Washington, DC 20332FT Benjamin Harrison, IN 46256 -
1 Air University LibraryDr. Frederick Steinheiser AUL/LSE 76/443U. S. Army Reserch Institute Maxwell AFB, AL 361125001 Eisenhower AvenueAlexandria, VA 22333 1 Dr. Earl A. Alluisi
HQ, AFHRL (AFSC)Dr. Joseph Ward Brooks AFB, TX 78235U.S. Army Research Institute5001 Eiserhower Avenue 1 Dr. Genevieve HaddadAlexandriE:, VA 22333 Program Manager
Life Sciences DirectorateAFOSRBolling AFB, DC 20332
1 Dr. Ross L. Morgan (AFHRL/LR)Wright -Patterson AFBOhio 45433
1 Research and Measurment DivisionResearch Branch, AFMPC/MPCYPRRandolph AFB, TX 78148
1 Dr. Malcolm ReeAFHRL/MP
Brooks AFB, TX 78235
1 Dr. Marty Rockway (AFHRL/TT)
Lowry AFBColorado 80230
1 Dr. Frank SchufletowskiU.S. Air ForceATC/XPTD
Randolph AFB, TX 78148
1 Jack A. Thorpe, Maj., USAF
Naval War CollegeProvidence, RI 02846
Page 5
Air Force Marines
Dr. Joe Ward, Jr. 1 H. William GreenupAFHRL/1PMD Education Advisor (E031)Drooks AFB, TX 78235 Education Center, MCDEC
Quantico, VA 22134
1 Director, Office of Manpower UtilizationHO, Marine Corps (MPU)BCB, Bldg. 2009Quantico, VA 22134
1 Headquarters, U. S. Marine CorpsCode MPI-20Washington, DC 20380
1 Major Michael L. Patrow, USMCHeadquarters, Marine Corps(Code MPI-20)Washington, DC 20380
1 DR. A.L. SLAFKOSKYSCIENTIFIC ADVISOR (CODE RD-1)HO, U.S. MARINE CCRPSWASHINGTON, DC 20380
Page 6
CoastGu ard Other DoD
Chief, Psychological Reserch Branch 12 Defense Documentation CenterU. S. Coast Guard (G-P-1/2/TP42) Cameron Station, Bldg. 5Washington, DC 20593 Alexandria, VA 22314
Attn: TCMr. Thomas A. WarmU. S. Coast Guard Institute 1 Dr. Dexter FletcherP. 0. Substation 18 ADVANCED RESEARCH PROJECTS AGENCYOklahoma City, OK 73169 1400 WILSON BLVD.
ARLINGTON, VA 22209
1 Dr. William GrahamTesting DirectorateMEPCa4/MEPCT-PFt. Sheridan, IL 60037
I Director, Research and DataOASD(MRA&L)3B919, The PentagonWashington, DC 20301
1 Military Assistant for Training andPersonnel Technology
Office of the Under Secretary of Defensefor Research & Engineering
Room 3D129, The PentagonWashington, DC 20301
1 MAJOR Wayne Sellman, USAFOffice of the Assistant Secretary
of Defense (MRA&L)3B930 The PentagonWashington, DC 20301
Page 7
Civil Govt Non Govt
Dr. Susan Chipman 1 Dr. Erling B. Andersen
Learning and Development Department of StatisticsNational Institute of Education Studiestraede 61200 19th Street NW 1455 CopenhagenWashington, DC 20208 DENMARK
Dr. Lorraine D. Eyde 1 1 psychological research unitPersonnel R&D Center Dept. of Defense (Army Office)Office of Personnel Management of USA Campbell Park Offices1900 EStreet !.,W Canberra ACT 2600, AustraliaWashington, D.C. 20415
1 Dr. Alan BaddeleyJerry Lehnus Medical Research CouncilREGIONAL PSYCHOLOGIST Applied Psychology UnitU.S. Office of Personnel Management 15 Chaucer Road230 S. DEAR.ORN STREET Cambridge CB2 2EFCHICAGO, IL 60604 ENGLAND
Dr. Andrew R. Molnar 1 Dr. Isaac BejarScience Education Dev. Educational Testing Service
and Research Princeton, NJ 08450National Science FoundationWashington, DC 20550 1 Dr. Werner Birke
DezWPs im StreitkraefteamtPersonnel R&P Center Postfach 20 50 03Office of Personnel Managment D-5300 Bonn 21900 F Street NW WEST GERMANYWashington, DC 201115
1 Dr. R. Darrel BockDr. H. Wallace Sinaiko Department of EducationProgram Director University of ChicagoManpower Research and Advisory Services Chicago, IL 60637Smithsonian Institution301 North Pitt Street 1 Dr. Nicholas A. BondAlexandria, VA 22314 Dept. of Psychology
Sacramento State CollegeDr. Vern W. Urry 600 Jay StreetPersonnel R&D Center Sacramento, CA 95819Office of Personnel Management1900 E Street NW 1 Dr. Robert BrennanWashington, DC 20415 American College Testing Programs
P. 0. Box 168Dr. Joseph L. Young, Director Iowa City, IA 52240Memory & Cognitive ProcessesNational Science Foundation 1 DR. C. VICTOR BUNDERSONWashington, DC 20550 WICAT INC.
UNIVERSITY PLAZA, SUITE 101160 SO. STATE ST.OREM, UT 84057
Page 8
Non Govt Non Govt
Dr. John P. Carroll 1 Dr. A. J. EschenbrennerPsychometric Lab Dept. E422, Bldg. 81Univ. of No. Carolina McDonnell Douglas Astronautics Co.Davie Hall 013A P.O.Box 516Chapel Hill, NC 27514 St. Louis, MO 63166
Charles Myers Library 1 Dr. Leonard FeldtLivingstone House Lindquist Center for MeasurmentLivingstone Road University of Iowa
Stratford Iowa City, IA 52242
London E15 2LJENGLAND 1 Dr. Richard L. Ferguson
The American College Testing Program
Dr. Kenneth E. Clark P.O. Box 168
College of Arts & Sciences Iowa City, IA 52240
University of RochesterRiver Campus Station 1 Dr. Victor FieldsRochester, NY 14 27 Dept. of Psychology
Montgomery CollegeDr. Plorman Cliff Rockville, MD 20850Dept. of PsychologyUniv. of So. California 1 Univ. Prof. Dr. Gerhard Fischer
Univ,rsity Park Liebiggasse 5/3
Los Angeles, CA 90007 A 1010 ViennaAUSTRIA
Dr. William E. CoffmanDirector, Iowa Testing Programs 1 Professor Donald Fitzgerald334 Lindquist Center University of New EnglandUniversity of Iowa Armidale, New South Wales 2351Iowa City, IA 52242 AUSTRALIA
Dr. Meredith P. Crawford 1 Dr. Edwin A. FleishmanAmerican Psychological Association Advanced Research Resources Organ.
1200 17th Street, N.W. Suite 900Washington, DC 2C036 4330 East West Highway
Washington, DC 20014Dr. Hans CrombagEducation Research Center 1 Dr. John R. Frederiksen
University of LeyJen Bolt Beranek & NewnanFoerhaaveloan 2 50 Moulton Street2334 EN Leyden Cambridge, MA 02138The NETHERLANDS
1 DR. ROBERT GLASER
LCOL J. C. Eggenberger LRDCDIRECTORATE OF PERSONNEL APPLIED RESEARC UNIVERSITY OF PITTSBURGH
NATIONAL DEFENJCE 11Q 3939 O'HARA STREET
101 COLONEL BY DRIVE PITTSBURGH, PA 15213
OTTAWA, CANADA ',IA OK2
...
Page 9
Non Govt Non Govt
Dr. Daniel Gopher 1 Dr. Douglas H. JonesTndustrial & Management Engineering Rn T-255Technion-Israel Institute of Technology Educational Testing ServiceHaifa Princeton, NJ 08450ISRAEL
3 Journal Supplement Abstract ServiceDr. Ron Hambleton American Psychological AssociationSchool of Education 1200 17th Street N.W.University of Massechusetts Washington, DC 20036Amherst, MA 01002
1 Professor John A. KeatsDr. Chester Harris University of NewcastleSchool of Education AUSTRALIA 2308University of CaliforniaSanta Parbara, CA 93106 1 Dr. Stephen Kosslyn
Harvard UniversityGlenda Greenwald, Ed. Department of Psychology"Humrn Intelligence Newsletter" 33 Kirkland StreetF. 0. Box 1163 Cambridge, MA 02138F~irmingham, rMI '8012
1 Mr. Marlin Kroger
Dr. Lloyd Humphreys 1117 Via GoletaDepartment of Psychology Palos Verdes Estates, CA 90274University of IllinoisChampaign, IL 61E20 1 Dr. Alan Lesgold
Learning R&D CenterLibrary University of PittsburghPumRRO/Western Division Pittsburgh, PA 1526027157 Berwick DriveCarmel, CA 93921 1 Dr. Michael Leviner.te210 Education Building
Dr. Steven Hunka University of IllinoisDepartment of Education Champaign, IL 61820University of AlbertaEdmonton, Alberta 1 Dr. Charles LewisCANADA Faculteit Sociale Wetenschappen
Rijksuniversiteit GroningenDr. Earl Hunt Oude BoteringestraatDept. of Psychology GroningenUniversity of kishington NETHERLANDSSeattle, WA 98105
1 Dr. Robert LinnDr. Huynh Huynh College of EducationCollege of Education University of IllinoisUniversity of South Carolina Urbana, IL 61801Columbia, SC 29208
1 Dr. Frederick M. LordEducational Testing ServicePrinceton, NJ 08540
I.._
Page 10
Non Govt No n G vt
Dr. Jaries Lumsden 1 Dr. James A. PaulsonDepartm ent of Psycho~logy Portland State University
University of Western Australia P.O. Box 751Nedlands W.A. 6009 Portland, OR 97207AUSTRALIA
1 MR. LUIGI PETRULLO
Dr. Gary Marco 2431 N. EDGEWOOD STREETEducational Testing Service ARLINGTON, VA 22207Princeton, NJ 08450
1 DR. DIANE M. RAMSEY-KLEE
Dr. Scott Maxwell R-K RESEARCH & SYSTEM DESIGNDepartnent of Psychology 3947 RIDGEMONT DRIVE
Univernity of Houston MALIBU. CA 90265Houston, TX 77004
1 H1INRAT M. L. RAUCH
Dr. Samuel T. Mayo P II 11Loyoln University of Chicago BUNDESMINISTERIUM DER VERTEIDIGUNG
q20 North Michigan Avenue POSTFACH 1328Chicago, IL 60611 D-53 BONN 1. GERMANY
Dr. M.Irk Mliller I Dr. Mark D. ReckaseComputer Science Laboratory Educational Psychology Dept.Texas Instruments, Inc. University of Missouri-ColumbiaMail Station 371, P.O. Box 225936 4 Hill HallDallas, TX 75265 Columbia, MO 65211
Professor Jason ?illman 1 Dr. Andrew M. Rose
Department of Education American Institutes for ResearchStone Hall 1055 Thomas Jefferson St. NWCornell University Washington, DC 20007ithaca. NY 14P53
1 Dr. Leonard L. Rosenbaum, Chairman
Dr. Allen Vunro Department of Psychology
Pehavioral Technology Laboratories Montgoery College
1P,15 Elena Ave., Fourth Floor Rockville, MD 20850Redonjo Bea ch, CA 90277
1 Dr. Lawrence Rudner
Dr. Melvin R. Novick 403 Elm Avenue
356 Lindquist Center for Measurment Takoma Park, MD 20012
University of IowaIowa City, IA 522112 1 Dr. J. Ryan
Department of Education
Dr. Jesse Orlansky University of South Carolina
Institute for Defense Analyses Columbia, SC 29208400 Army Navy DriveArlington, VA 22'n2 1 PROF. FUMIKO SAMEJIMA
DEPT. OF PSYCHOLOGYUNIVERSITY OF TENNESSEEKNOXVILLE, TN 3716
Page 11
Non Govt Non Govt
DR. ROBERT J. SEIDEL 1 Dr. Brad SympsonINSTRUCTIONAL TECHNOLOGY GROUP Psychometric Research Group
HUMRRO Educational Testing Service300 N. WASHINGTON ST. Princeton, NJ 08541ALEXANDRIA, VA 22314
1 Dr. Kikumi TatsuokaDr. Kazuo Shigemasu Computer Based Education ResearchUniversity of Tohoku LaboratoryDepartment of Educational Psychology 252 Engineering Research LaboratoryKawauchi, Sendai 980 University of IllinoisJAPAN Urbana, IL 61801
Dr. Edwin Shirkey 1 Dr. David ThissenDepartment of Psychology Department of PsychologyUniversity of Central Florida University of KansasOrlando, FL 32816 Lawrence, KS 66044
Dr. Richard Snow 1 Dr. J. UhlanerSchool of Education Perceptronics. Inc.Stanford University 6271 Variel AvenueStanford, CA P 1305 Woodland Hills, CA 91364
Dr. Kathryn T. Spoehr 1 Dr. Howard WainerDepartment of Psychology Bureau of Social SCience ResearchBrown University 1990 M Street, N. W.Providence, RI 02912 Washington, DC 20036
Dr. Robert Sternberg 1 Dr. David J. WeissDept. of Psychology N660 Elliott HallYale University University of MinnesotaPox 11A, Yale Stition 75 E. River RoadNew Haven, CT (1520 flinneapolis, MN 55455
DR. PATRICK SUPPES 1 DR. GERSHON WELTMANINSTITUTE FOR MATHEMATICAL STUDIES IN PERCEPTRONICS INC.
THE SOCIAL ",CIENCES 6271 VARIEL AVE.STANFORD UNIVERSTTY WOODLAND HILLS, CA 91367STANFORD, CA 9005
1 DR. SUSAN E. WHITELYDr. Iariharan Swiminathan PSYCHOLOGY DEPARTMENTLaboratory of Psychometric and UNIVERSITY OF KANSAS
Evaluation Research LAWRENCE, KANSAS 66044School of EducationUniversity of M;-jsachusetts 1 Wolfgang WildgrubeAmherst, MA 01007 StreiJ'.raefteamt
Box 20 50 03D-5300 Bonn 2WEST GERMANY
Recommended