4
PSYCHOMETRIKA--VOL.51, NO. 1, 53-56 MARCH 1986 50TH ANNIVERSARY SECTION STATISTICS, DATA ANALYSIS AND PSYCHOMETRIKA: MAJOR DEVELOPMENTS MARK I. APPELBAUM UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL Approximately 15% of all papers appearing in Psychometrika during the period 1960-1984 have been statistical or data analytic in nature. While these papers cover a very wide range of issues and are technically quite similar to those appearing in other major statistical journals, collectively they possess features which make them identifiably Psychometrika. In 1973 Bert Green began his editorship of Psychometrika by noting in his Letter from the Editor: Psychometrika has always been, and will continue to be, devoted to the development of psychology as a quantitative rational science. The past 36 years have witnessed remarkable growth in this area. Companion journals have appeared, notably the British Journal of Mathematical & Statistical Psychology, the Journal of Mathematical Psychology, and Multivariate Behavioral Research, as well as others devoted mainly to educational measurement. Some saw the rise of these journals as evidence that Psychometrika was not doing its job, but there are enough good manuscripts to fill all these journals, so the alternative would have been a four-fold expansion of Psy- chometrika. Plurality has its virtues; its price is the tendency toward com- partmentalization. In the belief that multivariate analysts, psychometricians, and mathematical modelers still have much to learn from each other, Psychometrika in- tends to resist being typed. We originally sought and continue to seek papers from all quantitative areas in psychology. Since the time of that statement the number of journals publishing essentially statistical work of the type commonly seen in Psychometrika has continued to expand with the addition of the Journal of Educational Statistics, the Journal of Classification, and, with a revised editorial policy, the Psychological Bulletin. Yet despite these and other expanded channels for the communication of statistical ideas, Psychometrika has remained a pri- mary journal among those behavioral scientists whose work centers upon statistical issues. I hope that in the processes of examining the last 25 years of statistical papers in Psychometrika to shed some light on the reasons that this might be so. In doing a retrospective review of this nature one is first forced to decide which papers are within one's assigned domain and which are not. At first glance this seems a rather simple task. However, the very nature of papers appearing in the journal make that task far more difficult than might first be imagined and one in which there might be some disagreement among raters. (Is the analysis of disagreement among raters, for instance, part of test theory/psychometrics or contingency table analysis/statistics?) After some re- flection an "if a and not b" strategy was adopted. Thus, if a paper was fundamentally statistical in nature (i.e., resulted in p-values, estimates of parameters of sampling distri- bution, or focused upon descriptive indices) and did not belong to one of the other areas Requests for reprints should be sent to Mark I. Appelbaum, The L. L. Thurstone Psychometric Labora- tory, University of North Carolina, Davie Hall 013A, Chapel Hill, NC 27514. 0033-3123/86/0300-F001 $00.75/0 53 © 1986 The Psychometric Society

Statistics, data analysis and Psychometrika: Major developments

Embed Size (px)

Citation preview

PSYCHOMETRIKA--VOL. 51, NO. 1, 53-56 MARCH 1986 50TH ANNIVERSARY SECTION

STATISTICS, DATA ANALYSIS A N D P S Y C H O M E T R I K A : MAJOR D E V E L O P M E N T S

MARK I. APPELBAUM

UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL

Approximately 15% of all papers appearing in Psychometrika during the period 1960-1984 have been statistical or data analytic in nature. While these papers cover a very wide range of issues and are technically quite similar to those appearing in other major statistical journals, collectively they possess features which make them identifiably Psychometrika.

In 1973 Bert Green began his editorship of Psychometrika by noting in his Letter from the Editor:

Psychometrika has always been, and will continue to be, devoted to the development of psychology as a quantitative rational science. The past 36 years have witnessed remarkable growth in this area. Companion journals have appeared, notably the British Journal of Mathematical & Statistical Psychology, the Journal of Mathematical Psychology, and Multivariate Behavioral Research, as well as others devoted mainly to educational measurement. Some saw the rise of these journals as evidence that Psychometrika was not doing its job, but there are enough good manuscripts to fill all these journals, so the alternative would have been a four-fold expansion of Psy- chometrika. Plurality has its virtues; its price is the tendency toward com- partmentalization. In the belief that multivariate analysts, psychometricians, and mathematical modelers still have much to learn from each other, Psychometrika in- tends to resist being typed. We originally sought and continue to seek papers from all quantitative areas in psychology.

Since the time of that statement the number of journals publishing essentially statistical work of the type commonly seen in Psychometrika has continued to expand with the addition of the Journal of Educational Statistics, the Journal of Classification, and, with a revised editorial policy, the Psychological Bulletin. Yet despite these and other expanded channels for the communication of statistical ideas, Psychometrika has remained a pri- mary journal among those behavioral scientists whose work centers upon statistical issues. I hope that in the processes of examining the last 25 years of statistical papers in Psychometrika to shed some light on the reasons that this might be so.

In doing a retrospective review of this nature one is first forced to decide which papers are within one's assigned domain and which are not. At first glance this seems a rather simple task. However, the very nature of papers appearing in the journal make that task far more difficult than might first be imagined and one in which there might be some disagreement among raters. (Is the analysis of disagreement among raters, for instance, part of test theory/psychometrics or contingency table analysis/statistics?) After some re- flection an "if a and not b" strategy was adopted. Thus, if a paper was fundamentally statistical in nature (i.e., resulted in p-values, estimates of parameters of sampling distri- bution, or focused upon descriptive indices) and did not belong to one of the other areas

Requests for reprints should be sent to Mark I. Appelbaum, The L. L. Thurstone Psychometric Labora- tory, University of North Carolina, Davie Hall 013A, Chapel Hill, NC 27514.

0033-3123/86/0300-F001 $00.75/0 53 © 1986 The Psychometric Society

54 PSYCHOMETRIKA

assigned in this symposium, the paper was included in the domain of "statistics in Psycho- metrika." This decision rule forced the unhappy exclusion of papers such as Ramsay's 1969 "Some Statistical Considerations in Multidimensional Scaling," Andersen's 1973 "A Goodness of Fit Test for the Rasch Model," Hakstian and Whalen's 1976 "A k-sample significance test for independent alpha coefficients," to name but a few. These omissions are painful for in several cases they represent some of the most innovative uses of statis- tical techniques and ones which illustrate one of the most valued functions of Psycho- metrika to the behavioral statistician--the integration of distinct subareas and the adap- tation of statistical methods to new areas of inquiry.

In one case, however, I could not resist including a few papers which by this rule should have been assigned to another category--namely some of the early papers in the Analysis of Covariance Structures, for example that by Bock and Bargrnann (1966). These papers are not only clearly statistical in nature (both motivation and method), but illus- trate another important feature of statistical ideas in Psychometrika---they don't remain neatly within a single content domain. Developments in one area often have major impli- cations for others. We are, I believe, a more integrated field than we sometimes recognize or choose to admit.

Having thus classified each paper that appeared in Psychometrika from 1960 to 1984, we find that a total of 173 papers meet the criteria of being a "statistical paper." This is an average yearly rate of seven papers per volume, with statistical papers of this type ac- counting for about 15 percent of all papers published in Psychometrika during that period.

Might these have just as well been 173 papers appearing in JASA or any other of the numerous statistical journals? Individually, perhaps so, for each (or at least most) is of the technical quality one usually associates with journals such as JASA. But taken together they exhibit a set of features which make them distinctly "our own."

First let us consider the breakdown of these 173 papers into some general topical areas. In Table I we see a fairly typical set of statistical topics. However, certain issues seem to appear somewhat more frequently than one might expect in other statistical journals of the same period. Perhaps the most obvious of these is the very great frequency of papers dealing with correlation and association, in general, as well as specific indices of correlation such as the biserial and tetrachoric. We also observe a high incidence of papers on repeated measures, the use of extreme group designs, and the general topic of multiple correlation and regression. This topical breakdown reflects, I believe, both our concern with issues typical of the behavioral sciences as well as our common heritage in psychometrics and factor analysis.

The special nature of statistical papers appearing in Psychometrika cannot be ap- preciated, however, from simply arraying the content areas into which they might be classified. It is the nature of the content which is vital. To this end I would like to consider very briefly five papers which have appeared during a short ten year period. These, I think, illustrate some of the unique features of statistical papers in Psychometrika. Jack Carroll's (1961) paper, "The Nature of the Data, or How to Choose a Correlation Coef- ficient," should still, twenty-five years later, be required reading in every psychology de- partment. It reflects the need for a close relationship between data and statistics which is, or should be, the hallmark of the behavioral statistican--particularly with the wide- spread availability of the all-too-easy statistical analyses offered by "friendly" statistical packages. Meredith's (1964) paper, "Canonical Correlation with Fallible Data," illustrates the blending of traditional statistical thinking and the psychometric model - -an approach which has not yet been fully developed. Cleary's (1966) paper, "An Individual Differences Model for Multiple Regression," exemplifies the concern for individual differences as a focus of study rather than as unwanted error variance. This theme is continued in Winer's (1968) paper, "The Error," with a thoughtful consideration of error as something more than that which needs to be normally distributed with mean zero and constant variance.

MARK I. APPELBAUM 55

T a b l e 1

TOPICAL COVERAGE OF PSYCHOMETRIKA "STATISTICAL" PAPERS 1960-84

General Statistics (38)

Hypothesis testing (and related issues) 15 Non-parametric tests, (rank, randomization, e t c . ) 11 Distributional theory and/or sampling distributions 6 Data generation 3 Matrix theory 2 Computational "aids" 1

Correlation and Association (38)

General considerations 22 (including special indices not below)

Restriction of range, truncation, censoring 5 Tetrachoric 5 Biserial/point biserial 4 Part and partial 2

A_NOVA (24)

General theory (GLM, LS, etc.) Ii Repeated measures 9 Special application 2 Multiple comparison 1 ANACOVA 1

Multivari,ate Techniques (23)

Canonical correlation i0 Classification 6 Principal components 3 Profile analysis 2 Discriminant analysis 1 General theory 1

Continsency Table Issues (17)

Multiple Correlation/Resression (16)

General theory 11 Selection of variables 3 Special applications 2

Experimental Design (9)

Extreme groups 5 M a t c h i n g 4

C o v a r i a n c e S t r u c t u r e s * (3)

Robus t Methods (3)

Chan_ 8e (2)

T o t a l 173

* p a p e r s a p p r o a c h i n g c o v a r i a n c e s t r u c t u r e s f rom a d e c i d e d l y s t a t i s t i c a l a p p r o a c h

56 PSYCHOMETRIKA

Finally, Gollob's (1968) paper, "A Statistical Model which Combines Features of Factor Analytic and Analysis of Variance Techniques," represents the continuing theme of inte- gration of methods developed rather separately by workers within our many subfields.

Changes From the First Twenty-Five Years A few observations seem pertinent on the last 25 years of Psychometrika in compari-

son with the first twenty-five. First there has been almost a 50% increase in the number of statistical papers during the last 25 years, coupled with a dramatic decline in the necessity for dealing with computational aspects of statistical techniques. During the first 25 years nearly one third of all statistical papers published in Psychometrika focused primarily upon computational routines, tables, or nomographs. In the last 25, only one out of 173 papers had such a focus. Second, there has been a steep decline in the "psychological" content of papers in Psychometrika. While the papers still reflect our particular concern with issues that arise in the behavioral sciences, it is less clearly reflected in the content of our papers than it was during the twenty-five years. Whether this reflects a change in our graduate training programs, in the technical demands of our field, or whatever leads to some interesting speculation.

And finally, a few facts, probably trivial, that may be worth mentioning. The first paper appearing in Psychometrika dealing with non-desk calculator methods of compu- tation appears to be Tucker's 1940 paper "A Matrix Multiplier," which shows how to use an IBM Scoring Machine to multiply matrices. The statistical issue with the longest run- ning history in Psychometrika appears to be the computation of the tetrachoric corre- lation coefficient.

As to the future of statistical papers in Psychometrika, who but those of you currently writing and reviewing manuscripts can know? I see no reason, however, to expect major changes from the themes we have seen over the past twenty-five years of statistical papers: namely detailed attention to problems which arise in the behavioral sciences, a continued blending of the statistical and psychometric model, and a fascination with the inter- relationships among our various techniques.

References

Andersen, E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38, 123-141. Bock, R. D., & Bargmann, R. E. (1966). Analysis of covariance structure. Psychometrika, 31, 507-534. Carroll, J. B. (1961). The nature of the data, or how to choose a correlation coefficient. Psychometrika, 26,

347-372. Cleary, T. A. (1966). An individual differences model for multiple regression. Psychometrika, 31, 215-224. Gollob, H. F. (1968). A statistical model which combines features of factor analytic and analysis of variance

techniques. Psychometrika, 33, 73-116. Green, B. (1973). A letter from the Editor. Psychornetrika, 38(1), v-vi. Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psycho-

metrika, 41, 219-232. Meredith, W. (1964). Canonical correlations with fallible data. Psychometrika, 29, 55-66. Ramsay, L O. (1969). Some statistical considerations in multidimensional scaling. Psychometrika, 34, 167-182. Tucker, L. R. (1940). A matrix multiplier. Psychometrika, 5, 289-294. Winer, B. J. (1968). The error. Psychometrika, 33, 391-404.