21
http://jca.sagepub.com/ Journal of Career Assessment http://jca.sagepub.com/content/3/4/391 The online version of this article can be found at: DOI: 10.1177/106907279500300410 1995 3: 391 Journal of Career Assessment David P. Campbell The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution Published by: http://www.sagepublications.com can be found at: Journal of Career Assessment Additional services and information for http://jca.sagepub.com/cgi/alerts Email Alerts: http://jca.sagepub.com/subscriptions Subscriptions: http://www.sagepub.com/journalsReprints.nav Reprints: http://www.sagepub.com/journalsPermissions.nav Permissions: http://jca.sagepub.com/content/3/4/391.refs.html Citations: What is This? - Jun 1, 1995 Version of Record >> at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013 jca.sagepub.com Downloaded from

The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution Journal of Career Assessment 1995 Campbell 391 410(1)

Embed Size (px)

DESCRIPTION

The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution

Citation preview

Page 1: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

http://jca.sagepub.com/Journal of Career Assessment

http://jca.sagepub.com/content/3/4/391The online version of this article can be found at:

 DOI: 10.1177/106907279500300410

1995 3: 391Journal of Career AssessmentDavid P. Campbell

The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution  

Published by:

http://www.sagepublications.com

can be found at:Journal of Career AssessmentAdditional services and information for    

  http://jca.sagepub.com/cgi/alertsEmail Alerts:

 

http://jca.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

http://jca.sagepub.com/content/3/4/391.refs.htmlCitations:  

What is This? 

- Jun 1, 1995Version of Record >>

at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from at UNIV OF NORTHERN COLORADO on October 3, 2013jca.sagepub.comDownloaded from

Page 2: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

391

The Campbell Interest and Skill Survey (CISS):A Product of Ninety Years of

Psychometric EvolutionDavid P. Campbell

Center for Creative LeadershipColorado Springs, CO

This article describes the historical evolution of the CampbellInterest and Skill Inventory (CISS; Campbell, Hyne, & Nilsen,1992), beginning with Alfred Binet’s work in 1900-1905 in developingstandardized methods for psychological assessment. It extendsthrough the CISS’s early roots in the publication of the StrongVocational Interest Blank for Men by Strong in 1927, with Strong’sinnovative use of the Men-in-General concept to create empiricalscales, continuing with Strong’s publication of a similar form forwomen in 1933. It continues further with the addition of the

homogeneous Basic Scales in the 1960s and then with the merger ofthe Men’s and Women’s form in 1974 in the publication of theStrong-Campbell Interest Inventory, which also organized the profileinto the occupational conceptual theory suggested by Holland, andculminating in the publication of the CISS in 1992, which retainedthe concept of empirical Occupational Scales and homogeneousBasic Scales, but which also used an improved item format, addeditems to assess Skills, slightly modified and extended the Hollandtheoretical structure and used more sophisticated computer graphicsto create a user-friendly report form. A brief overview of thecharacteristics of the CISS is also presented.

Overview of the CISSThe CISS is a career planning survey that asks respondents to assess their

degree of interest in 200 academic and occupational topics and their degreeof skill in 120 specific occupational activities. The respondent’s answers arescored with standardized scales and an 11-page profile (description following)is produced; an additional 2-page Report Summary is provided for thecounselor.

The profile contains three kinds of scales, analyzed conceptually inincreasing levels of detail:

1. Seven Orientation Scales, which are Influencing, Organizing,Helping, Creating, Analyzing, Producing, and Adventuring.These Orientations, and their combinations, are designed to mapthe entire occupational world.

Published and copyright @ 1995 by Psychological Assessment Resources, Inc. All rights reserved.

Page 3: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

392

2. Twenty-nine Basic Scales, such as Law/Politics, Counseling, andMathematics. These homogeneous scales, which are categorizedunder the Orientation Scales, provide more precise guideposts.

3. Sixty Occupational Scales, such as Attorney, Engineer, GuidanceCounselor, and Math-Science Teacher. These Occupational&dquo;marker&dquo; scales, also categorized under the Orientation Scales,help respondents understand the specific areas of the occupa-tional world that are most likely to feel appropriate for them.

In addition, Two Special Scales are provided: the Academic Focus Scale,which helps respondents understand how comfortable and successful theyare likely to be in a formal, traditional academic setting and the ExtraversionScale, which reflects the amount and intensity of interpersonal interactionsthat respondents are likely to seek in their daily working life.Two scores are provided for each scale: an Interest score, which reflects

the respondent’s degree of interest in the designated area and a Skill score,which reflects the respondent’s level of confidence in performing well in thesame area. Noting which pairs of scores are high, which pairs are low, andwhich pairs show sizeable discrepancies between the two scores areimportant learning points in the interpretation process.

Several narrative paragraphs are produced for each respondent, summarizingthe patterns within their scores, and three Procedural Checks are calculatedfor each answer sheet to detect possibly distorted response patterns.

Genealogy of the CISSThe CISS has one of the longest, most distinguished lineages of any

currently used psychological survey. Its features have been hammered outover 65 years, beginning with the publication of the Strong VocationalInterest Blank (SVIB; Strong, 1927), extending through the publication ofthe Strong-Campbell Interest Inventory (SCII; Campbell, 1974) andculminating in its current form in the publication of the CISS.

Following, in outline form, are the major events in this historical stream.This sequence is important in that it highlights four major themes runningthrough the development of the SVIB/SCII/CISS.

1. The sequential evolution of the item content and format. Thisapproach has provided a series of rolling revisions that has keptthe item content up-to-date and led to periodic improvements inthe item format.

2. The evolution of the scoring and norming algorithms. Thesechanges have improved the scale standardization process, whichdirectly impacts how the respondent’s answers are empiricallyanalyzed and interpreted.

3. The conceptual theories underlying the CISS. These theories haveadded a great deal of explanatory power to the meaning of theempirical scores.

4. The practical experience that has been accumulated by practi-tioners. From this collective wisdom has evolved an implicit set ofguidelines about how, when, where, and with whom such surveyscan profitably be used.

Page 4: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

393

With only a slight stretch, the genealogy of the CISS can be traced backto a Frenchman, Alfred Binet, who is generally considered to be the Fatherof Psychological Testing. The major milestones are as follows:

Circa 1900-Binet was asked by the Paris School Board to develop amethod for identifying children who could not profit from the usual classroominstruction-essentially slow or disadvantaged learners. For this project, hepioneered two techniques: first, asking children to solve familiar problemslike &dquo;Here is a clock, what time is it?&dquo; and second, norming the responsesby finding out at what age the typical child learns to tell time. With hisactions, psychological testing was born. These two features-standardizeditems and normed responses-are still characteristic of virtually allcontemporary psychological tests and surveys (DuBois, 1970).

1916-Binet’s techniques were brought to America by Professor LewisTerman of Stanford University in the publication of the Stanford-BinetIntelligence Test.1917-1920-The Binet/Terman techniques &dquo;went to war.&dquo; The Army

Personnel Classification Committee, using the techniques of standardizeditems and normed responses, developed the Army Alpha and Beta tests. Twoyoung officers on that committee were Edward K. Strong, Jr., and DonaldG. Paterson; they were to become lifelong friends. For an account of this workand a group portrait of the commissioned psychologists (showing Paterson,but not Strong), see Yerkes (1921).1920-D. G. Paterson arrived at the newly established Department of

Psychology at the University of Minnesota, where he would remain for 40 years.1923 -E. K. Strong arrived at the School of Business at Stanford

University, where he would remain until his retirement in 1949.

1927-Building on the work of a few earlier researchers, Strong publishedhis Vocational Interest Blank (VIB), containing 420 items, adopting theL-I-D (Like -Indifferent-Dislike) item format and pioneering empirical scalesby statistically comparing men in specific occupations with a referencegroup of Men-In-General. Eighteen occupational scales were scored andarranged in clusters that predated the eventual occupational structuresuggested by Holland (1973). The VIB was published by Stanford UniversityPress. Strong provided the only scoring service.

1933-Strong published a parallel form for women, containing 410 items.At some later point, the surveys became known as the Strong VocationalInterest Blank for Men (SVIB-M) and the Strong Vocational Interest Blankfor Women (SVIB-W).1938-The SVIB-M was slightly revised, with mild changes in item

content, and shortened to 400 items, because that was the maximum numberof items that would fit on an IBM 805 answer sheet. Scores for 38

occupational scales were provided. Strong continued to provide a scoringservice out of his office at Stanford for $2 per respondent. These fundssupported his research during the Depression.

1943-Strong published his massive tome, Vocational Interests of Men andWomen, reviewing his research results to date.

Page 5: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

394

1946-The SVIB-W was also updated and shortened to 400 items. The firstcommercial scoring company, TestScor, was founded in Minneapolis byElmer Hankes, using a one-of-a-kind mechanical scoring machine. For thefirst time, a graphic profile, primitive by today’s standards, was produced.1949-Strong retired from Stanford University, but continued to work on

his momentous longitudinal study of the interests and eventual careers ofthe Stanford students whom he had surveyed many years earlier.1955 -The University of Minnesota Press published Strong’s book,

Vocational Interests Eighteen Years After College, and The University ofMinnesota hosted a symposium to honor Strong. A committee was formedto oversee the revision and modernization of the SVIB-M. The two committeecochairmen were Professors of Psychology at The University of Minnesota:Ralph Berdie, Director of the University’s Student Counseling Bureau (andStrong’s son-in-law) and Kenneth E. Clark, Chairman of the PsychologyDepartment.1958-David Campbell, a new graduate student, was hired to do the

committee’s data processing work.1960-D. G. Paterson, by now a prominent Professor of Psychology, a

longtime E. K. Strong friend, and an ardent advocate for his methodology,retired and passed away 4 months later. Campbell was Paterson’s lastresearch assistant.

1962-The University of Minnesota established the Center for InterestMeasurement, largely to receive Strong’s treasure trove of archival data.Campbell, now an Assistant Professor of Psychology, was appointed as itsfirst director. National Computer Systems was founded in Minneapolis byHarlan Ward; modern digital computers were used in the test scoringprocess and a more professional looking profile was produced. Althoughapproximately 50 occupational scales were now scored, the scoring chargedropped to about $1.00, even less for large volumes.1963-E. K. Strong passed away in Palo Alto, California.1966-The revised SVIB-M was published under the authorship of Strong,

Campbell, Berdie, and Clark. The Basic Interest Scales were added to theprofile to accompany the Occupational Scales. The item content wasmodernized, but the L-I-D format was retained. The SVIB-M was printed onblue paper (Campbell, 1966).1968-Campbell prepared a brief history of the SVIB to this point; with

unintended foresight that proved to be accurate but naive, he wrote:The history of a psychological test is not a thrilling saga; there

have been no gatherings in dusty squash courts to see if the gadgetwill really work, no agonizing waits to see if the graft or transplantsurvives, no tension ridden countdowns. Like most science, it hasbeen a succession of drab and trivial activities with only anoccasional significant statistic to brighten the day....

Lively times have not been entirely absent; any research withpeople creates its own sparks, whether angry letters fromparticipants, friendly testimonials from the audience, orprofessional disagreements over the philosophy, techniques, or-

Page 6: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

395

not the least-finances of the operation. The 1966 revision of theMen’s SVIB involved two major investigators separated by fiftyyears in age and training, two others, more tangential but withbright, innovative and occasionally disruptive ideas, a family whofeels justifiably possessive about the inventory, a university presspublisher who values quality over profit, commercial distributorswith somewhat different feelings, four scoring services with avariety of approaches, and a host of professionals who, from yearsof personal use, feel the Strong is in some sense &dquo;their&dquo; test. Withthese diverse outlooks and in the absence of corporation-chartlines of command and communication, the pot boiled occasionally.

However, for the most part, things have been staid and most of theissues have been no more controversial than &dquo;What color paper shallwe use?&dquo; (Campbell, 1968, reprinted in Campbell, 1971, p. 343)

1968-The revised SVIB-W was published under the authorship of Strongand Campbell. It was printed on pink paper, which shortly led to the mostcolorful public relations fiasco in the history of psychological testing.

1971-Campbell’s Handbook for the Strong Vocational Interest Blankwas published, just in time to be made obsolete by social change.

1972-Frequent protests and investigations instigated by the women’smovement focused on the gender bias inherent in the separate, color-codedbooklets; class action lawsuits were threatened. (In a Doonesbury cartoonfrom that era, Joanie Caucus was briefed during her orientation to lawschool, &dquo;The blue form is for men, the pink form is for women.&dquo; She wasshown with her head in her hands in disbelief.) A merged gender revisionwas quickly planned.1974-The Strong-Campbell Interest Inventory (SCII) was published; it

was formed by taking the best items from the men’s and women’s booklets.Gender sensitivity was emphasized and, in addition, only those items thattranslated easily into other languages were retained; that is, Americanslang and idioms were eliminated. Serendipitously, this also eliminateditems likely to offend any cultural subgroup within the U.S.A. Men andwomen now responded to the same items, but they were still compared todifferent norms. In a major parallel change, the profile was reorganizedinto the theoretical occupational structure suggested by John Holland(Campbell, 1974; Campbell & Holland, 1972; Hansen & Johansson, 1972;Holland, 1973).

1981, 1985-Extensions of the SCII were prepared, largely supervised byProfessor Jo-Ida Hansen of The University of Minnesota (who had beenappointed as the second Director of the Center for Interest MeasurementResearch when Campbell moved to the Center for Creative Leadership). Themajor improvements were: (a) a more balanced gender composition of theGeneral Reference Sample, (b) an expansion of the profile coverage toinclude more blue-collar occupations, (c) a concerted effort to provide bothfemale and male scales for almost all of the occupations, and (d) an increasein the average size of the occupational samples (Campbell & Hansen, 1981;Hansen & Campbell, 1985).

Page 7: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

396

1983-Stanford University Press attempted to turn control of the SCIIover to a commercial publisher, including management of the R & D function.

Campbell protested, citing the author-publisher contract. Stanford alsoreminded Campbell of the noncompete clause in that contract, forbidding himto work on any similar inventory.1987-The Ninth Circuit Court of Appeals in San Francisco ruled that

Stanford’s actions &dquo;constitute a prohibited transfer,&dquo; but failed to establishdamages or to resolve the noncompete dispute. (See Campbell v. The Boardof Trustees of the Leland Stanford Junior University, No. 85-1678, 9thCircuit. 1987, p. 3.)1988-In an out-of-court settlement, Campbell and Stanford University

Press &dquo;got divorced&dquo;; Stanford took over all rights to the inventory andrenamed it the Strong Interest Inventory (SII). Campbell took back allrights to his name.1992-The Campbell Interest and Skill Survey (CISS) was published,

expanding on the earlier work by adopting a more modern item pool and amore flexible item format. For the first time, skill items were added to thebooklet, all scales were standardized on the same population, and both sexeswere scored and normed in exactly the same manner. The Holland theoreticalstructure was slightly modified and expanded (Campbell et al., 1992).

This lengthy succession of events has led to a new career planning surveythat has the dual strengths of a productive history and modern dataprocessing techniques. The specific characteristics of this survey follow.

The CISS ItemsDecisions regarding item content and format are crucial in a new survey

because they completely determine the type and quality of data that can becollected with the survey.

In publishing his initial form, Strong was marvelously experimental inhis choices of both item content and item format. He had a wide range ofcontent categories: Occupations (e.g., &dquo;Actor,&dquo; &dquo;Social Worker,&dquo; &dquo;Wholesaler&dquo;);Amusements (e.g., &dquo;Auctions,&dquo; &dquo;Fortune tellers,&dquo; &dquo;Pet monkeys&dquo;); SchoolSubjects (e.g., &dquo;Algebra,&dquo; &dquo;Civics,&dquo; &dquo;Philosophy&dquo;); Activities (e.g., &dquo;Pursuingbandits in a sheriff’s posse,&dquo; &dquo;Repairing a clock,&dquo; &dquo;Teaching children&dquo;);Peculiarities of People (e.g., &dquo;Blind people,&dquo; &dquo;Fashionably dressed people,&dquo;&dquo;People who get rattled easily&dquo;); and Ratings of Present Abilities (e.g.,&dquo;Plan my work in detail,&dquo; &dquo;Usually start activities of my group,&dquo; &dquo;Winfriends easily&dquo;).One might wryly note that political correctness had little impact on

Strong’s first item pool; some of the more startling items were: &dquo;Gruff men,&dquo;&dquo;Negroes,&dquo; &dquo;Men who use perfume,&dquo; and &dquo;Women cleverer than you.&dquo;Curiously, the item &dquo;Negroes&dquo; was removed from the Men’s Booklet in 1938,but remained on the Women’s Booklet until 1969.

These items used the familiar L-I-D format, except the last group, whichused a Yes-?-No format.

In addition, Strong included two forced-choice formats, one whererespondents were asked whether they preferred the item on the left, the one

Page 8: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

397

on the right, or both equally (e.g., &dquo;Street car motorman&dquo; vs. &dquo;Street carconductor&dquo;). In the second forced-choice format, 10 items were listed andrespondents were asked to indicate which three they liked most andwhich three they liked least. One set contained names of famous people,such as &dquo;Enrico Caruso, singer,&dquo; &dquo;Thomas A. Edison, inventor,&dquo; and &dquo;J. J.Pershing, soldier.&dquo;This range of content and format provided rich comparative data

throughout the years as Strong meticulously surveyed occupational sampleafter sample. Of course, he was handicapped by the lack of data processingfacilities; he never used anything more complicated than a mechanicaladding machine. Indeed, most of his analyses were done laboriously byhand with lead pencils on lined accounting tablets.

Further, Strong made two procedural mistakes early on that were toplague his item research for the rest of his career. First, in his initialbooklet (1927), he neglected to number the items, arranging some of themin alphabetical order, but others merely randomly; and second, in his firstrevision (1938), he changed the order of the retained items. Only researcherswho have tried to study item characteristics over time within a changingitem bank or who have tried to use scoring templates developed at onetime with booklets developed at another time can understand the distressingimplications of those decisions. Even with computers, such complicationsare challenging.

Strong bemoaned these problems until his death. One of his firm chargesto those of us working on the 1960s revisions was, &dquo;Whatever else you do,do not change the order of the retained items!&dquo;

For serious scholars, these earlier forms have been reproduced in theAppendix of the Handbook for the Strong Vocational Interest Blank(Campbell, 1971).During the late 1950s and early 1960s, Strong’s original data were

transferred to IBM cards at The University of Minnesota, and for the firsttime, a wide range of comparisons was possible at both the item and scalelevel. Sadly, Strong passed away during these projects; he never lived to seethe results. With his intense reverence for data, he would have beenfascinated.

The item analyses were illuminating. They indicated that, although arange of content is desirable, some categories are better than others inproducing scales that will discriminate between occupational samples. Forexample, Occupations items are more potent than either Amusements orPeculiarities of People items. These analyses also suggested that so-called&dquo;subtle&dquo; items are not very useful. Future decisions about which items toretain and which to discard closely followed the lessons learned from theseearly item analyses.As for format, although the three-choice L-I-D format worked adequately,

the frequency distributions often showed grossly unequal pileups ofrespondents in one or another of the choices, suggesting that more choiceswould be desirable. Additionally, through the years, a small but notablenumber of SVIB respondents have commented, &dquo;Some occupational activities

Page 9: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

398

I like, and some I REALLY LIKE, and this (L-I-D) format does not allow meto express that difference.&dquo;

The forced-choice formats also proved troublesome, especially the &dquo;Pickthe three most and three least&dquo; format, as some respondents had troublefollowing directions; consequently this format was dropped in the 1974publication of the SCII.

All of the previous editions, revisions, and extensions of the SVIB and SCIIhave retained the L-I-D format. Because of the rolling waves of revisions,each of which drew on earlier normative data, to change would have beendisastrous; it would have meant starting all over.

The CISS Item Format

In the development of the CISS, when necessity created both the luxuryand the burden of starting over, an early decision was made to adopt amore flexible and more user-friendly item format. After considerableexperimentation, a 6-point response scale was chosen; it has worked well,both statistically and in respondent reactions. For example, this moreprecise item format has permitted the use of shorter scales with no apparentloss in reliability.

The CISS Item Content

The CISS item pool was constructed by paying close attention to thelessons learned from the earlier SVIB/SCII research. Occupationally orienteditems were favored, more modern items were included, and any items thathad an outdated tinge were avoided. Items were excluded that were likelyto become dated, such as proper nouns, and that might be insensitive to anydemographic category such racial, gender, social class, ethnic, physical, orgeographic groupings.Apparently these efforts have been successful; in the 3 years since the

CISS was published, there has not been a single complaint aboutinappropriate item content.

The Addition of Skill Items to the CISS

By far, the biggest conceptual improvement has been the addition ofitems to assess self-reported skills. This was done for two reasons: first, asany experienced counselor knows, a certain percentage of respondents,when shown their interest profiles, react by saying something like, &dquo;Well,maybe I scored high in those areas, but I certainly don’t have any talentthere,&dquo; which leaves the counselor with the necessity of reconciling aconfusing pattern of self-assessment. This common situation suggestedthat a standardized system for measuring self-assessed skills might beuseful. Second, in informal experiments over several years in career planningsessions, several hundred people were asked to evaluate their skills usinga variety of experimental checklists. The preliminary results suggestedthat individuals are quite capable of realistically assessing their skills andthat their responses are related to occupational membership. For example,Army officers reported a different pattern of skills than did life insurancesales personnel.

Page 10: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

399

After several false starts in developing the skill scales, a useful approachfinally emerged, essentially mimicking Binet’s techniques of standardizeditems and normed responses. The combination of the interest and skillscales has proven effective, although experience has shown that the Skillscores are more appropriately interpreted as measures of confidence ratherthan as empirically accurate measures of true skill. High scores indicate thatrespondents are confident of their ability to perform well in the designatedareas, and a sense of self-efficacy has been shown time and again to be aneffective force. Thus, the Skill scores provide a useful reflection of theindividual’s pattern of self-confidence.

The CISS Scales

The Orientation Scales

The conceptual framework used for the CISS consists of sevenOrientations, each covering a specified area of occupational content. TheseOrientations, which generally resemble the major dimensions that have beenused for years to stratify the occupational world, were identified throughfactor analysis and other statistical procedures. They generally resemblethe Holland General Occupational Themes formerly used on the SCII(Hansen & Campbell, 1985; Holland, 1973). The following few differencesbetween the Holland themes and the formulation used here are noted.

Influencing-This Orientation involves leading and influencing otherpeople. High scorers are interested in making things happen. They want tobe in charge, and they are willing to accept responsibility for the results.Influencers are confident of their ability to persuade others to theirviewpoints, and they enjoy the give and take of verbal jousting. Theytypically work in organizations and want to be responsible for the specificactivities within the organization that particularly interest them. Theyenjoy public speaking and like to be visible in public settings. Typical highscoring occupations include company presidents, corporate managers, andschool superintendents.The Influencing Orientation resembles the Holland Enterprising theme,

except that Holland’s theme is rather more focused on sales and marketing,whereas the Influencing Orientation is tilted more toward leadership and politics.Organizing-The Organizing Orientation includes activities that bring

orderliness and planfulness to the working environment, such as managingprojects, planning systems, overseeing budgets, and directly supervising thework of others. People who score high emphasize efficiency and productivity.Organizers are good with details and enjoy solving the day-to-day problemsthat inevitably appear in organizations. They understand cost effectivenessand cash flow and are often good in planning and overseeing investments.Typical high-scoring occupations include accountants, financial planners,office managers, and administrative assistants.

The Organizing Orientation resembles Holland’s Conventional theme,except that the Holland theme is more focused on office practices, whereasthe Organizing Orientation has a stronger flavor of financial services andinvestment activities.

Page 11: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

400

Helping-The Helping Orientation involves helping and developing othersthrough activities such as teaching, counseling, or healing. People whoscore high are compassionate and deeply concerned about the well-being ofothers. Helpers enjoy having close personal contact with those for which theyare responsible and are genuinely concerned with helping their students,clients, or patients lead full, satisfying lives. They readily understand thefeelings of others and can provide emotional support. Typical high-scoringoccupations include counselors, teachers, and religious leaders.The Helping Orientation is quite similar to the Holland Social theme.

Creating-The Creating Orientation includes artistic, literary, and musicalactivities, such as writing, painting, dancing, working in the theater, anddesign activities such as interior design and fashion design. People who scorehigh are interested in, and confident of their ability to create, new products,new visions, and new concepts within these artistic areas. Creators seethe world through innovative lens and are often uncomfortable withorganizational constraints. They see themselves as free spirits and arefluent and expressive. Typical high scoring occupations include artists,musicians, designers, and writers.The Creating Orientation is quite similar to the Holland Artistic theme.

Analyzing-The Analyzing Orientation includes scientific, mathematical,and statistical activities. People who score high are comfortable withnumbers and data, and they have a strong need to understand the world inan analytic sense. They prefer to work alone or in small groups in laboratoryor academic settings. Analyzers have a strong need to be autonomous andlike to work through problems for themselves. Typical high scoringoccupations include chemists, medical researchers, statisticians, and otherscientists.

The Analyzing Orientation is quite similar to the Holland Investigativetheme.

Producing-The Producing Orientation covers practical, hands-on,&dquo;producing&dquo; activities, such as construction work, farming, and mechanicalactivities. People who score high like to work with their hands, enjoy beingoutdoors, and like to be able to see the visible results of their labors.Producers are usually good with tools, and they enjoy taking on newconstruction projects or repairing mechanical breakdowns. Typical high-scoring occupations include mechanics, veterinarians, and landscapearchitects.

For comparison with the Holland themes, see the comment following theAdventuring Orientation.Adventuring-The Adventuring Orientation covers athletic, police, and

military activities involving physical risk-taking, endurance, and competingwith others. High scorers enjoy physical activities, and they like to confrontcompetitive situations. They are confident of their physical skills and oftenseek out excitement. Adventurers enjoy winning, but they also are resilientin defeat. They often like working closely with others in teams. Typicalhigh-scoring occupations include military officers, police officers, andathletic coaches.

Page 12: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

401

In the CISS, the Holland Realistic theme has been separated into theProducing Orientation-which is focused on mechanical, construction, andoutdoor activities-and the Adventuring Orientation-which is focused onphysical risk-taking activities, such as police, military, and athletic activities.Although the two Orientations are related, both statistical analyses andcommon sense suggested that they be separated.A further difference between the Holland and CISS formulations is that

verbs, or action words, have been used to name the CISS Orientationsinstead of the adjectives used in the Holland system. The rationale is thatthese Orientations are a reflection of the ways that individuals prefer tointeract with the world and, as such, action words are a better way tocapture their flavor.

The Basic Scales

A set of Basic Scales has been developed by clustering together itemsreflecting homogeneous content areas, such as Financial Services,Counseling, and Mechanical Crafts; these scales are more detailed than theOrientations and this approach essentially provides a set of subscales foreach Orientation. The Basic Scales, clustered by Orientation, are listed onthe CISS profile, which is reproduced in Figure 1. The content of eachscale is captured by its scale name, and its interpretation is quite straightforward. High scores indicate an attraction for and confidence in performingwell in the specified activities; low scores reflect aversions and a lack ofconfidence for the designated activity.

The Occupational ScalesThe Occupational Scales were developed by surveying a sample of

successful, satisfied workers in each occupation and comparing each sample,item-by-item, with a general reference sample of employed individualsfrom a wide variety of occupations. Items showing a substantial differencein response patterns between the occupational and reference samples areused as the occupational scoring scale.The CISS Occupational Scales are clustered under the appropriate

Orientation according to the occupational sample’s mean scores. An exampleof this is shown in Figure 2, where the page showing the InfluencingOrientation scores has been reproduced.

The Special ScalesTwo Special Scales have been included on the CISS because they represent

areas central to the career planning process: the Academic Focus scale,which reflects the individual’s interest and confidence in purely academicpursuits, especially those activities related to science and the arts and theExtraversion scale, which reflects the individual’s interest in and confidencein highly personal and interactive social activities in the work place.

The CISS Norming ProcessThe evolution of the CISS norming process is an excellent example of how

improvements have been introduced over the years. Strong had the early

Page 13: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

402

brilliance to recognize the benefits of standard scores, and he was one of thefirst test authors to introduce the now familiar concept of T scores, that is,standard scores where a defined sample has a mean of 50 and a standarddeviation of 10. In 1927, when Strong was developing his initial empiricalOccupational Scales, he used each occupational sample to set the norm forthat scale, a revolutionary breakthrough at the time. This novel, creative,and effective standardization method did lead to a peculiar situationwhereby each occupational scale was normed on a different sample.

Forty years later, in 1966, a set of Basic Scales was added to the SVIBProfile for Men (and to the Women’s Profile in 1969); these scales weredeveloped by using the clustering technology suggested by Clark (1961). Toamend the &dquo;different normative samples situation,&dquo; the Basic Scales werestandardized by using the same Men-In-General sample (and then Women-In-General sample) to establish the normative mean and standard deviationfor each scale. Unfortunately, in a decision that lacked a certain amount offoresight, these figures were set at 50 and 10, respectively, just as with theOccupational Scales, which led to yet another peculiar situation whereby thenorms for the Occupational Scales (M = 50, SD = 10) were based on specificoccupational samples and the norms for the Basic Scales (also M = 50,SD = 10) were based on the General Reference Sample, which indicated thata score of 50 meant quite different things for different scales.Although professional users quickly adapted to these norming methods,

which were regrettably repeated in the 1974 publication of the SCII, theseprocedures created an unnecessarily complicated situation. Counselors hadto understand that an individual who scored 50 on an Occupational Scalewas scoring at the occupational mean for that scale, whereas an individualwho scored 50 on a Basic Scale was scoring at the general population mean,and then the counselor had to pass this information along correctly, inlayperson terms, to the client. To further complicate matters, in aconservative decision based on the wish to proceed slowly with radicalchange, the norms in 1974 were kept separate for men and women.These procedures also complicated the research process as scores on

different types of scales could not be directly compared, nor could scoresfor men and women be directly compared, even on identically namedscales.

To avoid these peculiarities in the CISS, the Orientation, Basic, andOccupational Scales have all been normed on the same reference sample sothat a score of 50 means the same thing over all scales. Additionally, thegenders have been combined in the norming process, mainly by assuringthat, in both scale construction and scale norming, gender differences wereminimized and occupational differences were maximized. The main benefitof these changes is that a standard score of 50 means precisely the samething over all scales and over all respondents. A longer discussion of thesechanges can be found in Campbell et al. (1992).

Still, the major point is that Strong’s original standardization method hasbeen retained in concept over the years, but with more precision in itscurrent applications. Future authors/researchers will inevitably turn up newpeculiarities, and the evolutionary process will undoubtedly continue.

Page 14: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

403

After 65 years of efforts resulting in literally thousands of relevantstudies, we can now say with confidence that scales developed with theabove procedures validly capture characteristic patterns of occupationalinterests and that people who score high on any given occupational scale willtend to enter, be satisfied, and remain more often in that occupation thanwill people who score low. Further, the scores are stable over time, especiallyfor adults, thus permitting their use in long-term planning.The same pattern of findings is appearing for the Skill items though

much less data are as yet available.

The Interest and Skill QuadrantsAfter the CISS normative work had been finished, a serendipitous burst

of insight appeared. When the two scores for each scale (i.e., the Interest andSkill scores) were compared, it became apparent that each scale couldusually be categorized into one of four patterns: High-High, High-Low,Low-High, and Low-Low. Further, each of these quadrants lent themselvesto a useful psychological interpretation as follows:

1. High interest and high skill scores-Pursue. The respondent hasexpressed both a strong preference for and a strong sense of con-fidence in performing well here. Definitely an area to be pursued.

2. High interest but lower skill scores-Develop. The respondent hasexpressed a strong preference, but less confidence in performingwell. In such cases, respondents should be encouraged to furtherdevelop their skills in this area or, failing that, to appreciate thisarea as an avocational focus.

3. Low interest but higher skill scores-Explore. The respondentis not particularly attracted to this area, but feels confidentabout performing well. In such cases, respondents should beencouraged to be creative in exploring how these skills might beused in applications that do interest them.

4. Low interest and low skill scores-Avoid. The respondent hasreported both aversion and lack of confidence in these activities,strongly suggesting that this area should be avoided in careerplanning.

In counseling sessions, most respondents seem attentive to thesecategories as the action-oriented labels gives them a sense of possibleactions to be taken.

The Procedural ChecksThe CISS has three routine scoring checks that are performed on each

answer sheet to detect possibly distorted patterns of responses.1. Omitted Items Check-If the respondent has omitted too many

items, the answer sheet is flagged.2. Inconsistency Check-If the respondent has given several

inconsistent pairs of responses, such as Strongly Like to &dquo;Artist&dquo;and Strongly Dislike to &dquo;Studying Art,&dquo; the answer sheet isflagged.

Page 15: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

404

3. Response Percentage Check-If the respondent has spread theitem responses over the six possible choices in a highly deviantway, such as answering every item Strongly Like, the answersheet is flagged.

Experience has shown that in a small but notable number of cases, thislast check can provide useful clinical information about the respondent’sframe of mind when responding to the booklet. When the responsepercentages are tilted substantially above the population base rate,suggesting perhaps a lack of contact with reality or, in contrast, are tiltedsubstantially below the population base rate, suggesting perhaps a sense ofdespair, even depression, the counseling session usually takes on a differenthue. A lengthier discussion of these dynamics can be found in the CISSmanual (Campbell et al., 1992).

The CISS Individual ReportThe CISS Report, an 11-page document, has been designed graphically for

easy understanding. The counselor’s main task is not to interpret thescores, which are usually self-explanatory, but rather to help respondentsunderstand their scores within the context of their personal situations.

Three pages are reproduced here for illustrative purposes; the first is theso-called &dquo;profile,&dquo; which reports both Interest and Skill scores for theseven Orientations and 29 Basic Scales (Figure 1); the second is the pageshowing the scores for the Influencing Orientation (Figure 2). Analogouspages for the other six Orientations are also prepared for each respondent,but are not included here because of space limitations. The third pageshows the Special Scales and the Procedural Checks (Figure 3). A two-page Report Summary, containing all of the scale scores in a condensedformat, is produced for the counselor, but, again, is not reproduced herebecause of space limitations.

ApplicationsHistorically, career surveys have been used in high-school and college

career planning sessions, and the CISS is designed for those purposes.Additionally, over the last 3 decades, career planning sessions have movedinto adult corporate and public service settings, usually within the humanresources function for specialized applications, such as outplacement orreorganizational situations created by mergers, leveraged buyouts,reengineering, or downsizing operations. When occupational turbulence ishigh, a well-planned career can be a useful anchor.The CISS is generally used in one of three major ways:

1. In one-on-one counseling settings. The individual’s profile isreturned in the context of a longer discussion about the indi-vidual’s personal situation. Other psychological tests or surveysare often used for a more complete assessment, with the goalbeing to help individuals think through their next educationalor occupational actions. Such sessions can last anywhere fromone to several hours.

Page 16: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

405

Page 17: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

406

Page 18: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

407

Page 19: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

408

2. In group sessions. Many organizations now use group sessionswhere individual profiles are returned to several people in aclassroom setting. This approach is more efficient in terms ofcounseling resources and also can provide a broader perspectiveto the respondents, as they can observe how others handle theircareer planning decisions and can perhaps exchange careerplanning strategies. Group sessions are often more informative,and perhaps more enjoyable, than one-on-one sessions.

3. In team building sessions. These are also group sessions, but aspecific kind of group session in that everyone in attendance isfrom the same working group. Thus, colleagues have the oppor-tunity to see the variability of vocational interests and skillsreported by their colleagues. With skilled facilitation, thisapproach can encourage the development of a feeling of cama-raderie, and thus help team members achieve a higher levelof trust, empathy, and understanding within their workingrelationships.

Each of these approaches is described in greater length in the manual forthe CISS (Campbell et al., 1992).

A Retrospective LookEarlier on, the importance of four major themes in the evolution of

modern surveys was noted. Now, almost a century after Binet began hispioneering work, this importance stands out. These themes have impactedthe CISS as follows:

1. Item content and format. After experiments with a wide range ofitem content, the current pool is more occupationally oriented,more sensitive to cultural concerns, less susceptible to becomingdated, less apt to be offensive to any group of respondents, andnow contains skill items. After trials with a variety of formats,the advantages of the 6-point scale have led to its adoption.

2. Scoring and norming procedures. A combination of a strongeritem format and more precise scale construction techniques hasled to shorter, equally effective scales. Subtle items have beeneliminated. Men and women are now analyzed identically. Astandardized population mean of 50 and S’D of 10 have beenused for all of the scales, thus permitting direct comparisons ofscores over all scales, all individuals, and all groups.

3. Conceptual theories. From the early, crude groupings of occupa-tions by Strong, a sophisticated, comprehensive, theoreticalstructure of the world of work has emerged, based in large parton the work of John Holland. These theoretical advances haveled to a deeper understanding of what it means for an individualto report, say, artistic interests, suggesting in this case that theindividual is probably free-spirited, uncomfortable with organi-zational constraints, perhaps moody and impulsive, intrinsicallymotivated, and so forth.

Page 20: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

409

4. Accumulated practical experience. Literally hundreds of practicallessons have been learned about the use of psychological surveys,all the way from &dquo;How long should the survey be?&dquo;, &dquo;How can weprogram the scanner to distinguish between an erasure and acorrect response?&dquo;, and &dquo;How many days do we have to allow formail-in scoring?&dquo; to, ominously, &dquo;What color paper shall we use?&dquo;

Collectively, the benefits of these lessons have brought us far beyondthe work of Binet though, to be honest, he would still recognize theglimmerings of his approach in most contemporary surveys.

It might finally be noted that the CISS is one of the CampbellDevelopmental Surveys, the others being the Campbell OrganizationalSurvey (Campbell & Hyne, 1995), the Campbell Leadership Index (Campbell,1991), the Campbell-Hallam Team Development Survey (Hallam &Campbell, 1994), and the Campbell Community Survey, which is in thefinal development stages. These surveys, which use the same item formatand scale norming procedures as the CISS, differing only in content andareas of application, have also been developed with the aid of the lessonslearned from the SVIB/SCII/CISS evolution. Because of this consistencyacross surveys, and with the emphasis on user-friendly graphics, userswho become familiar with any one of these surveys will immediately feelcomfortable with the others.

ReferencesCampbell, D. P. (1966). Manual for the Strong Vocational Interest Blank. Stanford, CA:

Stanford University Press.

Campbell, D. P. (1968). An informal history of the SVIB. In P. McReynolds (Ed.), Advancesin psychological assessment (pp. 343-365). Palo Alto, CA: Science and Behavior Books.

Campbell, D. P. (1971). Handbook for the Strong Vocational Interest Blank. Stanford,CA: Stanford University Press.

Campbell, D. P. (1974, 1977). Manual for the Strong-Campbell Interest Inventory. Stanford,CA: Stanford University Press.

Campbell, D. P. (1991). Manual for the Campbell Leadership Index. Minneapolis, MN:National Computer Systems.

Campbell, D. P., & Hansen, J. C. (1981). Manual for the Strong-Campbell Interest Inventory(3rd ed.). Stanford, CA: Stanford University Press.

Campbell, D. P., & Holland, J. L. (1972). A merger in vocational interest research: ApplyingHolland’s theory to Strong’s data. Journal of Vocational Behavior, 2, 353-376.

Campbell, D. P., & Hyne, S. A. (1995). Manual for the Campbell Organizational Survey (2nded.). Minneapolis, MN: National Computer Systems.

Campbell, D. P., Hyne, S. A., & Nilsen, D. (1992). Manual for the Campbell Interest andSkill Survey. Minneapolis, MN: National Computer Systems.

Clark, K. E. (1961). Vocational interests of nonprofessional men. Minneapolis, MN:University of Minnesota Press.

DuBois, P. H. (1970). A History of psychological testing. Boston: Allyn and Bacon.Hallam, G., & Campbell, D. (1994). Manual for the Campbell-Hallam Team Development

Survey. Minneapolis, MN: National Computer Systems.Hansen, J. C., & Johansson, C. B. (1972). The application of Holland’s vocational model to

the Strong Vocational Interest Blank for Women. Journal of Vocational Behavior, 2, 479-493.

Hansen, J. C., & Campbell, D. P. (1985). Manual for the Strong-Campbell Interest Inventory(4th ed.). Stanford, CA: Stanford University Press.

Page 21: The Campbell Interest and Skill Survey (CISS): A Product of Ninety Years of Psychometric Evolution    Journal of Career Assessment 1995 Campbell 391 410(1)

410

Holland, J. L. (1973). Making vocational choices: A theory of vocational personalities andwork environments. Englewood Cliffs, NJ: Prentice-Hall. (Now published by PsychologicalAssessment Resources, Odessa, FL)

Strong, E. K., Jr. (1927). Vocational Interest Blank. Stanford, CA: Stanford University Press.

Strong, E. K., Jr. (1933). Vocational Interest Blank for Women. Stanford, CA: StanfordUniversity Press.

Strong, E. K., Jr. (1938). Vocational Interest Blank for Men (rev. ed.). Stanford, CA:Stanford University Press.

Strong, E. K., Jr. (1943). Vocational interests of men and women. Stanford, CA: StanfordUniversity Press.

Strong, E. K., Jr. (1946). Vocational Interest Blank for Women (rev. ed.). Stanford, CA:Stanford University Press.

Strong, E. K., Jr. (1955). Vocational interests eighteen years after college. Minneapolis, MN:University of Minnesota Press.

Strong, E. K., Jr, Campbell, D. P., Berdie, R. F., & Clark, K. E. (1966). Strong VocationalInterest Blank for Men (rev. ed.). Stanford, CA: Stanford University Press.

Strong, E. K., Jr., & Campbell, D. P. (1968). Strong Vocational Interest Blank for Women(rev. ed.). Stanford, CA: Stanford University Press.

Terman, L. M. (1916). The measurement of intelligence. Boston: Houghton Mifflin.

Yerkes, R. M. (Ed.). (1921). Memoirs of the National Academy of Sciences: Psychologicalexamining in the United States Army. Washington, DC: U.S. Government Printing Office.