008-015 the Use of Projective Methods in Group Testing

8/6/2019 008-015 the Use of Projective Methods in Group Testing

1/8

THE USE OF PROJECTIVE METHODS INGROUP TESTING1B Y RUTH L. MUNROE

SARAH LAWRENCE COLLEGE

r-pHEPROBLEMI would like to dis-X cuss is the feasibility of using theprojective approach in those normalsituations where time is at such a pre-mium that the only possible answer isthe group test. In education, in indus-try, in a host of ordinary life situationsand in research the personality factoris of acknowledged importance, but itis impossible to study large groups ofordinary people with even a limitedclinical battery of tests. They must betested, if at all, by relatively quickgroup or self-administering methods.The test materials must be evaluatedquickly, and the results made availablein intelligible form. For some purposes,especially for research, the test proce-dures must be repeatable under condi-tions clearly described, and the finalstatement must be made in quantifiableor at least easily manipulable terms.The projective method as developedin the clinic meets none of these require-ments. It has somany advantages, how-ever, that it seems important to considerwhether and how it can be modified forgroup use. In the following pages Ishall rely to some extent upon empiri-cal data, but my aim is to discuss possi-bilities rather than to present finishedprocedures. I think available data arepromising enough to suggest the valueof careful analysis of the assets of theprojective approach in group testingand open-minded consideration of the1Paper read at the Bound Table on Diag-nostic Testing, American Psychological Asso-ciation, Philadelphia, Pa., September 6, 1946.

problems involved. The purpose of thispaper is orientation to the special taskof adaptation.Two aspects of the projective ap-proach seem to me basic. The first in-volves the presentation of complex butrelatively unstructured materials witha minimum of instruction in order toobtain a complex specimen of sponta-neous action from the subject. The sec-ond involves acceptance of the unique-ness and complexity of this specimen inevaluationa condition especially diffi-cult in group testing, which we willpostpone for later examination.PROJECTIVE MATERIALS IN GROUP

TESTINGIn regard to the first aspect, the pre-sentation of materials, it is easy enoughto ask the subject towrite out in a groupwhat he sees in inkblots, the story sug-gested by pictures, answers to vagueleading questions, associations towords;

to have him draw or complete pictures,etc. If there is any merit in such complexspontaneous products as such, then thegroup products should have merit, eventhough the examiner has not been ableeither to make those adaptations in ad-ministration which encourage the sub-ject to fullest expression or to follow upspecial points for clarification and elab-oration. I think that most projectivetesters (clinicians) have been so dis-turbed by the inevitable loss entailed ingroup administration that they have notpaid sufficient attention to the possibleresidual value. It is very hard for a


2/8

THE USE OF PEOJECTIVE METHODS IN GROUP TESTING 9clinician accustomed to responsibilityfo r judgment about an individual caseto accept as legitimate compromise er-rors which he could easily avoid by ask-ing the subject what he meant. Theoreti-cally, however, the complex spontaneousproduct of the subject might be the mainthing, and the loss sustained in groupadministration no greater than the low-er aspiration level of the group testwould warrant.This theoretical expectation has bynow fairly substantial empirical con-firmation. One way or another clinicianshave been forced by practical or re-search requirements to be less exigentabout methods of administration. Manyinstances of successful use of group ma-terials are reported in the literature andmany more in conversation. The resultsobtained from the group Rorschach atSarah Lawrence College to be describedlater will serve as example. Statisticalstudies and experience indicate that sub-jects see about the same things in aboutthe same way when inkblots are shownon the screen as when they are held inthe hand. Minor variations in the re-action of the subject, and fairly seriousdifficulties in scoring2 operate to reduce

2 By far the greatest amount of group pro-jective testing has been done with the Ror-schach blots. For reasons about to be consid-ered this seems to me desirable at presentit is the test we know most about. Actuallyit is not a good test for group administrationbecause basic scoring depends too much onpoints the subject may not spontaneously re-port, for instance that it was the blackness ofthe blot which made him think of a bat. Es-tablishing these points requires special inquirywhich can only be approximated in group ad-ministration.Stories, drawings and the like would seemto be more self-contained, to require less spe-cial inquiry to determine essential data. Theingenuity of psychologists can surely devisebetter methods of eliciting a complex sponta-neous product more easily scored than ourpresent adaptations of tests developed in theclinic, as soon as we understand more clearlyjust what is needed.It is for this reason that elaborate researchon the correspondenceof group and individualEorschach protocols does not seem to me worth

the accuracy of evaluation in the indi-vidual case. Some protocols must be dis-carded as unscorable. Some groups ac-cessible to individual testing are notsuitable for group work. Under no cir-cumstances can the examiner rely uponthe group protocol for sensitive diag-nosis with anything like the confidencehe may feel for a carefully administeredRorschach. Nevertheless the satisfac-tory results obtained by applying famil-iar test principles to the analysis of thegroup products would seem to showclearly that they are essentially similarto the individual tests.The similarity of group test materialsto those obtained in individual testinggives the group test adapted from cur-rently used individual methods an un-precedented advantage. It can start withan enormous background of informationderived from experience with thousandsof cases carefully observed in the clinic.The loss in accuracy and richness of thegroup Rorschach protocol as against in-dividual administration becomes trivialif one compares it with the differencebetween the questionnaire and the clini-cal interview. The questionnaire cannotbe considered a relatively inaccurateversion of the interview. Checking yesor no to printed questions involves avery different process from answeringsimilar questions adaptively phrasedand followed up in face-to-face encoun-ter, a fact too generally recognized torequire further discussion here. Eachquestionnaire is really a new instrumentwhich must be independently studied.Measures derived from factorial analy-sis are also new measures whose mean-while. I believe that the test is extremely use-,ful at present, but that it will be supersededby group methods less intrinsically d i f f i c u l t . Itseems more profitable to get ahead with basicinformation by means of a test involving anobvious margin of error than to make finickingefforts to reduce an error small relative to thetotal present contribution and likely to be ir-relevant in later test developments.


3/8

JOURNAL OP CONSULTING PSYCHOLOGYing must be empirically established.Tests based upon the measurement ofsimple perceptual or motor functionsseem to me very promising but againthe experimenter starts from scratch ininterpreting and validating his observa-tions. The same problem applies to theMultiple Choice Rorschach which de-pends upon recognition rather thanspontaneous production of responses,and alsoto those new complexprojectivematerials which I hope will ultimatelysupersede the Group Rorschach (seepreceding footnote).An even more unique advantage inadapting current clinical tools to grouptesting is the introduction of a possibleflexibility in evaluation. A questionnaireis standardized andvalidated as a whole,or with a few prescribed subscores. Themoment the examiner departs from theofficial scores and uses the subject's ans-wers to specific questions in any otherway, he is strictly on his own. The goodclinician often does so, doubtless suc-cessfully. In so doing, however, he relieson his own intuition and experience.Practically nothing is printed about theimportance of a single question or acombination of questions as indicativeof some refinement in diagnosis. In re-search the official scores are used, or thewhole test is revamped as in Flanagan'scontribution to the Bernreuter.The most complete sceptic can agreethat a Group Rorschach protocol yieldsdata about as valid on one point as an-other.8 Rorschach literature and teach-

3 This statement is not strictly true. Sub-jects are more likely to describe action _ in amanner which makes an M scoring inevitablethan to mention that the worms in card X aregreena point crucial for the FC score, butincidental for the subject. Inquiry may estab-lish that he had in mind "those green things",but the spontaneous comment of the normalurbanite may easily omit this refined distinc-tion among worms as unimportant.Even with accurate scoring there is moreagreement among Rorschach examiners onsome aspects of performance than others.

ing and experience give a substantialbackground to the almost infinite per-mutations and combinations observablein the test performance. The examineraiming at individual evaluation or re-search is not restricted to the judgmentsindicated by the few scores previouslyestablished, but may ask: "What out-standing characteristics does this per-son show in his Rorschach perform-ance?" ; or "What differences can be ob-served between these two groups?". Ifhe examines the test materials resource-fully, the psychologist may be able topoint out facets of the subject's person-ality especially important in the immedi-ate situation which are not covered bya few prescribed test dimensions. Or hemay suggest trends common to groupsof individuals which are obscured bypronounced individual differences in be-havior. These trends can at least serveas useful hypotheses for more focusedstudy.W e cannot here consider the specialproblems of repeatability and quantifi-ability involved in research on grouptrends. The points I have tried to stressare (a) that the projective method of-fers a complex specimen of spontaneousaction even when administered togroups and (b) that where current in-dividual methods are adapted to groupuse, the group tester for the first timecan approach the problem of evaluationwith something of the resourcefulnessand knowledge available to the clinicianworking with similar individual meth-ods.

METHODS OP EVALUATION FOE THEINDIVIDUAL CASE

The second aspect of the projectivemethodacceptance of the product as aunique and complex specimenpresentsthe most serious difficulty in fulfillingthe requirements of group testing. Theprojective method has value essentially


4/8

THE USE OP PROTECTIVE METHODS IN GROUP TESTING 11because isolated observations are inter-preted in relation to the total picture.The Rorschach CF score, for instance,seems to represent affective lability inthe subject. Relatively uninhibited useof color appears to have similar signifi-cance in spontaneous drawings withfree choice of media. Yet to rate a sub-ject's affective lability in terms of thenumber of CF scores or amount of colorin drawing would be meaningless. Eval-uation of CF as a personality trait de-pends upon a dozen other factors in thesubject's performance. The final descrip-tion of a subject giving 3 CF mayvary all the way from an aggressivebully to Caspar Milquetoast. Fragmen-tation of any projective method into theseparate item scored or observed mayyield quick and comfortably "objective"results. Unfortunately the results arevery likely to be wrong, as has beenrepeatedly demonstrated where thescoring fragment is taken as the meas-ure of a particular personality trait.Undoubtedly results could be im-proved by a more insightful selection ofseparate items. Improved selection ofrepeatable items seems to me essentialfor some types of large-scale research.Furthermore the aggressive bully andCaspar Milquetoast may well have somebasic trends in common which could befruitfully examined in an investigationof dynamic patterns for scientific pur-poses. In evaluating the individual case,however, one must try to grasp howthese trends interact with others toproduce the unique functioning person-ality. The employer is more concernedwith Caspar's overtly timid behaviorthan with the repressed aggression hemight show to his psychoanalyst. For agrasp of the interrelationships of com-plex data I think there can be no substi-tute for the wit of the trained psycholo-

It is worthwhile to point out that in-

terpretation of the results of standard-ized tests in the individual case also de-pends upon the judgment of the psychol-ogistexcept in so far as they are de-livered over to the lay administrator forjudgment. Everyone knows that all ofthe standardized tests, even intelligencetests, may be simply erroneous in somecases and that the significanceof an ac-curate score on a particular trait mustbe seen in relation to scores on othertraits and background data. It seemsproper to emphasizethat the projectivemethod attempts to apply within theconfines of the test situation a processof judgment fully accepted in clinicalpsychology generally. The novelty seemsreally to lie more in the point at whichjudgment is introduced rather than inthe introduction of judgment.Statistical checks on reliability andvalidity have tended, I think, toward asomewhat spurious security in the ob-jective nature of testing devices inpractical evaluation. Reliability figuresare never perfect, which means thatsome individuals may very well showentirely different results on retest.Which individuals may shift is notspecified. Validation figures are uni-formly rather poor, even in the field ofintelligence testing. A statistical state-ment of the margin of error found in aparticular experimental group in apar-ticular situation is not a realistically ef-fective guide toward decision as to thevalidity of the score in an individualcase in another situation. For instance,a study of Bernreuter scores at my col-lege indicated high statistical signifi-cance. The probability (P) for the bestcombination of scores empirically de-termined for our group was .00001. Ac-tually 20 out of 25 students consideredseriously maladjusted by the psychia-trist had poor scores. When one consid-ers, however, that 139 students in theexperimental group had similar poor


5/8

12 JOURNA L OP CONSULTING PSYCHOLOGYscores and did notso far as is known-show any particular maladjustment, onemay wonder just how much help a P of.00001 is to the teacher in decidingwhich one of 7 students with poor scoreshe should worry about. Precision ofstatement of error for the group andrepeatability of scoring proceduresreally offer no sort of scientific guaran-tee as regards the correctness of judg-ment in the individual case.Where the judgment of the examineris included in the test procedure itself,the margin of error cannot be stated soprecisely. The avowed purport of theprojective test is not definitely delimit-ed, nor can scoring by a second examin-er be expected to yield exactly the sameresults. Certain uncontrollable scourcesof error are introduced. In actual test-ing of the individual, however, the state-ment that the test hopes tomeasure "in-troversion", and exact repeatability ofscoring, constitute sosmall a part of thetotal problem involved in effective evalu-ation of a person in a given situationthat concentration on control of thesepoints seems a rather meager version ofscientific responsibility. Naturally thescientist wants a demonstration of thevalidity of his procedures, and will wantto assure himself that positive resultsare ascribable to the instrument ratherthan to extraneous factors. Once thisdemonstration is made, however, itseems to methat the over-all correctnessand usefulness of the results are themost important criteria. If the value ofa test can be generally improved by al-lowing a competent examiner to use histraining, it seems to me no less scientifictoacknowledge this "uncontrolled" vari-able as a possible source of error thanto control a relatively minor variablewith neglect of other factors known tobe of enormous importance i.e., theconstellation of trends in the individualsubject, attention to data not yet re-

duced to a formal score but of clear clin-ical significance, and the like.The preceding papers have taken theimportance of these latter points forgranted. I approach them more self-consciously because the group testis traditionally "standardized", so thatreliability and validity figures are almostsynonymous with its scientific repute.Clinicians accustomed to the fuller in-formation and time for considerationavailable in individual testing view thequick handling of imperfect materialswith just as great suspicion. My sug-gestion is that the philosophy of thegroup test might well preserve the ac-ceptance of a margin of error in the in-dividual case beyond the point at pres-ent tolerable for the responsible clini-cian, and also view more realisticallythe problem of control of error. I sug-gest that statistically precise reliabilityand validity figures do not sufficientlyanswer this problem, and that it is moreimportant to make useful judgmentswith as few errors as possible than to"control" a small aspect of a measurein a manner which actually prohibitsutilization of our best source of infor-mation. / suggest that safeguarding theindividual subject from errors in evalu-ation depends more upon careful use oftest results than upon establishing th ecustomary statistical figures for a group.At this point I would like to discussour experience with the Group Ror-schach at Sarah Lawrence College as anillustration of the philosophyof testingjust described. The Group Rorschachhas been administered to every enter-ing student since 1940. The examinerwrites a descriptive sketch of each girlcovering such aspects of her intellectualand emotional development as seem ob-servable with some security in quickwork. For several years conditions ofblind analysis were maintained and theaccuracy of the sketches checked as ob-


6/8

THE USE OF PEOJECTIVE METHODS IN GROUP TESTING 13jectively as possible. Teachers weregiven the sketches without names ingroups of 4 or 5 and asked to identifythe students intended. This they wereable to with a degree of success far be-yond chance expectation.4 Teacherswere also asked to underline in blackstatements they considered correct andin red those they considered false. Outof 1,934 separate statements 143 (8 percent) were underlined only in red and322 (17 per cent) in red by one teacherand in black by another. Errors in iden-tification seemed to be due mostly tomiddle-of-the-road descriptions of mid-dle-of-the-road students, or to a singleespecially vivid wrong or badly phrasedstatement in a long paragraph whichseemed otherwise essentially correct.For the most part teachers acceptedthese sketches as reasonably accuratewhen they were told which student wasintended. A few appeared definitelywrong in major respects, and almostevery sketch contained dubious details.Perhaps one may say that the realcheck on the sketches began after theexperimental period when they were putto the test of practical use. The consen-sus of opinion after several years seemsto be that the sketches are substantiallycorrect and helpful. Obviously no state-ment is used in guidance which is notconfirmed by firsthand observation, butthe test frequently calls attention toaspects of the student's psychology easi-ly missed by teachers or learned belated-ly. A simple example is the test revela-

4 Obtaining judgments on every freshmanfrom every teacher who had enough freshmenin his class for such a "matching experiment"involved complications for which we could findno simple statistical answer. We have 8 sepa-rate Chi-squares differently calculated depend-ing on the number of judges per student, i.e.,upon the number of courses a girl happenedto take where there were at least 4 other en-tering students. All of the Chi-squares aregood. The most accurate estimation seems be-tween 15 and 20, one degree of freedomhighly significant.

tion of a deep uncertainty masked by acocksure manner. An apparently inde-pendent and belligerent girl may re-spond far better to friendly but firmcontrol than to the freedom of actionshe stridently demands or the discipli-nary putting-in-her-place which hermanner tends to provoke. If the testremarks upon a bent of mind whichfundamentally dislikes literal fact, theitem may be useful in helping the stu-dent decide whether she should continuea major in science. Spontaneous curric-ular choice sometimes comes from a ro-mantic attachment to a scientist or from,a glamorized view of science. Test de-scription of a bent of mind should not,of course, decide such an issue, but itmay be a valuable ingredient to add toother data. The college is very carefulto use the Rorschach results only in con-junction with other data, indeed prob-ably rejects rather too promptly com-ments which do not seem immediatelyintelligible."The use of the Group Rorschach atthe college may be compared with theuse of various standardized tests, espe-cially the Bernreuter which was admin-istered over a period of eight years andcarefully studied. For two years theGroup Rorschach and the Bernreuterwere both administered to every stu-

6 Careful review of many cases indicatesthat some of the errors in the sketches are dueto examiner-failure in quick evaluation, or thatthe interpretive slant suggested by extra-testdata could modify a particular test commentand offer important information about the stu-dent not immediately observable to the teacher.A sizeable number of test comments initiallyrejected by teachers prove essentially correctin the long run. Time permitting, a more care-ful integration of test and other materialsshould be made for the best understanding ofeach case. Since time never permits such in-tegration for every case, the policy of the col-lege in rejecting statements which do not fitseems the most expedient. Of course this su-perficial use of test materials in the ordinaryrun of educational problems does not precludemore intensive study of especially difficult stu-dents.


7/8

14 JOURNAL OF CONSULTING PSYCHOLOGYdent. As described elsewhere [1] theRorschach was handled in a mannerwhich yielded a quantitative generalscore for "adjustment" in addition tothe qualitative sketch now under discus-sion. Checked in the same manner, thisquantitative score proved more accuratethan the Bernreuter scores on the samestudents. I think the most importantmoral to be drawn, however, is that thecollege actually uses the qualitative Ror-schach results whereas the quantitativescores mildewed in the files. A descrip-tive statement is much harder to dealwith statistically than a single score,but it is much easier for a teacher tohandle in practical judgment. A teacheris legitimately baffled by a poor scorewhich may or may not be worth bother-ing about in the individual case. A de-scriptive evaluation of such assets andliabilities as he can observe himself inthe student is on the whole more easilyused and more easily criticized. Someteachers (or other "lay" users of psy-chological tests) will be overenthusiastieabout the test statements or reject themwith too little consideration. This phe-nomenon has also been observed in thehandling of scores from standardizedtests. (E.g.,Some colleges admit no ap-plicants who fall belowa specified scoreon the routine intelligence tests; otherspay little or no attention to the score.)If one's aim in group testing is mere-ly useful application of "test" informa-tion, I believe that the projective meth-od with qualitative description deservesvery serious consideration. Without fora moment suggesting that the projectivemethods at present can offer results ofthe calibre we would like in group test-ing, I think I can urge (a) that thepractical possibilities of such testing areenhanced by allowing the trained ex-aminer to use his judgment in handlingpromising test materials, and (b) thata descriptive account adapted to the spe-

cial requirements of a particular situa-tion is on the whole handled more ap-propriately by the "lay" user than asingle score where a large margin oferror is precisely stated in statisticalterms irrelevant to judgment in the in-dividual case.THE QUANTIFICATION OF PROJECTIVE

METHODSThus far I have discussed the possi-bilities of group projective testing forthe purpose of individual evaluation in

the hands of thoroughly trained psy-chologists and ultimate consumers (e.g.,teachers) sufficiently informed fo r sen-sible, critical application. I believe thatthis purpose is potentially of widespreadvalue if on the one hand psychologistscan be trained to orient their work tothe special needs of the practical situa-tion, and on the other the ultimate con-sumers can learn more about how per-sonality factors affect the concrete per-formance which is their special pur-view. Far from despairing of the end-less complications of this mutual learn-ing process, I think it the main hope ofthe future. It is being instrumentedalready by forces beyond the scope ofthe test-maker.For some purposes, however, for somelarge-scale screening jobs and for cumu-lative research, further codificationofthe projective methods is essential. Testprocedures must be described with suf-ficient definition for exact repeatabilityunder varying conditions. Test resultsmust have a less cumbersome formatthan pages of description to allow forfeasible summation of materials con-cerning group trends drawn from manyindividual records. These requirementsnecessarily entail very serious loss inaccuracy and richness of interpretationfor the individual case. Hence the im-portance of keeping the aim of testingclearly in view. Projective methods may


8/8

THE USE OF PROJECTIVE METHODS IN GROUP TESTING 15be used in one way for maximal insightinto the individual, and in other waysfor learning about common trendsamong groups of subjects.Careful analysis of the special prob-lems of quantification is not germane tothe present symposium. I cannot for-bear mention of one general point, how-ever, Project!ve test literature, espe-cially the Rorschach literature, is fullof quantitative studies, many of whichhave yielded very disappointing results,some of which seem very promising. B yand large it seems clear that poor re-sults are obtained when a limited num-ber of the traditional scores (whichwere never intended for use in isola-tion) are equated with personality traitswithout further ado and correlated withexternal factors. Promising results areobtained when combinations of scoresare used which are known clinically tohave some stability of meaning fromone individual to the next, or when theclinically experienced examiner observescertain recurrences in the performanceof special groups which are not gener-ally encountered. Such observationsmay often be reduced to quantifiableterms and statistically analyzed, eventhough they may involve aspects of testperformance usuallynot formally scored.In short, clinical knowledgeof the in-strument must be used to suggest likely

lines of quantification. It can also offersome clue to the meaning of statisticallysignificant group differences as actuallyobserved. When the judgment of theskilled examiner is applied in setting uplarge-scale research, much of the re-sourcefulness and flexibility of the pro-jective method may be retained in spiteof the limiting requirements of quanti-fication. It seems to me most unfortu-nate that so many investigators confinetheir research efforts to careful tabula-tion of familiar items instead of usingtheir knowledge of the complexities ofindividual performance to select quan-tifiable data of deeper import, less sub-ject to fluctuation in significance fromcase to case.Even in quantitative research the twobasic aspects of the projective methodmay be retained, complexity of the sub-ject's product, and acceptance of thecomplexity in evaluative procedures.The simplification necessary for objec-tive mathematical treatment can capi-talize clinical experience in determiningfruitful lines of codification.

REFERENCE1. M U N B O E , R. L. Prediction of the adjust-ment and academic performance of collegestudents by a modification of the Rorschachmethod. Appl. PsychoL Monogr., No. 7.Stanford Univ., Calif.: Stanford Univ.Press, 1945.

Documents

008-015 the Use of Projective Methods in Group Testing