Acta Psychologica 3 8 , 2 0 5 - 2 1 3 . © North-Holland Publishing Company 1974

AN ANALYSIS OF THE STAGES UNDERLYING THE PROCESS OF RESPONDING TO PERSONALITY ITEMS

T.B. ROGERS 1

University of Calgary, Calgary, Alberta, Canada

Three exper iments were done to determine the additivity of pairwise comparisons of three personality i tem characteristics: (1) Length, (2) Ambigui ty and (3) Controversiality, using Sternberg's (1969) method. Each of these i tem at tr ibutes was chosen as an experimental factor to represent a stage in a postulated model o f responding to personali ty items, Results indicated non-additivity o f the exper imental factors which was interpreted as support ing the postulated model.

1. Introduction

Sternberg (1969) has presented a methodology for exploring various stages in a cognitive processing sequence. His approach focuses upon assessing the degree to which various postulated sub-processes or stages can be seen as independent. The method involves determining the degree of non-additivity between reaction times (RT) associated with various levels of experimental factors expected to effect the postulated sub-stages. The present paper applies Sternberg's (1969) approach to the analysis of a complex cognitive event; namely, the process of re- sponding to personality items.

In responding to personality items the subject is presented an item (e.g., I envy scholars who have the time to think) and required to indicate whether or not this statement describes him, by responding 'true' if it does and 'false' if it does not. Since the number of possible stimuli (items) exceeds the number of responses, this task is an informa- tion reduction task in Posner's (1964) terms. The present s tudy also represents an at tempt to delineate some of the parameters dictating the nature of this reduction task.

I Current address: Depar tment of Psychology, The University o f Calgary, Calgary, Alberta, Canada T2N 1N4. This research was supported by a grant f rom the Canada Council. Thanks are extended to P.J. Rogers and J.G. Ells for their comments on earlier drafts.

206 T.B. Rogers, Process o f responding to personality items

In relation to the kinds of tasks typically considered in RT studies, personality item responding appears to be more complex by virtue of the stimuli being sentences. Furthermore, item responding appears to be affective in nature (i.e., Rogers 1971) and to have a strong evaluative component (i.e., Edwards 1953). In view of these considerations it is not surprising that research into the process of responding to personal- ity items has been slow in emerging (see Rogers 1971).

The present study deals with three postulated stages in the item responding process. The first of these, Stimulus Encoding, is assumed to involve the translation of the physical stimulus into terms which will permit operation by subsequent stages. This would involve activities such as reading and could be thought of as input time. The experi- mental factor of item length is postulated to effect RT associated with this stage. Hanley (1962) has shown that longer items require more time for response, giving support to the use of item length in the present study.

The next stage has been labeled Stimulus Comprehension, and in- volves an operation on the output of stage 1 to render the item under- standable to the respondent. It is assumed that this stage would be affected by the experimental factor of item ambiguity, with unambigu- ous items requiring less time in this stage, than ambiguous ones. Rogers (1973) assessed RT to items of varying ambiguity and found no systematic relations. However, the operational definition of ambiguity used in that study (Goldberg 1963) may not have been appropriate (Rogers 1973). In the present study, item ambiguity was assessed using subject ratings (Rogers et al. 1972). These ambiguity ratings are expected to influence the amount of time spent on stage 2 of the model, and can be thought of as similar to stimulus degrading, with ambiguous items being degraded.

The third postulated stage involves the binary 'true/false' decision about the item. Previous research has delineated some of the parameters attending this decision. Hanley (1962) and Rogers (1973) have shown that RT is highest for items showing proportions of 'true' re- sponses {P(t) } around 0.5. These items not only have elevated RT's but also receive higher subjective ratings of difficulty (Rogers 1973) and lower ratings of confidence (Rogers, in preparation). Conversely, items showing extreme P(t)'s require less time for responding, and are rated as easier to answer. From these findings there is indication of an inverted U-shaped function relating P(t) and RT. This U-shaped curve can be

T.B. Rogers, Process o f responding to personality items 207

transformed to a linear function by considering the distance a particular item P(t) is from 0.5. This transformation yields what have been labeled controversiality values for items. Items near a P(t) of 0.5 are controver- sial while items showing extreme P(t)'s are non-controversial. The re- sults of Hanley (1962) and Rogers (1973) indicate a correlation between item controversiality and RT, with controversial items requir- ing longer for responding. These findings have led to the use of con- troversiality as the experimental factor which influences the amount of time spent in stage 3 of the proposed model.

ITEM ITEM ITEM LENGTH AMBIGUITY CONTROVERSIALITY

RESPONSE

Fig. 1. Diagram of postulated model of responding to personality items.

The present study involves three experiments consisting of pair-wise comparisons of the three experimental factors outlined above (item length, ambiguity, and controversiality). Under conditions of additivity of the three stages, the theoretical processing sequence is diagrammed in fig. 1. The extent to which additivity is present will dictate future research bearing on stage 3 of the proposed model.

2. Method

2.1. Subjects

A total of 30 volunteer university students participated. Five male and five female Ss were used in each of the three experiments, with each S being paid $ 2.00 for his time.

2.2. Rems

The items used throughout this study were selected from Rogers et al. (1972). In this item pool, item length was defined by constructing a long and a short set of items. The long items fell between 70 and 80 letters in length (blanks between words included) and the short items showed a range of 40 to 50 letters. Two hundred long and 200 short items were constructed. Item controversiality values were determined from the responses given these items by 50 university students. Fifty other students rated the ambiguity of these items. In this item pool

208 T.B. Rogers, Process of responding to personality items

the correlation between Ambigui ty and Length is 0.13, be tween Ambigui ty and Controversial- ity is - 0 . 07 , and between Length and Controversiality is - 0 . 0 1 . This indicates that it is possible to select i tems for the present s tudy wi thout concern for large dependencies in the i tem characteristics.

In the Rogers et al. pool each i tem characteristic was represented by two levels in each of the three experiments . For each experiment , i tems were chosen to maximize the difference between the two levels of each i tem charactistic. However, because controversiality and ambigu- ity were defined by responses (as opposed to control by E) it was not possible to obtain an equal number o f i tems for each experiment . In all three exper iments the i tems were presented in a r andom order.

2. 3. Equipment

The i tems and instructions were photographed on a 35 m m film strip, to be used in a Singer Graflex rear screen projector. An opaque rear-projection screen was placed about 1 meter in front o f the projector and S seated approximately 1 meter f rom the screen. One three-but ton response panel was placed in front of the rear-projection screen. This response panel consisted of a middle bu t t on which was painted red, plus two white bu t tons marked ' t rue ' or 'false', which were on either side of the red bu t ton . A display panel with one red and one green cue lamp was placed between the screen and response panel. The film strip projector and 2 panels were connected to a PDP 8/I on-line computer which controlled all s t imulus presentat ions and recorded the data.

2.4. Procedure

The following procedure was used for all three experiments. Ss were run individually. After the instruct ions (presented by the film strip projector), the S responded ' t rue ' or 'false' to 20 items as a warm-up and familiarization task. After this warm-up, additional instruct ions were presented which informed the S that the green light would come on prior to a trial. S was then to depress and hold down the red bu t t on with the index finger o f his preferred hand. This action caused the green light to go off and the red light to come on. The onset of the red light signalled the beginning of a randomly chosen interval (range: 5 0 0 - 4 0 0 0 msec), at the end of which the i tem was projected on the screen. The purpose of the random interval was to minimize anticipation of the item onset. The S was instructed to cont inue holding down the red bu t ton until he was certain of his ' t rue ' or 'false' response, and then to release the red bu t t on and indicate his response, by pressing the appropriate response bu t t on with the same finger he used to hold down the red bu t ton . The computer recorded two times: (1) the t ime from slide onset to release o f the red bu t t on (RT), and (2) the t ime f rom release of the red bu t ton until onset of the response b u t t o n (movement t ime - MT). The random anticipation interval was also recorded along with the S's response. After a 2-second inter-trial interval the green light came on signalling the beginning of the next trial. The Ss were given five practice trials using this procedure, before moving on to the major part o f the study.

T.B. Rogers, Process o f responding to personality items 209

3. Experiment 1 : Length and Controversiality

3.1. I t ems

One hundred short and 100 long items were chosen from Rogers et al. (1972). Fifty of each were controversial (average P(t) = 0.49), and 50 were non-controversial. These latter items were subdivided into two groups of 25 with one group showing an average P(t) of 0.13 and the other an average P(t) of 0.86.

3. 2. Resul t s

The mean RT data for the four combinations of Length and Controversiality are shown in fig. 2. For each S the mean RT for items of varying Length and Controversiality was subjected to a 2 × 2 analysis of variance with repeated measures on each variable. The main effect of

6000 u

E

W 5000

~u o9 Z o 4000 n (n uJ rr

o Controversial

• Non- Controversial

..o /

/ t

/

3 0 0 0 I I SHORT LONG

I T E M L E N G T H

Fig. 2. Average responding time for items of varying Controversiality and Length obtained in Experiment 1.

Controversiality was significant (F = 14.45, df = 1/9, p < 0.01) with controversial items show- ing an average RT of 4342 msec compared to 3940 msec for non-controversial items. A large effect of item length obtained (F = 130.00, df = 1/9, p < 0.01), which showed mean RT to short items as 3559 msec, and 4723 msec, for long items. The interaction between Length and Controversiality was not significant (F = 2.21, df = 1/9).

210 T.B. Rogers, Process o f responding to personality items

4. Experiment 2: Length and Ambiguity

4.1. I tems

Fifty long and 50 short i tems were further subdivided into equal numbers of low and high ambiguous items. The average ambiguity rating for the high ambiguous i tems was 2.52 (a 5-point rating scale was used) and 1,88 for the low ambiguous items. A t-test be tween the ambiguity values of the high and low items indicated a significant difference between them (t = 23.04, d f = 98, p < 0.01).

4.2. Results

The mean RT data for this exper iment are shown in fig. 3. Analysis of variance of the Ss' mean RT yielded a main effect of Length 2 (F = 37.01, d f = 1/9, p < 0.01), with the long i tems

6000 o

E

5 0 0 0

w

4 0 0 0

W

3000

~0

/

o High Ambiguous

• Low Ambiguous

I I SHORT LONG

ITEM LENGTH

Fig. 3. Average responding t ime for i tems of varying Ambigui ty and Length obtained in Experi- ment 2.

requiring 1511 msec longer than the short items. A significant main effect of Ambigui ty also emerged (F = 6.70, d f = 1/9, p < 0.01) with the mean RT for high ambiguous i tems being 5529 msec compared to 5108 msec for low ambiguous items. The interaction was not significant (F = 0.02, d f = 1/9).

2 In cases where a test involves the replication of an already observed result (i.e., main effect of Length documented in Exper iment 1) a one-tailed test was used.

T.B. Rogers, Process o f responding to personality items 2 ] 1

5. Experiment 3" Ambiguity and Controversiality

5.1. Items

It was possible to obtain only 80 i tems for this experiment , given the constraints of the larger pool, as only short i tems were considered. Twenty i tems represented each of the four cells in this experiment . The high ambiguous i tems showed a mean ambiguity rating of 2.55, compared to 2.04 for the low ambiguous ones (t = 13.34, d f = 78, p < 0.01). The controversial- ity variable was represented by 40 controversial (average P(t) = 0.51), and 40 non-controversial items. Twenty of these latter i tems showed an average P(t) of 0.11, while the remaining 20 had an average P(t) of 0.79.

5.2. Results

The mean RT data for all 10 Ss is presented in fig. 4. The main effect of Controversiality was significant 2 (F = 6.35, d f = 1/9, p < 0.05), with controversial i tems showing a mean RT of

6 0 0 0

E v

W

5 0 0 0 w

w

z o 4 0 0 0

w

~o

s

=. . . s

0 ~

J J • f

o Controversial

• Non - Controversial

3 0 0 0 I I LOW HIGH

I T E M A M B I G U I T Y

Fig. 4. Average responding t ime tbr i tems of varying Controversiality and Ambigui ty obtained in Exper iment 3.

5965 msec and non-controversial i tems showing a mean RT of 5050 msec. The main effect of Ambigui ty was significant 2 (F = 4.78, d f = 1/9, p < 0.05), while the interaction was non-signifi- cant (F = 0.02, d f = 1/9).

6. Discussion

The present results indicate that the effects of item Controversiality,

212 T.B. Rogers, Process o f responding to personality items

Length and Ambiguity upon RT to personality items are additive. That is, the present results show these three item characteristics to have independent effects upon RT. Each item parameter was chosen as an experimental factor to represent an operational manipulation of one stage in a postulated model of responding to personality items. The obtained additivity offers support for the formulation diagrammed in fig. 1. The experimental factors used in the present study do not appear to have common effects on the postulated stages, by virtue of the present results. Hence the simple, modular model presented in fig. 1 appears able to account for the observed results.

Of major interest from the assessment viewpoint is the binary deci- sion stage of the responding model. The present results indicate that further research into the binary decision stage can proceed without concern for carry-over or interactive effects associated with the item characteristics o f Length and Ambiguity.

The present paper subscribed to a relatively descriptive interpretation of the binary decision stage, by focusing on the known effect of Con- troversiality. It should be noted that the use of variables correlated with Controversiality (i.e., Desirability) possibly would have led to similar findings of additivity. However, the impact of the present results lies in the documentat ion of independence of the binary decision stage from stages earlier in the processing sequence. With this independence established, it becomes possible to pursue a less descriptive model of the binary decision stage, as well as to consider other stages that could be operative during the answering of personality items.

Even though the present task is more complex in several respects than traditional RT tasks, it is encouraging that Sternberg's methodolo- gy has proven its worth in attempting to break down a complex task into a series of stages of workable size. It would seem that this method has the potential to serve as a fundamental paradigm for a comparative psychology of cognitive tasks. This would involve an at tempt to map complex tasks such as responding to personality items into a larger set of smaller sub-processes or stages that could be assessed for com- monality across tasks. The interaction of these component stages with subject variables, as in the work of Fleishman (1972), could represent a meeting of the traditionally disparate domains of experimental and correlational psychology (Cronbach 1959).

T.B. Rogers, Process o f responding to personality items 213

References

Cronbach, L.J., 1957. The two disciplines of scientific psychology. Amer. Psychol. 12, 671-684.

Edwards, A.L., 1953. The relationship between the judged desirability of a trait and the probability that the trait will be endorsed. J. Appl. Psychol. 37, 90 -93 .

Fleishman, E.A., 1972, On the relationship between abilities, learning, and human performance. Amer. Psychol. 27(11), 1017-1032.

Goldberg, L.R., 1963. A model of item ambiguity in personality assessment. Educ. Psychol. Meas. 23,467 492.

Hanley, C., 1962. The 'difficulty' of a personality inventory item. Educ. Psychol. Meas. 22(3), 577 584.

Posner, M.I., 1964. Information reduction in the analysis of sequential tasks. Psychol. Rev. 71, 491-504 .

Rogers, T.B., 1971. The process of responding to personality items: some issues, a theory and some research. Mult. Behav. Res. Monogr. 6(2).

Rogers, T.B., 1973. Toward a definition of the difficulty of a personality item. Psychol. Rep. 33, 159--166.

Rogers, T.B. Secular trends and models of personality item responding. Calgary: Univ. of Calgary, ms. in preparation.

Rogers, T.B., J.C. van der Lee and A.R. Kasper, 1972. Desirability, ambiguity and response characteristics for a pool of items controlled for length. Research Report 9. Calgary: Univ. of Calgary.

Sternberg, S., 1969. The discovery of processing stages: extentions of Donder's methods. Acta Psychol. 30, 276 -315.