FULL REPORT OF RESEARCH ACTIVITIES

Background

There has been a recent upsurge of interest in individual differencesin reasoning which has been well summarised by Stanovich & West(2000). The reason for this is that certain types of individualdifferences can shed light on unresolved theoretical debates withinthe psychology of thinking and reasoning.

One of the major debates concerns human rationality: the extent towhich humans are capable of rational, logical thought. There arenumerous instances of people giving non-rational answers toreasoning problems and the question arises as to whether thisdemonstrates lack of rationality or whether there are alternativeexplanations. It has been claimed by some that the incorrectnormative model has been applied, and that humans are rational butnot in the sense of adhering to the tenets of logic. One line ofevidence which goes against this is the well-established finding thatpeople higher in intelligence are more likely to produce the logicallycorrect answer to reasoning problems. This seems to indicate that thecorrect normative model has been applied (Stanovich & West, 2000).

A related issue concerns the existence of two different types ofthinking. It has been claimed by many that there are two ways ofthinking, one slow, conscious, rational and controlled, the other fast,unconscious, experiential and automated. Unfortunately these aregiven different names by different people and are also defined indifferent ways (see Newstead, 2000). Nor is there any accepted wayof determining which type of thinking is being used or of determininghow they relate to each other. Once again, individual differences maythrow light on this. Recently-developed tests such as the Rational-Experiential Inventory (REI) (Pacini & Epstein, 1999) have thepotential to measure the extent to which a person relies on rationalthought or prior experience. Investigating how such thinking stylesrelate to reasoning performance can indicate which mode of thinkinghas been applied. Previous research, principally on statisticalreasoning, has indicated that rationality as measured by the REI,which is essentially a measure of the willingness to engage in logicalthinking, is related to accuracy on these problems. On the other handexperientiality, which is the tendency to use intuition and to rely onprior experience, is related to susceptibility to biases.

Objectives

There were a number of objectives in the present researchprogramme.

To determine the reliability and usefulness of the REI. This measurehas been developed and used only in the USA, and an importantpreliminary to the present research was to establish whether thescales are robust and suitable for use on a UK sample.

To replicate the correlation between intelligence and performance ondeductive reasoning tasks such as syllogistic reasoning and theWason Selection Task. In addition, the aim was to look at therelationship between intelligence and statistical reasoning tasks, andto extend this to other reasoning tasks not previously used.

To investigate the relationship between thinking styles andperformance on reasoning tasks. The initial interest was primarily inrationality and experientiality as measured by the REI. It waspredicted that rationality would tend to correlate with overallaccuracy and experientiality with biases in responding. Our researchalso went beyond these measures of thinking style and also used theThinking Disposition Composite, the Approaches to StudyingInventory, and a range of measures designed to investigate the abilityto produce different representations of the information presented.

It should be noted that the objectives of the present research were tosome extent modified as the research progressed in order to takeaccount of novel findings which emerged. In particular, the firstseries of experiments produced results which were completelyunexpected and flew in the face of previous experimental findings.Rather than blindly following our original plan of research, whichwas in any case substantially undermined by these anomalousfindings, we were inevitably side-tracked into investigating thesource of the discrepancies. This led to us carrying out extendedstudies using a much wider range of individual differences measuresthan anticipated and a much wider selection of deductive reasoningtasks.

This search for an explanation of our findings was ultimatelysuccessful and led to novel empirical findings which haveconsiderable theoretical importance. However, it meant that we hadless time to spend on the latter part of our initial project proposal,

with the result that we did not, as planned, investigate individualdifferences in statistical reasoning.

Methods and Results

In total eight studies were conducted, three more than were outlinedin the original proposal. The method and results for each of these willbe outlined in turn.

Study 1

In Study 1 we validated the REI using a postal sample of 128members of the general population and 98 undergraduate students.For the student sample we also administered a test of generalintelligence. The internal and test-retest reliabilities of all the REIscales were high and the factor structure was exactly as expected.Consistent with previous findings, men scored higher than women onrationality whereas women scored higher than men on experientiality.These scales were unrelated to general intelligence.

This study confirms that the REI is a scale with good psychometricproperties and one which could be used effectively on a UK sample.

Study 2

Study 2 examined the relationship between the REI, intelligence, andperformance on abstract and realistic versions of the Wason selection task.This task is probably the most widely used in the psychology of reasoning.In the abstract version people are given four cards which have a letter onone side and a number on the other. They can see the top side of the cards,for example A, D, 4 and 7, and are asked which cards they need to turnover to determine whether the rule ‘If there is an A on one side of the cardthen there is a 4 on the other side’ is true. The correct answer is to turnover the A and 7 cards, the latter because it would disprove the rule ifthere was an A on the other side. However, the great majority of peoplegive the wrong answer, saying it is necessary only to turn over the A and 4cards. Certain realistic versions of the task, e.g. using the rule ‘If a personis drinking beer they are over 18’, have been shown to produce massivefacilitation (Griggs & Cox, 1982).

It was expected that there would be a correlation between intelligence andperformance on abstract selection tasks since this is the typical finding inprevious research (e.g. Valentine, 1975; Stanovich & West, 1998a). It was

also expected that the measure of experientiality would correlate withperformance on realistic tasks since the correct response has been assumedto derive in part from the use of prior experience. No prediction was madeconcerning correlations with rationality.

The 98 participants, were given the 40-item version of the REI(Pacini & Epstein, 1999), and Part 1 of the AH5 intelligence test(Heim, 1968). Four selection tasks were used, two abstract and twodeontic. The abstract problems were the original letter-numberversion (‘If a card has an A on its letter side then it has 4 on itsnumber side’) and a destination version (‘If Glasgow is on one side ofthe ticket, then train is on the other side of the ticket’). Theinstructions asked participants to indicate which of the four cards(e.g., A, K, 4, and 5 in the first example) needed to be turned over totest whether the rule was true or false. The two realistic tasks were ananglicised version of the Sears problem in which Sears becameDebenhams (‘Any sale over £30 must be approved by the manager,Mr Jones’) and the drinking age problem (‘If a person is drinkingbeer then the person must be over 18 years of age’). The instructionsasked participants to choose the cards they thought needed to beturned over in order to test whether the rule was being violated.

There was the usual facilitation of realistic over abstract selectiontasks. However, the correlations between the selection tasks and theindividual differences measures were quite unexpected. In contrast toprevious findings, there was no correlation between intelligence andperformance on abstract tasks. However, there was a significantcorrelation between intelligence and the realistic problems. Thesesurprising findings challenge the received wisdom in this area.

Neither of the REI scales correlated with performance on theselection tasks. We had predicted that experientiality would correlatewith accuracy on the deontic (realistic) tasks.

One possible explanation for the discrepancy with previous findingsis that we had used an inappropriate measure of intelligence or thatthe measures of deductive reasoning were unreliable or inaccurate insome way. In order to check on this, we decided to bring back asmany of the 98 participants as we could to repeat and extend thetesting we had carried out.

Study 3

In this study, 44 of the participants used in Study 2 were given theREI and the same selection tasks as before. In addition they weregiven a further measure of ability, the AH4 Parts 1 and 2 (Heim,1967). Letter distance estimation, a measure of working memory, wasalso used. Sixteen syllogistic reasoning problems were used whichlead to conclusions which were either valid or invalid and eitherbelievable or unbelievable. This is an example of a valid, believablesyllogism:

No dogs are unhappySome mammals are unhappyTherefore, Some mammals are not dogs

The short form of the Approaches to Studying Inventory was used(Newstead, 1992) which measures three approaches to studying. Thiswas included mainly for exploratory purposes.

The various individual differences measures were found to begenerally reliable. However, the selection tasks were less reliable,giving low test-retest scores. Nevertheless, the classic facilitation ofrealistic over abstract tasks was once again obtained.

There was again no correlation between intelligence and performanceon abstract selection tasks; in fact the correlations were(nonsignificantly) negative. As in Study 2 there were significantcorrelations between intelligence and performance on realisticselection tasks. There were no significant correlations between eitherexperientiality or rationality and performance on any of the selectiontasks. The unusual and completely unexpected pattern of findings onthe selection task observed in Study 2 was thus confirmed in Study 3.

With the syllogisms, the classic findings were broadly confirmed.There was a main effect of logic, with participants endorsing morevalid than invalid conclusions. They also endorsed more believablethan unbelievable conclusions, though this effect was not significant,and there was an interaction between these two variables.

The main interest is in how these variables relate to measures ofindividual differences. Intelligence was significantly negativelyrelated to belief; in other words, people with higher levels ofintelligence were less likely to respond simply in line with thebelievability or otherwise of the conclusion. The correlation betweenintelligence and logic was not significant overall, but on thosesyllogisms where belief and logic were in conflict the correlationachieved significance. Participants of higher ability are thus able to

resist the belief-based response and provide a solution consistent withthe logical requirements of the task. Rationality was related to neitherbelief nor logic, but experientiality showed a negative relationshipwith logic. The findings on syllogistic reasoning were thus similar towhat has been obtained in previous research.

Although Study 3 replicated the findings of Experiment 1, the samplesize was relatively small. The next study used a larger number ofparticipants and additional tasks.

Study 4

This study used 152 participants and similar materials to Study 3. TheAH4 and REI were used, and the same 16 syllogisms. Eight selectiontasks were used, four abstract and four realistic. The ASI and theletter distance tasks were dropped, but a further measure of thinkingstyle, the Thinking Dispositions Composite (Stanovich and West,1998b) was included. It was hoped that this might provide a measurerelated to intuitive thinking, though in actuality it produced fewresults of interest.

There were no significant correlations between selection taskperformance and any of the individual differences measures. Theunexpected positive correlation between intelligence and deontictasks was not significant in the present study. However, it waspositive and on two of the tasks (the two we had used in Experiments1 and 2), the correlation was significant. There was no overallcorrelation between intelligence and abstract selection tasks thoughfor the first time in this series of studies there was such a correlationon one of the tasks, the original letter-number version.

On the syllogisms, intellectual ability correlated with logic scores andwith scores on conflict problems (i.e., those where belief and logicpoint to different conclusions). In addition, rationality (as measuredby the REI) correlated positively with performance on nonconflictproblems. Experientiality did not correlate with any aspect ofreasoning performance.

Thus these results broadly confirmed the findings of the earlierexperiments. The curious correlation between ability andperformance on realistic selection tasks was found once again, thoughonly on the two selection tasks used in earlier studies. In thisexperiment, for the first time, we observed a link between ability and

abstract tasks, though this was observed on only one problem, thestandard letter-number task. We were still unsure as to why ourresults were out of line with previous research and decided to carryout another study, this time on sixth form school students.

Study 5

Participants were 126 secondary school children aged between 16 and18. The REI and AH4 were used and also an additional measure ofintellectual ability, Raven’s Progressive Matrices (Raven, 1983).Eight selection tasks were used, four abstract and four realistic, andthe syllogisms task as presented in Studies 3 and 4. AH4 and Raven’sMatrices scores correlated significantly and hence a compositemeasure of intellectual ability was used.

On the selection tasks, there was a significant but much smaller thanusual facilitation on the realistic tasks. The ability compositecorrelated with only one of the four abstract tasks and rationalitycorrelated significantly, but negatively, with one of the other abstractproblems. In contrast to the previous experiments there was noevidence that realistic tasks correlated with ability.

The pattern of results for the syllogisms was somewhat different tothat normally observed. There was a smaller than usual effect ofvalidity but a very large effect of belief, and the interaction betweenthese factors did not approach significance. Intellectual abilitycorrelated with logic scores and with accuracy on nonconflictproblems. However, in contrast to the previous experiments, abilitydid not correlate with conflict problems nor did it correlate withbelief. There were no correlations between syllogistic reasoning andeither rationality or experientiality.***[more here on data?]In some ways the results from this study are even more discrepantwith previous findings than our earlier studies, though theparticipants were younger than are typically used. We began tosuspect that our samples might have been different in some ways tothose used in earlier studies, though at this stage we were not surewhether this was to do with motivation, ability or some other factor.We felt we needed more data, and using different kinds reasoningtasks and different measures of individual differences.

Study 6

In addition to the measures used in previous studies (AH4 and REI),Study 6 used a new type of measure of thinking style involvingmeasures of the ability to generate alternative representations.

The highly influential mental models theory of reasoning (Johnson-Laird & Byrne, 1991) claims that the construction of alternativerepresentations is a key factor in reasoning. People are assumed toconstruct a representation (or model) of the premises, generate aconclusion, and then attempt to produce alternative mental models inwhich the conclusion does not hold. If this cannot be done, then theconclusion is assumed to be valid.

Torrens, Thompson & Cramer (1999) used a measure of alternativesgeneration involving the construction of diagrams of different modelsof syllogistic reasoning premises. They found that it predicted wellcertain aspects of reasoning performance, a finding that has beenconfirmed by Newstead, Thompson & Handley (in press). We usedthis measure and also a well known measure of creativity, the uses ofobjects test, and a measure of possible antecedents for conditionalstatements first used by Markovits (1984). This latter involvesproducing alternative ways in which the consequent of a conditionalmight be true. For example, given the conditional statement If it rainsthen the road will get wet participants were asked to indicate as manydifferent ways as possible for the road to become wet. In addition weused two tasks not previously used. One of these involvedconstructing unlikely scenarios to explain apparently impossibleevents, for example how a murder suspect might be guilty eventhough he was on a train some miles away when the murder occurred.The final task involved listing potential experimental confounds in asimple study. Our interest was both in how these various measures ofalternatives generation related to each other and in how they relatedto reasoning performance.

In addition to selection tasks and syllogisms we also used fourversions of the THOG problem (Wason and Brooks, 1979) and aconditional reasoning task involving shapes and colours. 70participants were used in this study.

Two new measures of individual differences used for the first time inthis study produced results of particular interest. The alternativesgeneration task in which diagrams representing syllogisms had to beproduced correlated with intelligence and with performance onabstract selection tasks. Multiple regression showed that it was

predicting variance in performance on these abstract tasksindependently of intelligence. The other measures of alternativesgeneration correlated with each other but did not serve as goodpredictors of reasoning performance.

The second measure of interest was one derived from the conditionalreasoning task: the tendency to resist the fallacious inferences. Thefallacious inference of affirmation of the consequent involvesinferring that p is true when given the conditional statement If p thanq, and the further information that q is true. Denial of the antecedentinvolves fallaciously inferring not q when given the same conditionalstatement and the further information that p is true. Both of theseerrors are extremely common. However, there are many people whoresist them and this tendency correlated with intelligence, withperformance on abstract selection tasks, with performance on deontictasks (though to a lesser extent), with logic scores on the syllogismsand with accuracy on conflict syllogisms.

Perhaps the most surprising aspect of the present results, however,was that the classic pattern of correlations on the selection tasks wasobtained: a significant correlation between intelligence and abstracttasks and a positive but nonsignificant correlation on the realistictasks. What is more, our data give a powerful clue as to why thispattern should have returned. In the present study the overall level ofintelligence was in line with the published norms for the AH4 test.Re-analysis of the scores from Studies 2-5 revealed that in each casethe intelligence scores were lower than the norms. In other words, wehad, quite by chance, sampled the lower end of the ability range inour earlier studies.

But why should this lead to the correlations with intelligencechanging? We believe we have evidence that there are three mainlevels of performance on the selection task. At the first level arecontext specific reasoners who treat every task as independent fromothers and use whatever contextual cues may be present in thescenario. At the second level are pragmatic reasoners who realise thatthere are commonalities between different selection tasks but fallprey to the pragmatic inference present in conditionals, for exampleinterpreting them as biconditionals. At the highest level are theformal reasoners who can overcome contextual cues and analyse thelogical structure of the problems. We believe that in Studies 2-5 wehad virtually no participants in the formal reasoning group, hencethere were very few correct answers on the abstract tasks and no

correlations with intelligence. Since we were sampling only from thecontext specific and pragmatic reasoners, a correlation betweenintelligence and performance on realistic tasks would be expected,since the more intelligent participants were those who are capable ofusing the pragmatic cues present in realistic tasks.

To test these suggestions we conducted extensive re-analyses of thedata from Studies 2-5. We assumed that context specific reasonerswould give different responses to the various abstract tasks that weregiven while pragmatic reasoners would give consistent responses. Inall of the experiments, consistent responders were of higherintelligence than inconsistent responders. Pragmatic reasoners wouldbe expected to give the cards named in the rule as their responses toabstract tasks since these are cued by the conditional statement. Theywould also be expected to give the correct answer on the deontictasks. Analyses indicated that those who selected the cards named inthe rule were of higher intelligence than those who did not, and thosewho gave this response plus the correct answer on deontic tasks werealso of higher intelligence.

The findings in Study 6 on conditional reasoning were also ofinterest in their own right. In particular, it was found that logicalperformance on most logical inferences correlated with intelligenceand alternatives generation, but that the modus tollens inferencebehaved very differently. This is the (valid) inference that not pfollows from not q. This inference was independent of intelligencebut correlated with belief bias, suggesting that it may be drawnthough pragmatic cues rather than logical analysis.

Study 7

The aim of this study was to look in more detail at individualdifferences in a variety of propositional reasoning tasks, thusreplicating and extending the findings of Study 6. This study used avariety of individual differences measures including the AH4,Markovits alternative antecedents task, uses of objects, experimentalconfounds, unusual scenarios, the REI, and possible diagrams. Inaddition to a variety of propositional reasoning tasks, various typesof abstract selection tasks were also used.

In the propositional reasoning task, the 102 participants were given64 statements containing abstract statements involving letters andnumbers. The connectives used were if, either … or but not both,

either or … or both, and if and only if. Negation was systematicallymanipulated in the antecedent and consequent of the first premise, aswas affirmation or denial of the antecedent and consequent in thesecond premise. An example of a statement using if and only if, andwith negated antecedent and affirmative consequent is and with asecond premise containing a denial of the consequent is:

If and only if there is not a 3, then there is a ZThere is not a ZWhich of the following conclusions is valid:

(a) There is a 4(b) There is not a 4(c) It is uncertain whether or not there is a 4

Participants also received abstract selection tasks using letters andnumbers, the same connectives, and with negation of the antecedentand consequent systematically varied.

As in Study 6, intelligence correlated with correctly resisting theaffirmation of the consequent and denial of the antecedent fallacies.It also correlated with correct performance on inclusive and, to alesser extent, exclusive disjunctives. The alternatives generation taskwas slightly less good a predictor in the present study, butnevertheless correlated significantly with performance on exclusivedisjunctives and conditionals. However, neither task predictedperformance on biconditionals, suggesting once again thatperformance on these is affected by more pragmatic factors.

***

Study 8

The AH4, the Markovits possible antecedents task, uses of objects,unlikely scenarios and uses of objects were all used in this study. Asin Study 7, the main experimental tasks involved conditionalreasoning and the selection task. In the conditional reasoning task,half the 114 participants were presented with conditional statementsin which there were many alternatives, for example ‘If it is a troutthen it has gills’. Since there are many animals other than trout whichhave gills, it is easy to bring to mind instances in which theantecedent is false and the consequent true (i.e. non-trout which havegills). The other half of the participants received statements in which

there were few alternatives, for example ‘If it is a fish then it hasgills’. They also received statements using abstract material.

In the selection task, the participants were given problems which alsoused statements with either many or few alternatives. For example, inthe many alternatives condition one of the selection tasks used thestatement ‘‘If a card has ‘cat’ on one side then it has ‘backbone’ onthe other side’’. Abstract selection tasks were also used.

***

References

Griggs, R.A., & Cox, J.R. (1982). The elusive thematic materialseffect in the Wason selection task. British Journal of Psychology, 72,407-420.Heim, A.W. (1967). AH4 group test of intelligence. Manual. London:National Foundation for Educational Research.Heim, A.W. (1968). AH5 group test of intelligence. Manual. London:National Foundation for Educational Research.Johnson-Laird, P.N., & Byrne, R.M.J. (1991). Deduction. Hove, UK:Lawrence Erlbaum.Markovits, H. (1984). Awareness of the ‘possible’ as a mediator offormal thinking in conditional reasoning problems. British Journal ofPsychology, 75, 367-376.Newstead, S.E. (1992). A study of two ‘quick and easy’ methods ofassessing individual differences in student learning. British Journal ofEducational Psychology, 62, 299-312.Newstead, S. E. (2000). Are there two different types of thinking?Behavioral and Brain Sciences, 23, 690-691.Newstead, S.E., Thompson, V.A. & Handley, S.J. (in press).Generating alternatives: A key component in human reasoning?Memory & Cognition.Pacini, R., & Epstein, S. (1999). The relation of rational andexperiential processing styles to personality, basic beliefs, and theratio-bias phenomenon. Journal of Personality and SocialPsychology, 76, 972-987.RavenStanovich, K.E., & West, R.F. (1998a). Cognitive ability andvariation in selection task performance. Thinking and Reasoning, 4,193-230.

Stanovich, K.E., & West, R.F. (1998b). Individual differences inrational thought. Journal of Experimental Psychology: General, 127,161-188.Stanovich, K.E., & West, R.F. (2000) Individual differences inreasoning: Implications for the rationality debate? Behavioral andBrain Sciences, 22,Torrens, D., Thompson, V.A., & Cramer, K.M. (1999). Individualdifferences and the belief bias effect: Mental models, logicalnecessity, and abstract reasoning. Thinking and Reasoning, 5, 1-28.Valentine, E.R. (1975). Performance on two reasoning tasks inrelation to intelligence, divergence and interference proneness.British Journal of Educational Psychology, 45, 198-205.Wason, P. C., & Brooks, P. G. (1979). THOG: The anatomy of a problem.Psychological Research, 41, 79-90.

Activities

Attendance by investigators at the following conferences:

Experimental Psychology Society/Canadian Brain and BehaviorSociety conference, Cambridge, July 2000 (Newstead and Handley).

Fourth International Conference on Thinking, Durham, August 2000(Newstead and Handley).

BPS Centenary Conference, Glasgow, March 2001 (Newstead).

*** [others?]

Outputs

Newstead, S.E., Handley, S.J., Harley, C. & Wright, H. (2000).Individual differences in deductive reasoning. Paper presented to theFourth International Conference on Thinking, Durham, August.

Newstead, S.E. (2001). Individual differences in reasoning. Paperpresented to the BPS Centenary Conference, Glasgow, March.

Newstead, S.E. (2000) Are there two different types of thinking?Behavioral and Brain Sciences, 23, 690-691.

Newstead, S.E., Handley, S.J., Harley, C. Wright, H. & Farrelly,D.(in submission). Individual differences in deductive reasoning.

Submitted to Quarterly Journal of Experimental Psychology.[Enclosed]

Handley, S, Newstead, S.E. & Wright, H. (2000) Rational andexperiential thinking: a study of the REI. International perspectiveson individual differences, 1, 97-113. [Enclosed]

ImpactsThe research has already attracted considerable interest both viaconference presentations and published articles. Stanovich hasreanalysed some of his own data and has confirmed our claim that thecorrelation between intelligence and performance on abstractselection tasks is moderated by the level of intelligence of theparticipants. This does not undermine Stanovich's theoreticalapproach but it does mean that certain caveats and restrictions arerequired.

***Future Research Priorities

Our research has confirmed the usefulness of individual differencemeasures in the study of individual differences. However, thus farsuch research has only scratched the surface. Clearly the range ofindividual difference measures and reasoning tasks that could be usedis large. At the same time our work also illustrates some of theproblems with such research. Because the individual differencemeasures are not perfectly valid and reliable and because there aredifferences between populations, it is quite common for results not toreplicate perfectly. In our research we found a plausible andtheoretically interesting explanation for the discrepancy between ourdata and those of earlier studies, but this will not necessarily alwaysbe the case.

Our results point in particular towards there being importantdifferences between individuals of different levels of intelligence inthe way in which reasoning tasks are carried out. They suggest thatthere is a progression from responding purely on the basis ofcontextual cues, to the use of pragmatic, conversationally-based cues,to the ability to decontextualise material and engage in high levelabstract thought. Further study of this progression on other tasks isclearly indicated. More importantly, these progressions probablyoccur developmentally as well as in adults of differing levels ofintelligence. There is existing evidence to support this claim but

further investigation of the development of the understanding oflogical connectives and their use in logical reasoning tasks is needed.

***