Loughborough UniversityInstitutional Repository

Objective examinations inA-level mathematics

This item was submitted to Loughborough University's Institutional Repositoryby the/an author.

Additional Information:

• A Master's Dissertation, submitted in partial ful�lment of the require-ments of the award of the Master Of Science degree of LoughboroughUniversity.

Metadata Record: https://dspace.lboro.ac.uk/2134/13116

Publisher: c© John Francis Potter

Please cite the published version.

This item was submitted to Loughborough University as a Masters thesis by the author and is made available in the Institutional Repository

(https://dspace.lboro.ac.uk/) under the following Creative Commons Licence conditions.

For the full text of this licence, please go to: http://creativecommons.org/licenses/by-nc-nd/2.5/

LOUGHBOROUGH

UNIVERSITY OF TECHNOLOGY

LIBRARY

AUTHOR

....... .. e..OTI~(S"i" ... ;.;:r .. ,£., ................................................................... .

. .

.... ~?p..: .. ~.?· ........ O'.I.31~/OL ........................................................... . VOl NO. CLASS MARK

--z. ~~.·i 1980

OBJECTIVE EXAMINATIONS IN

'A":'LEVEL MATHEMATICS

· by

JOHN FRANCIS POTTER,B. Se.

A.~~ster\s Dissertation submitted in partial

fulfilment :of the requirements for the award of

,the degree of M. Se. in Mathematical Education of

, Loughborough Univ:ersity of Technology, December 1975.

Supervisors :P. E.' LEW~S, M. Sc., Ph. D.

D. ,T. KELLY, B.Se.

"

@ by JOHN FRANCIS POTTER, 1975

L,\,_

..

• :-!. i ~.: -' .>

t"". i.:,o. J i ,~,) .; ~ .

... '~_, ~l ....... ' ~J, ,

.C ,.,' ,!' v ,

I';:. v, :'1; .• " ... ,-,," ~ ,- ... , -.... ~.' .• 1. .1_."

-'-'. ~,- ." . .. ; . . ..1\ • ..:.., "', '. '_' 'J.

pa s .

" Acc, OI~T~9/OI No,

, .:

. ABSTRACT

A review of the general theory and' uses of objective tests was

'made with particular reference to 'A'leve1 'work., The extent to

which these tests are used in this country up to,the.present was

examined. In Mathematics at this level no objective tests have yet

been used in public examinations and 'it 'is only the University of

Lona.on School Examinations Department which has made sufficient'

progress to be on the point of. combining these tests with

traditional style papers •. Objective tests are easy to administer,'

very easy to mark with absolute 'accuracy 'and :have considerable' .

advantages of : effectiveness compared 'with traditional" papers.

However, :they are Ill\lchmore difficult ,to 'construct·than the latter.

'Adetai1ed examination was made of.the advantages and disadvantages

, of objective ,examinations. , " " 1

An objective type Pilot Test was constructed with details' . ,- -, .

given of the method of forming items and their options.' The Test ,

of 45 items, to be attempted inH'hours}' was·givento 68

candidates from the UpperV1th Forms of three schools. The Test

was taken a few weeks beforethe'1~:n5 'A' level examinations.

The content otthe Testwasbasedon;the: same syllabus' (NUJMB'

MathematiCS, Syllabus A) studied by ,the candidates in their . '

V1 th Form course. 67· of _ the candi'da-Jes actually sat th.e otficia1

'A' level ex:aminations.

The Pilot'Test andits'detai1ed analYSis ~oth as a whole and

i temby , item) wel'e used to illustrate ·the . difficulties of

constructing an objective test single-handed and,with'the' G.C.E.

'A' level resu1ts~' to' examine the correlation between the two

types, of paper using the 'same syllabus.

The results' showed some glaring fau1ts,with individual items in

. the Pilot Test and the test as'a whole was rather,too difficult'

with a shortage of time. ,-The Kuder-RichardsonForIll\lla 20

produced a .resu1t Ot 0.68 with a standard e.rltorot2.9 showing a

. reliability/

3

reliability which:is,lowerthan that desirable.' :The range of

marks:was from 1 ,to 28.:out:of '45 which 'was 'reasonably distributed . .,

but did not give sufficientd:i.scrimination.' '1

Despite'thefaults;of the objective 'test, the'comparison with

the I A' 'level ; re suI ts produced a 'very reasonable mn$:: difference

correlation of O. 61'(Spearman). ',,'

There would seem to be ,strong 'grounds for 'suggesting 'tha.t pre­

tested'objective.tests, 'constructed 'by 'teams , of experts; can be

legitimately combined with t~aditiona.l tests to give an examination!

which is effective 'both on a wide front of the syllabus and in'

depth ,in particular areas.

, , ,

I

ACKN'07iLEDG:ii:i.'lliNTS

The kind assistance of the following is 'gratefully ,

acknowledged: - i .

Uni vers.i ty of -London; Entr:n:e and' School Examina tions Council.

for their permission to .use pre-test:rnaterials.-

4

Dr. R. ;Wood,SeniorResearchOfficerr UniversitY'of London School

Examinations Department. ' ,

The Northern Universities Joint Matriculation Board 'for their .'

permiSSion to reproduce Mathematics (Advanced) . Syllabus 'A,1975,

examination papers •.

. Mr. J. Gillespie .. Head' of. Mathematics at· Carl ton-le-Willows . , .

. ComprehensiveSchool,Nottinghamj 'and his Vlth Form candidates •.

Mr. R. T.Burton, The 'Gateway School, Leicester, 'and 'his Vlth .'

. Form candidates~ ,.

The Vlth Form candidates at MagnusGrammar School'j'Newark,Notts.

CONTENTS '. i '

,1. Objective:Examinations in general.

'2. Advantages and disadvantages of Objective Examinations.

3. Purpose 'of Pilot 'Test.:' .. , , .

4. Construction of. Pilot 'Test. . "i

5.' The Pilot' Test. : '

6. ',' Construction of Section 'r Multiple 'Choice ,items. '" '

7. Analysis of 'Pilot·'Test. ' '" '.

,8. Comparison with 'A" level results.""

9. Conclusions.-:'

, .,'

5

6

The phrase:'objective-type examinations' needs.to be'clarified

~nd explained since any examinations. are, or. should be, concerned

with obj ectivesand the very word. 'objective' has 'different'.

interpretations. Here,in teaching, I amassuming that an

'objective' is ,the successful conclusion ·of a learning experience

- it is a specific behaviour which' is required to build up the

long term aims of an educational course.' . The word. 'objective' in

the·descriptive·titleof examinations has. come to mean the.

opposite of' subjective' • Arguments and dis'cussions on the

validity of educational assessments tend to divide the,field into

two camps i- the traditional examinations'whichare labelled

'subjective'and :thenewer 'objective' 'examinations. 'It must be

stressed:at the'outset,that this labellingof'examinations refers'

solely to the methods of marking since the answers to traditional

type questions; even when brokendovm into constituent 'parts,can

often'produce 'considerablegenuinely'differing'asssssments

del?endingon the marker, while objective type questions have pre­

determined answers which are'sopreciselydefined'that the'

marking can be automatic. 'In other words the 'subjective' or :

'objective' title of each type of examination is,here, nothing

to do.with the objective of the course, ,the effect of which is!

being assessed, 'nor:does it have anything.to do with the

construction of the examinat,ion. As will appear later in this

dissertation, when the Pilot Objective Test is critically

examined,many of the construction faults·are subjective in'nature.

In the article by R. Wood (1967), ,"Objectives in the~Teaching of

lVIathematics", in the journal'''Educational Research" he says that -

'Lewis (1965 ) remarked that much of the dissatisfaction with

traditional methods of assessment is almost certainly due to

differing views ,about the underlying obj ectives •. It is 'arguable '

that 'differing views' should be amended to.' ignorance' •

7

Can anyone honestly provide. a description of wha:t mathematics

achievement at CSE or GCE level currently means?' . He is suggesting

that the traditional forms of examination papers dO.notencourage·

bringing specific mathematical objectives out into the light. of

day and by their very nature obscure any objectives which may have

been implicitly considered. On. the other hand 'objective-type'

examinations can be systematically built up to cover a defined

range of objectives. For example, based partly on Bloom's (1956) .', - '", ,', .

Taxonomy of Educational Objectives in the cognitive domain:

Knowledge

Comprehension, .

Application,

Analysis

Synthesis

Evaluation.

there is Wood's suggested classification of behavioural objectives'

(Wood R. (1967) - Item Bank project):-

A. Knowledge and information: recall of definitions,

notations,concepts.

B. Techniques and skill: computation, manipulation of

symbols.

C. Comprelmsion: capacity to understand problems, to

translate' symbolic forms, .to follow and extend reasoning:

D. Application: of appropriate concepts in ,unfamiliar

mathematical situations.

E. Inventiveness: reasoning creatively in mathematics •

. ' Aswi th Bloom's Taxonomy this classification is hierarchical

though some sub-divisions must overlap to a certain extent,' When

constructing a test the greatest difficulties arise when devising

tests'for the higher order objectives. As Williams (1967) says:

'It is relatively easy to define and construct tests of ability

to/

8

'.-:, .. , "tolbb~pUte, this being little more thanthe ability to pe~form'

certain specifiable kinds of operation- an ability which we,can

test by requiring the performance of a representative sample of

such operations •. However, as .. we at present use the term"

'understanding' is much more difficult to define, and it is more

difficul t to, know when we are ~naging to t~st i t-;' for we have' not

decided upon the precise behavioural implications of imputing it

to a performance or upon the kinds··of.·achievement that we can . .

take as its token." Rather .more bravely, Ma'thews (1974) in the

'report of the Chemical Society argues that objective tests can be

used to test understanding, application and evaluation. as well as

their more commonly appreciated use as a test' of factual recall,

and that they can be used to test, quite high levels of

attainment. He says:- , There is reason to suppose that objective

tests could be used at least to pass degree level and possibly as

an element in honours' examinations ••••• it must be stressed

though that tests of this sort do not give much scope to the

,divergent and imaginative .. thinker.' The emphasis that objective

type questions need not be' restricted to simple memory tests is

underlined in the 1961 booklet of the Educational Testing Service

of Princetonl~ New Jersey, which says in its opening paragraph:-:-~ , ' , ,

The purpose of this close look at a group of multiple-choice

questions is to dispel a myth: the myth of the multiple-choice

question as a superficial,exercise-:-one that requires little

thought,' less inSight, ,and no understanding. Like other myths,

this one may be based on a shadowy memory of the past, but it bears

little relation to present reality. "What is often overlooked ••• ,"

. writes Jerome Bruner in 'The Process of Education' , "is that

examinations can also be allies in the battle to improve curricula

and teaching. When an examination is of the 'objective' type

involving multiple chOices or of the essay type, it can be devised

sol

, '

, "~so' as'%o' em:phasisei'iil.AiundElrsH,nlJ.ing'bf~thetbroad'principles of a

subject. Indeed, even when one examines detailed knowledger it

can be done in such a way as to require understanding by the , , ,

student of the connectedness between specific facts." Neverthele

the use of objective-type examinations in this. country has been

slow to develop from the testing of lower levels of attainment to

the higher levels}' particularly in mathematics. Objective tests

have been used in the GCE in Science and General Studies by the

NUJI.'rB, the AEB and the University of London and the latter started

including '0' level mathematics objective-typ~ tests in 1970. So

far, only the University of London has made any progress in this

direction in 'A' level !l1athematics. Pre-tests in 'A' level

Mathamatics and Further Mathematics were organised in 1973 and

1974 with the aim of including the revised tests in the GCE

examinations in 1977· The items (questions) in the pre-tests were

chosen to cover the full range of syllabus topics, to spread

reasonably over levels of ability and to include a chosen

combination of objectives from the lowest to. the highest though

based on a simplified (and therefore somewhat crude) system, i.e.

recall

basic skills.

understanding .'

Thus the structure of the test pa.persdepended on.the purpose and

decisions' of emphasisy'such:decisions beil1gsubjective'in na.ture.

10 ADVANTAGES AND DISADVAHTAGESOF OBJECl'IVE·EXAMINATIONS

Objective-type'e:x:aminations have a number'of'adva,ntagesover

traditionalexaminatioIis~' 'The larger nUIuber,'ofitems '(usually

40 to'50) 'compared with 5 'or 6 traditiOnal questions taking the

same time, enables a far wider coverage of the syllabus ,to be made

It is 'possible ,to include almost every topic, in 'the,'syllabus 'at

one level:ofability or another while the traditional paper is'

considerably 'restricted. There'are many cases of 'traditional'

'A' levellllathematics, papers emphasising calculus, say, inone .:'

year, ,to" the, exclusion of lIlUch of the co-:ordina te geo:[lletry i 'while

in the ,following year the emphasis on particular 'subject topics!is

completely changed. This leads to the well knovm practice 'of

"question spotting" and ,the examinations lose considerably'in

reliability, Objective-type 'multiple choice papers avoid this

difficulty particularly as "item-banks" can be ,built up offering' ,

a choice of items which can be ,combined in a paper to cover the

required range of topics. Again the marking of objective papers

is rapid and automatic. No decisions have to be made'by the'

marker whereas in the case of the marking of traditional papers,

even when great care is taken to subdivide the answers and lay

down a rigid framework of marking, there always 'remain many value,

judgements to be made; The marking of an objective test is

speeded up by the simplicity of the usual scheme ~ zero for a wronl

answer and one for a 'correct answer, whatever the ,difficulty. ,

This is justified ,only if the number of items is fairly large (thu

covering the whole range of abilities and diffiCulties) and if the

number of candidates is also fairly large. It is also assumed

that each item stresses only one, or at the most two, of the cours

higher level objectives. ' 'This latter is ,very difficult' to

achieve when planning items for "advanced" course topics with a

higher,objective (e.g. reasoning creatively or applying

mathematics to an'unfamiliar situation) ata "hard" level of .

diffi culty /

11

. difficulty. In siwp1e terms, to produce an item to assess·

'understanding' (the lower level objectives of simple recall and ,

use of bastc skills being also involved) in 'A' level mathematics

requires. considerable ingenuity. . The usual' error is to 'produce an ;

item far more difficult than the writer intended. It is precisely

in eradicating these errors, and other test construction faults,

that objective-type tests emphasise their superiority. A

traditional paper can only be amended in its results, i.e. it is

only after the eKaminations that faults in the paper (causing

faults in the assessment) can be allowed for by adjusting the

marking scheme, the adjustments being mainly subjective onea.

Objective-type papers and their results are peculiarly amenable to

detailed analysis. Thus items used in pre-tested papers can be

carefully scrutinised and it is possible to build up a well

balanced paper of vetted items. The meticulous way in which the

results of a pre-test can be analysed in order to choose the most

suitable items for future use, is illustrated by parts of an

extract from a 1974 analysis carried out by the School

Examinations Department of the University of London:-

. '

RA\7 SCORE"

ITEM No. '28 .

OPTIONS' , :' ,

SCORErO-13

SCORE :13-19 :

SCORE 19-23,

SCORE . 23-29,'

SCORE ' 29-40

' .

,

MEAN CRITERION' " "

"

. '

,

MEAN TRANSFORIIIED

.

A

4

.04

1

0

1 "

2

0

20.25

12.53

A. L. M.AJ:HS.

rB"" C :D E iO

39 8 26 8 4

·44 .09 ;29 • 09 ,.04

1 :3 ,7 :3 2

'5 3 6 2 2

5 1 9 , :2 0

10 l' 4 ,1 0 .

,18 0 0 0 0

26.74 16.88 17.27 16.25 12.00

15.79 10.83 11.03 10.52 8.38

FUNCTIONS :BASED ON TRANSFORfllEDCRITERION SCORE: '

ITEM STANDARD DEVIATION ·50

MEAN SCORE 12.78

ITEM DIFFICULTY'INDEX, ·13.62 , .;

BISERIAL;CORRELATION ·79

POINTBISERIAL CORRELATION .62

12

~OTAL: ' , , . ~N TEST"

'89 89

1;00 ' ,

17 " 18 . ,

.18

18

18

21.19 I :

13.00

In this test each item has five options, A to E, from which ,the

. candidates 'can choose an answer. The s'econ'd now in the above

table.gives the·number.ofcandidates (total number 89),who have . * chosen each option 'and B ·is.the'correct.option. The column

headed 0 gives the"number of candidates who have omittedthis

item (4);' The :tll:trd now gives the proportions of 'the candidates

who have chosen A to E and O.In the' next five rows the

candidates have been divided into 5 bands of more or less equal

numbers (18 except for the first band of 17) . arranged in

. ascending!

13

ascending order according to their scores in the vrhole test • . .';

Of the 'most able' candidates all 18 chose the correct option

whereas of the 'least able' candidates only lout of 17 chose the

correct option. There is a steady increa'se in the number of

successful CRndidates iiithis item from theiowest abili tybandto

the highest ability band. Only '0.04 o"f the "candid"ates ami tted the

i temwhich indicates that it was well within ,the ability range.

The, O. 44gi ves the, total 'proportion of candidates successful with

the item'and this is' called the facility value of the item. The

, ,row of results labelled MEAN CRITERION gives' the means of the :rB.>W

total scores of the groups of candidates who chose each'option (the

maximum raw'total,score wa's 45)," e.g., 'the-mean total score of all

the candidates succe"ssf'ul with this particular item was 26.74

whereas the'mean total of the candidates ,who omitted the item was , , .

12.00. The 21..19 represents the mean total score of all 'the

candidates. The row labelled :MEAN TRANSFOillvlED refers to the same"

figures in the row above which have been transformed into the -

e~uivalent figures giving a mean of 13.00 and a standard deviation

\ !

of 4 (the standardised scores of the system used by the University

of London GCE and the Educational Testing SEllivice of Princeton, ,

New Jersey). The fU:nctions beloVl .the ta~le are each calculated froIL

the standardised scores.

~ is (l-p), then the,item

i.e. jO.44 x 0.56 = 0.50.

If p is the facility value ofO.44,and

rit~ndarddev~ation is,/W,

The MEAN SCORE of 12.78 is ,the . . ~ -

transformed mean,total scor~ of all the candidates who attempted

the item. The !TEM DIFFICULTY, INDEX is calculated from (4z+13)

where z is ,extracted from a table giving the unit standard normal

'variate,(standard deviation = l'and mean =,0) with p = 0.44 (see

appendix B,Table, G-; of Guilford' s nFund~mental Statistics in '

Psychology and Education", page 522). The BISERIAL CORRELATION is

given by and the POINT BISERIAL CORRELATION e~uals

- ,

14

, where Mris the mean of the transformed total

scores of the candidates answering the .. item' correctly, . MA is the

mean of the trailsform.edtotal scores of all the candidates who

item,~ i~the standard deviation of the transformed . attempted the

scores of all who attempted the item, pis .the facility value.

wi th "Cl' = 1 - p, arid' y i~ obtained' from theuni t standard normal " " ,-: • > , •

.• variate table with p= 0.44 in this case (see' Guilford appendix B,

Table G). 'Bymeans of this variety of standardised indices the

different items in the test can be compared (and compared. wi th

those in 'other tests) and suitable ones chosen to fom a pattern

of abilities, . difficulties and reliabili ties to suit' the purpose

for which the examination as a whole" is intended." . The biserial'

correlations provide particularly important pieces'of inform.ation

. as they ,are measures of the internal reliability' of the test and

they give an indication of guesswork which may have takeuplace ! I

with the solutions to a particular item. The biserial correlation l

coefficient measures the correlation between two' sets of variables

when both sets are continuous but one 'setis divided for'

convenience into two possibilities only, such as "pass" and "fail".

The point .biseriaL coefficient, on the other hand, is the' relevant

one. to' be used when one set of.. variables is, continuous but the

other set is a' genuine dichotomy. .This is the case in an objectiv

type paper 'when a correct answer is given +1 while an incorrect

a:llswer scores O •. 'Strictly, to assess the result's of a single item

it is necessary to compare with an external criterion but in the

absence of such acriteriou'it is reasonable to compare with the " .: .

results on the whole test •• ,This assu,m,es, of course, .that the

whole te'st has asatisfactoryreliabili ty which' again seems a fair'

assumption if the test is along one (having 40 to 50 items), if

1 See Appendix A for pro~f of formula. the/

15 r ,- _____ -... __

, distributed and if adequate precautions are taken in the

construction of the test. Even so, this comparison of a part with

the whole automatically includes an "overlap" which, of course,

loses its significance the more the larger. the number of items in

the test.· That this "overlap" can have a devastating effect is - - . 1

illustrated in the example from Guilford as follows:-

"Demonstration of.how index numbers may acquire a high degree of

correlation because of a common denominator (an overlap)":.

an extreme case:-

CHILD CA MA MA IQ IQ I n .. I II

A 5.0 7 I 8 140 160

B 5·5 8 8 145 145

0 6.0 7 7 117 117

D 6·5. 8 .

7 123 108

E 7.5 8 8 106 106

F 8.0 7 8 88 100

G 8.5 8 7 94 82

H 9·0 7 . 7 . 78 78

Correlation between I~ I and MA II = 0.00

Correlation between IQ I and IQ II = 0.92 .

The chronological ages (CA) form the "overlap" since they. are

d · MA (mental age. ). t· 0 ". e th· IQ There; s use ~n CA chronological age b~V e. •

considerable positive correlation between IQI and IQ II even

though the MA's produce zero correlation.

However, in the case of an objective test of 45 items, the

"overlap" produces a very small error (upward) in the pOint

.biserial coefficient.

lFundamental Statistics in Psychology and Education;, J. P. GUILFORD, Pages 321-322.

16

, it 1. Sot f~:b'6n1:yla"sound' :ttefu! Ofi't:h~l$H~1iHtrc£rHedl'611:t'by the

University of London has been analysed - sound, because all the

options ha:ve been ;"sed (the"distractors" have beem effective:" - -

although option A (0.04) is not as useful as it might be), the

facility is acceptable"the discrimination is good and the point

biserial' correlation is quite high. By contrast, another item

from the same pre-test givesthe'ro11owing analysis:-

, RAW, SCORE ' A. L., MATHS.

ITEM No. 24 'fOTAL

OPTIONS A* ,B', C D E 0 IN TEST

77 6 0 2 3 1 89 89

187 .07 .00 ,.02 .03 .01 1.00

SCORE 0-13 11 4" " 0 2 0" 0 17

SCORE 13-19 '''15 0 0 0 2 1 18

SCORE 19-23 17 1 0 0 0 0 18

"SCORE 23-29 18 0 0 0 0 0 18

SCORE 29-40 16 1 0 0 1 0 18

, MEAN, CRITERION, 22.00 15.83 .00 10.0 21·5 13.0 21.19

MEAN 'TRANSFORMED , 13.41 10·31 2·35 7-37 3.07 8.88 3·0

FUNCTIONS BASED ON TRANSFORMED CRITERION SCORE:

ITEM STANDARD DEVIATION .~4

ThrnAN SCORE ' 12.78

ITEM DIFFICULTY INDEX ,,8.58

BISERIAL CORRELATION .42

POINT BISERIAL CORRELATION .27

The facility valuel'. of 0.87 is high, which means that the

" candidates, found this question fairly easy. ' ,This, :in itself, is"

not a condemnation of the item, but the 'discrimination index ,is

only 0; 28 (,16-11)' 'as compared with 0.96 for ITEM No. 28. , ,89~.' , ,

17

.' \<

Clearly the correct option 'A" has, been spread almost evenly

across 'the compiete ability range so that the item is not

distinguishing very well between the most able and the least able

candidates. The pointbiserial coI'r§.lation of 0.27 is much, lower

,than in the previous item (0.62) but it is not so low, as to suggest

complete unreliability Or a significant amount ofguesswork;i , , ,

Option '0 ' ·hasiJ.~toperated at all'and even options Dand 'E~~ve not

'been very effective; 'whichnieansthat the number of alternatives

has been virtually reduced to two.', This is:a very dangerous',' I I '

situation which could mean that in a future :test the use of the' ",

! I

item, in its present form,might unacceptably reduce 'the

reliability and give too mu~h opportunity for 'guesswork. 'If it ,

was considered desirable to have an'i temwi th afacili tyas high

this. (whiiLa'accepting'the lack of discrimination) it would be

necessary to find,three,new more plausible distractors.

as

, .' -

LA common criticism of objective-type papers is that the

provision of a number'ofchoices:leads to a guess-work approach , '

and the results 'ofa test could be completely distorted by

guesswork. Of course, guesswork can play a part.in conventional , ,

test papers, largely undetected and whose ,effec,tis very d~fficult

I

I

to estimate. With multiple choice,questions'an'item which is'

particularly Busceptibll'to random, guessing would be detected 'in a

pre-test, by"an unusuafiy low point biserial correlation coefficient;

To a certain extent the results ofa te'st can be corrected for

random guessing ,but ,the correction formula is based on the

assumption that the guessed answers are perfectly normally

, distributed. Thus, ' if,there «0,

S" opt:Cons are to 'each item" . ,~

the'

probability of 'guessing the correct answer' is 0.2, which means that

for every 4 items which are wrongly answered there should be one

correct solution. obtained by'guesswork. Considerable assumptions

are made here-the 'randomness'is spread in each individual item

and/

I

I

I

18

.. _ -. --.J.-'f." ..,.,/" .... ,,· •• --"j·.,,·,t,!'.I :~'1'" ('''''' -t' ""~'''''''Hr~~ 1"1 '.\ 1', ,. I "-1

and also 13.ctdss, 'the complete test, and ,the guesswork' is in 'no 'way;

directed.' In practice most candidates.indulge in-guesswork by a

combination of limited,knowledgl:land intuition -'it is "inspired"

'. guessing. Further, it would be a perfect test in. which theoptionE

'were equaliyplausible in each item and this is most unlikely.)

(Wei tzman (1970) 'defines an item as' "ideal, if all its

alternatives are equally attractive to every 'person who cannot, '

answer the 'item correctly without guessing." A well constructed

test will certainly aim at this perfection but tl).ere are bound to

be cases where some distractors are unconvincing and can be " ..

ignored thus reducing the number of options to two or, three.) , . . .

However,havingIDade the assumptions,thecorrected ,total score -for

~ test would be !i:; - W where C is the 'raw' score of correct n-l

items;W is the number of items answered wrongly (excluding items

omitted)andn is the number of -options for each item :r.e. 0'- W,

in the case of 5 options per item. This system is used in 'the , "4

, British 'National Mathematical Contest . which is held annually and'

which wes, an -American =1 tiple-choice paper' sponsored' jointly by

the Mathematical Association of America, the 'Society of Actuaries,

Mu Alpha Theta, the,'Na.tional Council of Teachers of mathematiCS

and the Casual ty' Actuarial Society and the . candidates are warned - '

that guesswork is penalised in this way while questions which are

. omitted·· are: ignored. The correction facto,r was also used in the

1973pre-tests of -Advanced level Mathematics papers 'carried out

by the University of London School Examinations Department, but-it

was fotind that the correction had a negligible effect on rank

orderand'th1s was presumably because there were very few 'omissions

In laterpre-tests the corr§ctionfactor was ignored. Another"

fOrIIlllla which is perfectly· correlated with the one above, gives ,a '

o ' corrected total score of C + n where 0 is,the number 'of omissions.

. ,

·····19

". r ... ~' • vr'" .f'·r~

This (not numericallY'equal'to 0 -:"'n-l) gives a positive

encouragement :to ·candidate.sto 'omit items .for which they.shave

inadequate 'k:tlOwledge 'orunders1;anding. The formula 0 ""n~l

can be 'put ina different form by means of ·the substitution

•. { I

W = N-'O 'where N is the total number 'of items, i.e. " "N-O" .!, On-Ni i' ; , 0 -: n-l . = n-l '

A ,corrected:facilityva1uefor an item can be produced of this

form, ~ i. e. 'correCte'd number: of·' corre'ct answers ford.an item = cn - N~ I' 'where 'c'is the" raw',number.of'corr§;ct 'responses, ' i ~" .,

n-l " '. Ni is :the 'number'of~ttemptsatihe 'item.',

. l' , '. 1 (cn;"'N");~f',' . Then l:feW 'facili ty value = -1 .

N. n-l '.

= n (C) -1' ", N'l,': i:""

n-l

= n . (old 'facility value ) '-1' . , •. ' n - 1 ..

e.g. uncorrected facility value 0.7, number of options 5, 'gives"

corrected facility value ,of 0.6) if 'the number'of options'is'only

3,the ;corrected facility value falls to 0.55. :

Olearly low 'facili tyvalues are more affected by this correction, '

and it 'should be noted that c .= 6.2 produces a ~ew facility Nl

.. :

value ,of zero and negative values could possibly ,be obtained 'when I

difficult.items have been intentionally included. There 'is an' ,

indication here that the formula over-corrects when the item is'

very difficult and'itunder;"'cQrrects when the number ofoptions(n)

.is unwittingly reduced'byhaving tmplausible distractors.·

(Another, " and more valid, criticism' of obj ective tests is' that

it is 'not possible ,to test quality of expression,' neatness of

solution, ability to produce a sequence of logical thought, ','

". originality or clarity of. presentation. ' There is little

1 .' Guilford 'J. P. The. determination of 'item difficulty . when '

chance success is a factor. PSYCHOMETRlKA 1936,1, 259-264'

opporturiw

to!

20 .. ....." -

to give credit to the divergent thinker. For these abilities to

be tested it would seem that the traditional kind of test paper is

more appropriate and the ideal is· to attempt to get the best of .

both worlds. by combining the results of both kinds of paper.

It has been suggested that objective testing concentrates too

much on the lower abilities with too much emphasis on factual

recall. In Mathematics some examination Boards (including the

University of London) have reuuced.the reliance on pure memory

in both types of examinations by.providing candidates with

booklets of basic formulae. Furthermore.i t has been shown that

objective papers can be devised to test higher abilities although

a great deal depends· on the ingenuity of the item writer. It is

only too easy to have what appears to be quite an acceptable item

(analysis completely satisfactory) 'lio test a particular ability

but which can be successfully solved by an ability of a lower

standard. An example of this is a straightforward multiple choice

question such as Jxe 2X .dx with fi~e options given. This is

intended to be a test of an integration technique but the

candidates can choose the correct option by· differentiating

the given Options. To avoid this, the stem can bea definite

integral but this is not too satisfactory as a simple numerical

slip·causes complete loss·of credit even though the candidate

may be quite corr~t with the. integration procedure. The style of

objective paper produced by the University of London allows

considerable scope for a variety of levels of ability to be teste~.

The writer's Pilot Test is ibased on the format of the pre-test

papers of the Uni versi ty of London School Examinations Department

with the kind permission of the Council and Dr. R. Wood, the

Senior Research Officer. The Test is divided into 5 sections.

Section I is a set of~ straightforward multiple-choice items

in which it is.possible to test some numerical work, some factual

re call/

I 21

recall and some basic skills v.i th rather less emphasis on the .,- i

r' . testing' of' understaiiding: ! ; S&ction' II is al. set H( 6 'iliU.1tiple

completion items which involve rather more understanding;

Section IH (8 items) tests relationships, Section IV (7 items)

tests understanding by means of assessing data necessity and

Section V(6 items) demands the assessment of data sufficiency.

As progress is made through the Sections there is more and more

emphasis on understanding. The five different styles of question

make it easier to cover a wider variety of topics, e.g. a

particular topic may not naturally produce five reasonably equally

plausible options for inclusio~ in Section I (when writing items

it often happens that. the fifth option is just not available as a

convincing alternative and it is not desirable to resort to "None

of these" more than once or twice), yet the same topic can be'

readily dealt with in one of the other Sections. Similarly, of

course, it is sometimes difficult to produce four pieces of data

to include a well-hidden "odd-man out" in the data necessity items

in Section IV. The variety of approach in the five Sections gives

the test-paper producer, or producers, greater flexibility. One

disadvantage of this scheme is that the understanding of the rubric

can become a test in·itself. Traditional test papers in 'A' level

Mathematics (particularly in mechanics) are often more difficult

to some candidates because· of the complexity of the English and it

might seem here that the objective paper is open to the same

criticism. However,thereis no reason at all why candidates

should not h!l,ve prior knowledge of the rubric and indeed there

seems to be no objection to them having previous practice with

the interpretation of the rubric. There is, of course, a danger

here that there could be too much "training for examinations" which

would.affect the quality of the teaching but if an examination

consisted of.the two types of paper, objective and traditional,

it is unlikely that the limited training would be educationally

objectionable.

22 PURIOSEOF PILOT ~~ST

When the intreductien ef an ebjective examinatien in 'A' level

!~thematics is being censidered there are many questiens which

naturally arise. Seme ef theseare:-

{(i) Are'ebjective questiens as effective as traditienal enes fer

evaluatien efroathematical ability and also. fer predictive

purpeses?

(ii) :Itis aclmewledged that a traditienal examinatien paper tests

a distinctive ability, i.e. to. sustain a mathematical argument,

which is net,tested by an ebjective paper. On what greunds can'

the results <f~'twe such differentexaminatien papers be cembined?

If a cembinatien ef results is accepted, in what ratio.' sheuld they

be cembined?'

(iii) Since ebjective papers must,fer reasensefvalidity

and reliability, have a cemparatively large number ef items, is

there ,tee much ef "a race against time"? De all the tests tend

to. beceme 'speed' tests rather than 'pewer' tests?

(iv) De multiple chOice questiens ever-enceurage attempting to.

selve a'preblem by "lmecking-eut the alternatives" thus "preving"

a prepesitienby preving its cenverse? Indefinite integratien

being cenvertgd to. differentiatien isa case inpeint.

(v) Can items be pre-tested in a manner where theeutceme appears

serieus to. the pupil? i.e. if the examinatienis ef ne persenal

censequence to. the pupil dees the attitude affect the reliability

ef the results fer afar mere serieus and tense situatien?

(vi) If, after a pre-test, items are rejected purely because they

apuear'tee'difficult, is there a danger that the examinatien

standard can beceme a functien ef the quality ef the teaching?

(vii) 'De the difficulties ef censtructing an ebjectivetest paper

including/ '

23

(inci-i1a.f:dg"thEil'f~Eg:thy;'groJp{discussfoi1s; 'the' pretests 'and i • i I i

analyses) offset the advantages? Gan an 'objective paper be

constructed in the same comparatively simple organisational way

as a traditional' paper, i~e •. the personal. cr~ation of a Chief'

Examiner (perhaps. wi th an assistant) aided by a moderator or a

small team of moderators?

The Writer's .PilotTest,with its results from 68 candidates

from 3' different schools, ' with the GCE' A' .' level re suI ts of the'

same candidates, was an attempt to throw.some light on some of

these:q,uestions; The Pilot Test was taken a few weeks before the

GCE examination. Throughout this exercise the limitations on the

conclusions due to the use of a comparatively small sample (69) '.'

must be 'recognised.

Questions (i) and (ii) above are partially answered by the

comparison of the results ,of the candidates in the Pilot Test and

the,GCE examinations. It is assumed, and this of course is

arguable, that the 'traditional papers have already established

some recognition as measures of general mathematical ability and'

predictors of future performance. For both kinds of examination be

predictive accuracy can only~effectively checked by a longitudinal

exercise involving' the Uni versi ty '. (or other further educational) .

and professional,performances. However,. if it can. be shown that

at least rank orders are comparable in the two examinations it is

reasonable to combine the results for use in selection as in the

present system. One certain gain is that the objective paper

. tests ,over a wide field while the traditional paper gives an

'opportunity to test both continuity and depth over a narrower fiel~

The ratio in which' the 'results can be combined is a subjective . . -', '

decision depending on the. importance which is attached ,to each

examination field. In the case of the University of London 'School'

Examinations Boardj'the ratio proposed is 1 to 4, emphaSising the traditional!

, " 24

traditional papers. The full examination is,to consist of 1

objective type paper ofli-hours and 2 traditional type papers

each of' 3 hours duration. ;

Question (iii) is dependent on the structure of the test-paper.' .' :. . .

The greater the emph~sisonthehigher abilities ~he greater the'

likelihood 'of pressure of time.' The greater the 'time pressure

and the greater the difficulty of the questions, the more

temptation there ,is to resorttoguessing,particu.1ar1y when

candidates are not warned of any penalty for wrong answers. ',The

Pilot~Test w~s'c1ear1y affected to'a certain extent by a shortage

of time' and, indeed, one group of'candidates when questioned

afterwards said that their performances would have been markedly

better given an additional t or -i of an hour;,: They found that

they were additionally hard-pressed because ,they had had 'no

previous experience of the rubr1c in Sec-hons'III to!V and' ,

, valuable time was used in 'ensuring its understanding. Clearly' '-'

this situation could be improved by explaining 'the rubric

beforehand and, lPerhaps, -by extending the time to fan 'average :of!3 ' -

minutes-per,question;instead of '2., - i

Question, (iv) is only a ,valid criticism 'in-the'case of ,Section ,1

and the possibility does not arise,in all ,the_ items. However, 'at; f

one of the schools concerned the staff admitted that they had! ' ,

advised their candidates 'to "test" ,each option ,instead of 'attacking

the problem, directly. ' '

The importance 'of question Cv) is difficult ,to :assess. 'Despite

the willing, cooperation which some schools undoubtedly give with

pre-testing 'and the serious intention of many candidates 'to "enter

into the spirit of the thing", the ,fact remains that the correct

atmosphere cannot ,be, reproduced when there-is nothing at stake.

" )

25

. - " .

The, effect idf Ithfs 'is 'impossfbie iid'esilima:'ee!;{~ :~on\.e car1didlites

~illworkmore calmly andmore slowly ~derthe 'false~conditions'

while 'others 'will work in a more '~slap";'h~ppy'~ 'maruie:r:- cheerfully'

giving way 'to intuitive guessing when the needarises~'This is

not to' say that pre-testing is yalueless'- some inconsistencies, , "

some over-difficult 'Hems; some ambiguities, some ineffective ' .',

options etc. are bound to be,brought:tOlight.' In, any 'case the

pre-testfindings will be ,backedu; by ,the 'analyses 'of the results . , -

when the items are used in actual examinations~ .

Question'(vi) raises the issue of whether a 'particular 'concept ,

is'of sufficient importance,to be included in the test even though

it, is a common source of error.' For example, in the Teacher's

Booklet' (for the multiple choice :test) .forthe Unive:r:-sity of, •

London GCE '0' tlevel examination, Mathematics SyllabusC (1970), :

page 16, appears the'fol1owing:-

"Two sauce bottles are identica,l:inlshape, but·notin:size. "I

The larger 'bottle is exactly twice ,as tall as the smaller'one.

If the 'smaller bottle 'holds 15 fluidounces:of sauce, what is'

the capacity (in fluid ounces) of the larger bottle?

A 30,

B 60"

C ,90

D 120

E 150

Analysis of test results.'

, ,Lowest . Fifth , ,

Next Lowest" Fifth' '

Middle 'Fifth :

Next High.est Fifth

Highest Fifth" :,"

Total

, Proportion

P, = 0.15

' , , ,

A: B C D~ E

16 ,14 3, 3 2

30 ' 23 8: 6 1

27 33 10 6 1

32 16 ,14 13 2

24 . 26 10 32 2,

129 112 45 60 8.

-33 .28 ' .11 .15 ~02 ' r = 0.44

0

13

6

10

8

4

41

.10

26,

This question proved to be so difficult (facility value 0.15 )

that probably i,t could not 'be used in, aria~tU.al 'examimttion even'

though 'it !discriminated ,well. ','The rea~on ,f6;r-the difficulty of

the question'is obviousfrolllas-htdy 6f'theincorrect "resp;nses. '

3310 of .the candidates, thought that the original volume was ','

multiplied by 2,:.~ •• ". ~"', '

The item'is concise:and,'is a direct test of ' an important basic

',concept~J The lowfacil:i,ty valu~ does: not, indicate'that 'the 'item

istciodiffi~Ult, but ,that there is widespread misunderstanding '\

which ,could be lduetothe irieffectivenessof the teaching. 'To. ' , ,

rej~ct this item could make the standard of the testt"oo 'closely'

related tothestaridardof'the:t~achirig.Inthe ,Viriter' s Pilot "

Test,item No. 44 : produced a 'very similar situation. ' 'Here'J :the '

item was co~cerned principally with the understa~dil1g' of the" : '

mechanics of direct 'collision and the, use of the coefficient of

'restitution. It had to be appreciated that Newton's 'relationship

involved relative velocities arid that the masse'sof' the particles I

we:neirrelevant.Theoutcome was that the facility value of the I

item was as low as ,0.16 anQ'there was no discrimination at all. I

The most revealing fact was that over 50% of the candidates thought!

that the masses were required to solve the problem, with a,

sUbstantial number of these candidates among the most able. An

indication that the candidates themselves did not think this was

an outstandingly difficult item was given by the fact that it was

attempted by 'all except one, despite it being the last item but

one orithe paper. I consider that the item is not toodiff'icult,

the concept is important, and the evidence is that there is a

weakness in the teaching in Which the writer himself stands

accused with his own,candidatesl

. .. ..

27

. Tests and Mathematical Learning" by Noel·Wilson (Oliver and Boyd, . ' : l

.1974), Chapter 9, page 35:-'- "atyarious pOints in the preceding

chapters, . chapter 9 has been held up both as. a golden carrot' and

as a palliative to ease the. ,tensions that arise when the enormity

of the task of producing a good objective test becomes

'aPI?arent. . ..•...

(The firs~ point I would like to 'stress, and stress again, is'

this: . a teacher, working aloner no matter how brilliant and

experienced, is, most unlikely to produce a really good test. This

is an absolute statement. But it is based on some years of .' . . .

professional work in test construction. . We know,;' from bitter

'experience, that indiv:i.duals just do not see the ambiguities and

faults in their own work. We know that a test produced by a !',>

single person has neither the tightness and clarity within items

nor the variety and;contrast between items, ofa test produced

by a co":operating panel." )

frhe expensive and time-consuming care which, by co-operative

effort,can be put. into the construction of an objective test is . .

not wasted, as the result is a battery of items of proved quality

which can be "banked" to form the nucleus of a store of items.

Wilson is very emphatic reg~rding the weakness of an individual

effort and in the face of this firm. opinion the writer feels very

rash, and almost arrogant, in producing the Pilot Test. It will . '

be seen that there are a number of flaws in the test·;· some of which

will be pOinted out, but no doubt a number will be missed. I was

warned! W. G •. Fraser and, J. N. Gillam in "The Principles of

Objective Testing.in Mathematics" (Heinema~, 1972), page 106, .-

also refer to the need for a team effort, . though rather less.

sW,eepingly:- "It is not easy to write goodobjective items at a

level above Imowledge. Even'knowledge i teme require care and

attention/

--- ---------------------------

28

;" attetltionFt61t:h~I·N'w'bil:srcrruies'th&.rhave·beeii.rgiv~nrrFet.; item

. 'writers will ever aCCluirethe ability to produce items immediately

to order: ideas have to be gathered' from a variety of sources as

they are met, and kept for subseCluent use. This difficulty in

producing origina.l ideas for high level items is not, however,

a difficulty that does not already exist in traditional forms of

examination. The new feature is that where before a setter might

only have to produce one or two such Cluestions, now the test

constructO-r:, will reCluire ten or twenty. . This. need emphasises.

the desirability of co-operative work among a number of teachers

in producing a test. It.mea.ns also that good, high level items,

whep they have been found, deserve. more use than one examination." ~

.29

CONSTRUCTION OF PILOT TEST

I '. As . the results of. the Pilot T.est were' to . be compared with the

NUJMB.GCE 'A' level Mathematics papers, Syllabus A, the subject

topics chosen for the test were matched as far as possible with

the pattern.of past. 'A'levelpapers.The 'A' level examination

ha.s two 3 hour papers, one of pUre rifathematics and one ofrifechanicf

. The pure llaperuinlslly has an emphasis on algebra and trigonometry,

,followed closely by calculus and with rather less emphasis on , '

'co-ordinate geometry. The mechanics paper is biassed towards

dynamics ina rati'owhichis v~ry,'n~arly 4 to 1. Of course, the

matching could not be exact as there have been variations from , .'

year to year and also, whereas the Pilot Test gave the candidates

the target of attempting all 45 questions, in each 'A' level

paper there was a choice of the equivalent of 7 questions out of

9~ When the 1975 papers appeared, the comparison of subj ect '

" t onics was as follows:- .

1975 'A' PAPERS PILOT TEST .' ..

<- .. --c-

PURE .. CO-ORD. GEml!. . 2 4

CALCULUS 2 + 2 halves = 3 . 8

ALGEBRA & TRIG. 8 halves = 4 12 I .

MECHANICS STATICS I + 2 halves = 2 4

DYNAIUCS 6 + 2 halves = 7 17

.. 18 45

No attempt was made to categorise items with fine distinctions of

abilities within a wide hierarchical range of mathematical

objectives, but a less ambitious target of three measures of

',ability were used, i.e. recall (No. I), basic skills (No. 2) and

understanding (No~ 3). The distribution of abilities tested in

each subject topic was:-

---------------------~----I

30' , . .... .

, . ABILITIES. ,. >! " 1 .. ' 2 3 .... TOTAL ,

..

CO-ORD. GEOM •• 1 2 1 4 .

. .. , '

CALCULUS 1. 4 3 8 .

.

ALGEBRA & TRIG.' , 2 4 6 . 12 ..

STATICS' ' .. ' . i '0 3' 4

DYNAMICS' . 1 1 15 17 .. .

.' .'

6 28 11 45

At this level of 'work the emphasis on understanding is

intentional and it is assumed that an item requ~ring.

'understanding' also requires the abilities of recall and

mani~lative skill. The mechanics items are almost entirely at

the level 'of understanding. These;gradinesof abilities 'are, of

course, . subjective value judgements and even more so are the

assessments of difficulty_ The aim was to produce a test

consisting of about 2/3 'of 'average' difficulty, the remaining 1/3

being divided into 'hard' and 'easy' in the,ratio of about I to 1.

Vlhen the test was completed a' check of the items produced the . . '

following:- . Easy 3, .Average 29, . Hard 13

These estimat$s could not be expected to be precise and it is

interesting to compare them with the reactions.of the candidates'

as shown by the facility values. ;rhe importance of pre-testing

is thus borne out!

31

" ~ .'.

'P:Liot :Test .. '1975 ' , (: : It hours' , " ,

-There are 45 '~uestions in this paper 'and you should try:to '

answer all 'of ',them. For each ~uestion there are five suggested' answers. ' 'When you have selected whEl,t you believe to 'be the correc1 answer, mark your choice~ontheanswer -sheet by drawing 'a firm' line through the appropriate capital letter. 'Mark only one " answer'for each ~uestion. ' Do not guess. '

Do, any 'necessary 'calculations, and rough work on the, paper " provided; 'do not scribble on :theanswer sheet or,the ~uestion paper.

Your result will consist~uite simply 'of the 'total number of correct answers you give.

In all ~uestionsthe value' of gcan, be' , , ' assumed to be given as ' 10 m/s 2. ,,' ' '

SECTION 'I' (18'~uestions) " ,

Directions' , 'Each of these~uestionsis followed by 'five' suggested answers. 'Select the correct answer, in, each case and mark the answer sheet accordingly.

1. The normal'to the curve y2 = 2x- a 2 at the . point (a2 ,a) is,'

A. y- ax = a _a3

'B. ya -x = a 2 - a

C. y + ax = a + a 3

, D. y'+ xy, = a + a 2y

E. ya + x = a 2 a·

2. S'tan 3x. dx:,:, "

A. tIn sin 3x +C

B. 3'sec23x + C

C. rln cot 3x + C

D. tIn sec 3x + C

E. ..Lsec 3 3x.tan 3x + C

3. A ship X sights another ship Y'which is 4 km due'West.of it "and which is sailing at 12 km/h in:a direction N. 30oE.The

maximum speed of X is 12 J3 km/h. The shortest time in'which X can intercept Y is ,,' ,

A.J~ mins. c. 5mins.,'

, 20. 'I

B. "2' mins. D. 10 mins."

E. 20 mins;

I

I

I

32

4. " . The' M1i:1H'esY"iff"tl¥aHce~ll'etweeii""!iily;Tpoirii"o:ii'tlH;' circle .... , '. '.. . x + Y . .;. 2x + 4y + 1='0' '. ..... .'

. and the line 3; _ 4y + 19 = 0 :is' .

A. 4 .~, f

B. 0 ,

C. 5 "

D. 4 ;

5"

E. 24 , 5

5. )In;.dx'=

. , '. 1 '. ~3 + C'; . A.· x 1n - --3' . x ' .

·.B. 1 1n 1. _ 1,+ C X X X

C. ,1n(ln x)+ C'

D. x + x 1n x + C

E. x'- x 1n x + C

6. 1n' (1 + x +x2 )"is expanded in a series of increasing powers' of x=forlxl(l. The third.term of the series is '

7· d

A. ~. x 3

2 3 B. -3' x

C. _ x3

D. i x3

E. - i x3

2' 1 -' ) dx

(x 'ex . 1

A.

B.

C.

D.

E.

2e~

. 2 1 __ eX x·

8. The sum ofthe'c~besofthe roots of the equation 2x+ 3x .;. 1 = 0 is

. 45 A. -g

. , . 45· 'B',--gC. "7 36

33

9. The square 'root of (J '- 4i) 'is 1 t: '

A. 2 - i

B. 2 + i,

C. [3- 2i,

D. 1:+ 2i

E; 1,':' 2i

10. 'Two smooth bodies of masses p and ,10 kg are moving on a smooth'horizon:!:tlsurface,in, opposite directions with speeds",:: of 4 and,12'ms ,respectively. They collide directly ,and remain in contact after ,the 'collision. The impulse 'between them is

A. 12N s 'B. 78.Ns 'C. 60 N s E. 36N s

D., 30N s,

11. A shell is'J2fojectedfrom a pOint on horizontal ground at a speed of 50 ms' .,' An obstacle ofheight'35'm'at a distance'

" of 150m from the point of proj ection is to be just cleared whEm the range on the horizontal through the point of projection ,is least. ,Neglecting airresistancei the tangent of the angle

'of projection 'is then

A ./14 B: l' • ""b • C. 8 D. '3 E. None of these. '

'12. The fifth root of 28.8 to four decimal places is,

A.l~9580 'B., 1.9581 ' C.', 1.9582 ,D. 1.9583 E. 1.9584

13· A : particle ,oscillates with simple harmonic motion in a straight line and has an amplitude of 4 m and a period of 2 s. Its speed when it is 3 ID from the centre ,is

A. -1 511" ms,

B. 17 ms-l

C. "-If ;"1 ' :a-7ms'

D. trrff ms-l

E. 11"17 ms-l

14.

Two smooth spheres of equal,radius56:nuit and of weights W,2W as 'shown, 'are:in e'luilibrium inside a smooth cylinder of radius 80 mm (open'atboth ends), standing ona horizontal surface. Vlhat is th weight 'of the cylinder if it is about to overturn?

A. 3W B. 3W """7 C. ,2W D. 4\'1

~ E. 8w

j

I 'i 15. r IY'; 917n777'Tt":1

34. ,l

x The region.shaded in. the diagram is rotated completely about.the y-axis. The volume of' revolution thus obtained 'is

A. 1Tln 8

B. 41T

C. lTln 9

D. 521T -3-'

E; 80fT • , '"lIT

'. . .' 2 . 16. Tang2nts are drawn t~ the curve x = 4ay at the pOints (2atl , 'atl ) and (2at2, at2). 'The ·t;angents meet at the point

17·

,A. t atl t 2, a(tl + t 2)j B. fa(tl ~. t 2),ail t 2}

c·l-a(tl +t2),atl t 2J. D •. {-a(tl + t 2), a(ti +tl t 2+ t~)}

. . ,

E. ta(t1 +t2); -atit~}

NO SKIDDING

.v

A vehicle moves horizontally with constant ~peed v ms-I •. The magnitude of the velocity' of the pOint P on the edge of a wheel, level with the axle, is

A. vl2

. '. B. v 12

, c; v·

D.' 2v

E. v 2'

•

35

18." A rough uniform hoop has a small particle of eClua1 mass attached to its rim. It is hung in,a vertical plane on a rough

, fixed horizontal peg. ,If ~he angle of friction at the point of contact with the peg is 20 , which one of the following positions is impossible for ~Clui1ibrium?

m

c.

m

m

".',-,

D. "

I \ I

45<\1 ---~-- -

I I

36 SECTION II (Six questions)

Nos. 19-24 Directions For each. of the following questions,' ONE or II'IORE of the responses given are correct •.. Decide which of, the responses is (are) correct. Then choose. .

A •. if 1, 2 and 3 are'correct Directions SUJDJDarized B. if only 1 and 2 are· correct A. 'B. C. D. E. C. if only 2 and 3 are correct D. if only 1 is correct 1,2;3 1,2 2,3 1 3 E. if only 3 is correct only only only only

19. A force is such that.it would move a mass of 1 kg through 1 m from rest in.l s. This force is applied to the 1 kg mass while it moves a distance of 100 m,

(1) The force is2'.N.

(2)" The work done. in the 100 m is 200 J.

(3) The maximt1mpower exerted is l0.{2-watts. '

20.. A real function f(x) i~ defined for all real values of x' and satisfies the relationship f(x+y) '=f(x).f(a-y) + f(y).f(a-x) for all real x and y, 'where a,is a fixed positive constant. Also f(O) = O. '

(l)f(a) =1

(2) f(2a) = 0

(3) {r(x)p + ff(a-x)}2 = 1

21. (1) d (eln xl 2x dx =

(2) d 'eln (X2

) = 2x . dx

(3) d e(ln x)2 = 2x dx

22. For all real x> 0

(1) cos x( x (2) tan x< x (3) 'In(l + x)( x

23· For the curve whose equation is = x

4 - x2

(1) there are only two asymptotes, x = 2 andx=-2

(2) the curve is symmetrical about the line y = 0

(3) there is no part of the curve in the 'interval- "'-2 (x < O.

. , . .

24. A particle, starting from rest, slides down a smooth inclined plane for,. tl s .wheni t then. moves on to a smooth horizontal surface without bouncing. The sketch graphs representing the acceleration (a),. the speed (v), and the distance (s), plotted against the time tare,

(1) a

o

(2) v

(3) s

. - '.'

L--.-:........,!------t 1

, "

1

.

t

t

37

SECTION III (Eight~uestions)

. NOSI t 25-32 ",' ; ....

Directions Each of the following questi~ns consists,of two statements (in some cases following very brief preliminary information). You are required to determine the relationship between these statements and to choose

A. if 1 always' implies 2 but 2 does not imply 1 , B. if 2 always implies 1 but 1 does not imply 2 C. if 1 always imples 2 and 2. always implies 1 D. " if 1 always denies 2 and 2 always denies 1 E. if none of the above relationships holds

38

25. A pa:r.ticle m£yes from X toY on ~ straightlinr' At Xthe velocity is 12 ms and at Y the velocity, is 24 ms •

(1) The velocity at the !!8:ddle of the time interval from X to Y is 18 ms •

(2) The acceleration is constant.

26. x(jJ$ a real function of the time. At t ,= a

(1) 'dx, = 0 (2) d2x' = 0 . dt dt2

27. Given a system of three coplanarforces.

(1) The system reduces to a non-zero couple., '

(2) The forces can be represented in magnitude . and direction by the sides of a closed triangle taken in order.'

28. (1) The pOints (Xl' Yl)' (x2 ' Y2) and (x3', Y3) are

co]Jinear.

= =

29. A body moves in a plane containing a fixed point O.

(1) It remains at a constant distance from the fixed pGint O.

(2),' It has an acceleration towards the fixed point o. 30. Two smooth bodies are in direct collision. "

(1)' They, are perfectly elastic.,

(2) The total kinetic energy is unchanged by the collision.

, 31. . a and b are any two real numbers.

32.

(1) a Ib (2) The.arithmeticmean of a and.b is greater.'

than their geometric mean •.

f(x)

(1)

(2) .

is continuous for a ~ x ~b.

fb' .

. 'f(X). dX)O a , .

" "

39

40 SECTION!iV '(seven questions)

Nos. '33-"39i' ii! '.' .' . . - .

Directioris'Each·of.the fOllOWing'qUestidns 'consists of a problem followed by four pieces of inforJll8.tion. Do not actually solve the problem but decide . whether the problem. could be solved if. any of the pieces ofinfor.'llation were oIili tted; and' choose'

•

A. if 1 could be oIili tted' . B. if 2 could .be omitted'

·C. if 3 could be oIilitted , D. if 4 could be omitted E. if none of the pieces of information could be omitted •..

33. Find the 'greatest speed at which a 'car :can !take a 'corner of a road without skidding 'if the corner 'is in the form of a circular arc. _The car maybe considered as-a-particle.·

. . . . : .

(1). the coefficierit'offriction between 'the tyres and' the road- is O.} . ....

(2 ~ the' mass of the· car' is l' tonne ,/, (43 the radius of the circular arc is 50 m ( the road is horizontal. . . !-

.' dx ,,/. 34. By 'Newton's Law of cOOli~g;dt ~,- kx where kis a constant and x degrees is.the excess of:the temperature:of a cooling body over that of its _surroundings. Find the time taken for the body' to cool to a temperature lt times ,that of its surroundings.

I" ., ,(, ,

(1)

(2)

~l~

whent = 0 the-temperature of the body is 3 times that of .. the surroundings . t =. 20 when it has cooled 'to twi ce the surrounding' . temperature , the ,surrounding temperature is 20 degrees the surrounding'temperature remains constant.

35. Aparticle is hangin~ in equilibrium at the end of a light spring whose upper end is fixed. The particle is displaced a. small amount downwardS and then released. 'Find the period of' the resulting oscillation.

. . .... / '. .

(11 the .initial displacement isam . (2 the IlJ1l.ss of the particle is m gm -.. (3 the modulus of, the spring is>, '.

(4the;unstretchedlength of the spring is y m.

'36. The·angles of elevation of amountain.top are measured from two points on a straight. road. Find the direction of the road •.

. /"\';'; " _.,'. " , ", . . (r) The first point is due South of the mountain and

I;· the ,angle of elevation is 300 •. '

(2) Thesecond'point is due Eastoof the mountain and . J - the angle of -elevation is 40. ' , .

,'(3). The ,road is horizontal. . . . / _' (4) The height of the 'mountain is 3,>,000 ft ...

~7. If(x) = :px3 . + 'qX2 + rx: +s. 'Find Pi q,.rands.,

(1) V/hen f(x) is divided 'by . (x-l) the ; remainder is 6. (2) f',(2) =0; (3) (l+i) is a root of f(x) = o. (4)p, q, r'and s are real.

41

38. '.A.f::hn~nhna~s': fs~r~Jsiie!{d;(f'by"a:"strini"from the !rddfdf 'a ' . darriage'of a train which is travelling at 'its maximum speed. The mass ,is 'caused:to 'oscillate as a simple 'pendulum •. Find the period of the 'small 'oscillaticins. ' , • .

~l) The length of ,the string is It m; 2). 'The train is 'moving 'on a straight 'horizontal

'3) The speed of the train is 100 lanperhour. (4)' The string'is inelastic.

39. A'bodYViwhich' can be considered to consist of a set of: fixed 'particles, has:twoof.its constituent particles interchanged in position. Find the d:i,.stance'movedby the centre, of mass of • '

. the whole.': ' t ' ,

(1) The'mass·of the whole is M

(2) The particles 'interchanged 'are of masses IIlr 'and :m2 ,1

(3) The distance between mi and m2 is d

'(4) The line joining In, and m.? passes through the : original . centre of 'mass of'thewhoIe.'· ,

\

SECTION rv icsi£ '~uestions)'.· . 42

Nos. 40..:.45 1'1 t I

-Directions'Each of the following questions consists of a problem and two statements, 1 and 2, 'in which .certain data are given.' . You are 'not 'asked. to solve' the 'problem: you:have to decide : whether the 'data given in the . statements are sufficient'·, . for solving the problem •. 'Using '.the data given in the ; statements; choose' .

A. if'EAcH statenient·(i.e.statement 1 ALONE and statement 2 ALONE) is.sufficient by itself to solve the problem~ ;

B •. if statement 1 ALO~"E 'is : sufficient but statement'2 alone is not'sufficient to solve the problem.'

C~ if statement 2 ALONE ,is 'sufficient but statement 1 alone is not 'sufficient to solve 'the problem.'

D. if BOTH statements Land 2 TOGETHER are·'sufficient ,to solve . the problem, but 'NEITHER statementALONE·is'sufficient. "

E. 'if statements '1 and 2.TOGETHER'are NOT sufficientto'solve' the problem, and 'addi tional. data specific to the problem are' needed~ . , ", .

40. A :boat ot' mass M is at rest in water which is also stationary. A man 'of mass m walks ,in the .boat from ·the bow'

, to the' stern· (p. to . Q). Neglecting the resistance of the water, what distance 'will the boat move?

(1) The distancePQ is d· (2)'· The walking speed of the man is v.

41. A particle is moving in a straight line withS.H.M. of: period'41Tabout a centre 0, and 'passes through a point A which is 6 m from O. Find the time which elapses.before'it is next passing throughA. '

(1)

(2)

, The velocity when it first . away from 0 •.

passes through A is 4 ms-l

The amplitude is 10 m.

42. Proyethatthe:sumof the series'

(2m +.1) + (2m.+ 3)':+ (2m:+5) +

is divisible by 12.

(1) m is a +ve'integer. (2) . m is an even integer.

• • • • • • • • + (4m ~ 1)

43. Two masses are attached to ,the ends of a light string which passes over a smooth light· pulley which is itself suspended by a light string from the roof ofa lift. The strings are inelastic. Find the tension in the; string .' supporting the pulley ~ .

(1) the masses are ml and m2 •

(2) The acceleration of the lift is a.

\

44. Two particles, moving horizontally, collide head-on at a height'h m above horizontal ground. ,The coefficient of restitution is 1;. Find how far they are apart when they reach the ground; ... ,

45.

(1) The masses.' of the particles are ~ and m2; ,

(2) Their velocities at the moment of collision are ul 'andu

2respectively.

~s ;~)~ 7(8.; b, .c,·d all real and ~df 0)

·(1) a)b

(2) c)d'

" ..

Most.ofthe items in this section are concerned with the'

lowerlevels of ability (Nos. :1 and 2) and. they are chosen because

they naturally produce.errors which are errors of memory or are

manipulative errors. Furthermore, when planning the items it

was found. that there were 5 plausible alternatives. In some

. instances items had to be abandoned completely because

reasonable.distractors could not be found insufficient number.

In other cases the same idea for the item could be used but in , . . ..

. another form suitable for presentation . in one of the Sections'

IIto V. In item No. 11 the use·.of "None of these" for the fifth

option was an admission of defeat but the temptation to resort to

. this procedure more than once was resisted. Item No. 1 was

intended to be a routine operation and . the distractors were," ~o

suggested by the usual errors which are encountered in the class­

room. . The correct gradiEmt of the normal is -a and this leads

to the correct option a.Option A is obtained by a simple sign

error taking the gradient as +a. Option B arises from a gradient 1 of a' . i. e. using the gradient of the tangent. Option D is caused

by taking the general gradient of ;...y which is a common source of

error despite the obvious nature of the result, i.e. a curve

instead of the re~uired straight line. It is significant that

18 candidates of· the 69 fell into this trap. Option E comes from

. a gradient of 1 - -. a Since only 2 candidates chose this option it

was not very effective. Perhaps a better option

obtained by taking the gradient of -2a (i.e. ~~

could have been

~V _1 = da dK - 2a. 'Qa

etc.), but this idea was discarded as it was thought that the

figure 2 would stand out far too clearly among the options.

The options in item No. 2 were again obtained from well known

errors. OptionrA," which arises from confusion between the tangent.

andl

~ "

45

and cotangent when expressed in terms of,the sine and cosine,

was not felt to be particUlarly plausible, but 9 candidates chose

it •. ' Option B comes. from using differentiation instead of

integration and this attracted 7 candidates. Option C is

produced (1 . JX. dx=

when the candidate uses the false analogy:-

In x, then 5 tan x. dx = In( .. l ) = In tan a

cot x.

Option E was. not expected to be effective 'enough as it depended

on an enol! similar to that in optionB but with even greater

confusion. Surprisingly it. attracted 12 candidates.

Item No. 3 was considered to be hard although it concerns a

basic concept. The correct solution (option D) only required a

simple velocity triangle and very easy trigonometry. Options A

and E were based on complete misunderstanding of relative velocity

i.e. using speeds of 12[3 and 12 respectively. The wrong

velocity triangle, i.e. AVE (12)

BVA

v . B E

(12i3)

V 20 . giving B A = 12{2, produces the optionB of W mms.

Option C arises from a .tri vial. ari thmetic error. The errors used

here have been found in previous classroom work with similar

problems. It was surprising to find that all the options

attracted a fair proportion of attention and to this extent the

item was a good one although it.was disconcerting that as many

candidates showed complete lack of understanding by choosing

~ption A as those who chose .the correct option D. ,

Item No. 4 required an understanding that the difference betwe~

the radius and the perpendicular from the. centre was being asked

for. An assessment of the problem was therefore needed together

with/

46

/

wi th the recall and skill' for obtaining the. centre' and radius' of

a circle and for writing doWn the length of a perpendicular from

a point to a line. The highestabili ty thought to be involved was

No. 2 (basic skill) but it was expected to be found to be hard

becaUse of· unfamiliarity. The 'distractors (options B to E) were'

. based on errors in the centre of the.circ1e, e.g. (-1,-2) or'.

(1,2) instead of the correct (1''72) and an error in the radius,

i. e. 1 instead of 2. . Option B attracted candidates who assumed

that the given line was a tangent to the circle •. · Again, all the .

options attracted a satisfactory number of candidates.

Items 5 and 7 were fully' e~pected to' be within the range of

ability of most of the candidates and the forecast of "average'"

seemed to bean over-estimate of their difficulty. However,

item 5 in particular proved to be unexpectedly difficult. The

source of the trouble appears to have been the unfamiliarity of

In 1. The latter was chosen instead of 1n x itself to test the x

basic operation of integration by parts, as it was felt that

~'ln x.dx was so 'well known as to be. a matter of simple recall.

Option A was chosen by the candidates .who knew. the method of

integration by parts but could not differentiate In! correctly. . '. . .. x .

Option B tempted the candidates who blindly quoted

51nz.dz =zln z - z and replacedz by ~; Option C was obtained

by wrongly equating ~{ln (In x)J with l~ x and then making a

further error by putting this equal to In ~. E was the correct

option and D was obtained by a sign error. All the options were

operative and it was astonishing that option C was chosen by as

many as 15 candidates,. more than the number (12) who chose the

correct option. Item·7·was rather more successful except that

option B was not operative at all and option A was not very

plausible. Differentiat:Lng a product seemed to .be well understood

but there were many candidates finding difficulty with

I' 'differentiating

l' x e •

47

r l \

Option D, which' was' chosen' by' 28' candidates,

illustrated this very clearly. The usefulness of this item could

have been greatly increased, by choosing more subtle options in

place of A and B.

Throughout Section I, ,the options were chosen to represent the

results of common errors, or were suggested by difficulties which

, have appeared in teaching similar operations in other problems:

It, was a matter of making use of classroom experience. ,In some

instances, notably with "average" or "easy" items, this was

"successful, e.g. in item No. 13 where the 5 options were all

utilised and a fair proportion of the most able candidates chose

the correct option E., However, one glaring case of an unsuitable

option in an otherwise acceptable item was No. 18 where option A , ,

was so clearly urireasonable as to be rejected by all the'

candidates. This was a case where plausibility was being

strained to find a t"ifth option to fit the item into Section I.

,

I I

48

A detailed analysis of each item is given'below. The

'categories "Lowest Fifth"etc~ refer to the total test scores

of the candidates. The choices of options are indicated with

the starred option giving the correct answer. The colu= headed

o gives the number of candidates who omitted the item. PI is the

facility value which is simply the proportion of all the"

candidates ~v:ho' successfully solved the problem~' The ,,'

discrimination index, d,isthe fraction"

NO"of successful 'candidates' ,iI). .1).~ghe st fifth,

No. of' successful' candidates' , , . _ ,in ,lowest : fifth • " , , '

No. in each group of 'fifths' .

In this.case the divisor was taken as approxirnately13 •

' .. . .

1. A B C"'" D E . O. LOWEST 'FIFTH' , ' , 0 1. ,5 6 0 1 , .

NEXT LOWEST-FIFTH' , 0 1 7 6 0 NIIDDLE ! FIFTH " '. ' ...• 1 2 8 2 0 1 NEXT HIGHEST FIFTH 0 0 '9 3 2 HIGHEST FIFTH : '. " 1 2. 9 1 0 TOTALS' : . ' ' 2 6 38 18 2 2

.. p,,=0.56 d = 0.29 ESTIMATED 'DIFFICULTY -' AVERAGE :-f ABILIrY' 2

--c

2 --c

A B ,C D* :E 0 • . . ' .

LOWEST'FIFTH' . ' , 1 3 .2 3 3 .1 NEXT LOWEST-FIFTH 4 1 0 4 4 1 MIDDLE FIFTH 0 1 1 6 .4 2 NEXT HIGHEST FIFTH. 4 1 0 8 .1

I .

HIGHEST, FIFTH " ' 0 1. 0 11 0 1 TOTALS· . 9 7 3 32 12 5 ( -' .

P, = 0.47 d = 0·57 ESTIMATED, DIFJ!'ICULTY -' AVERAGE]' , , . ABILITY 2

49

. '3· k B . di ' .¥ E 0 iJ. D

LOWEST 'FIFTH." . , 2. 2 .1 1 2 .5

NEXT LOWEST' FIFTH' . 7 1 3 2 1

MIDDLE :FIFTH : ; ': 3 1 3 3 1 3 1 3 ;". .

NEXT HIGHEST FIFTH' 5 '1 5 0

HIGHEST' FIFTH' . 1 2 .2 5 3 '. -'-

TOTALS 16 .. 11 10 16 7 8 p, = 0.24 d = 0.29 : : .'

ESTIMATED 'DIFFICULTY -, HARD:, ' I ABILITY· , ' 3 .. , , .

. , ' ' . .

.' A.>t. 'B ·C D E 0 . LOWEST 'FIFTH' )', " 1

.

1 1 3 2 5,

NEXT LOWEST' FIFTH: 4 1 2 3 3 1

. MIDDLE' FIFTH' .. 4 1 1 3 1 4

NEXT HIGHEST FIFTH l' ' 2 3 2 1 5 .

HIGHEST FIFTH ' . . , , 5 2 2 1 1 2 TOTALS' " .. 15 7 9 12·.' 8 17

p, ,= 0.22 d = 0.29 .

..

ESTIMATED DIFFICULTY-HARD I ABILITY . 2 , .

. ' ..

. . . . ' . lA B· C D E~ 0

LOWEST FIFTH' , •

1 3 3 2 2 2

NEXT LOWEST 'FIFTH' 4 2 6 0 1 1

lIlIDDLE'FIFTH , 1 0 4 2 4 3 NEXT HIGHEST 'FIFTH' 5 2 2 1 4 HIGHEST FIFTH .6 1, 0 5 1

TOTALS .. 17 8 15 10 12 6 .. , ,p, =, O~-:j.2 ' , 'd =-o~07

ESTHIJATEDDIFFICULTY-AVERAGE I' . 'ABILITY' . ;2 .

6. A '. >l

B . C D E 0

LOWEST' FIFTH' , . 3 1 2 3 1 3 NEXT LOWEST 'FIFTH 7 2 0 . . 2 2 1 MIDDLE FIFTH .. 2 1 3 '3 0 5 NEXT HIGHEST FIFTH· 11 '6 2 2 1 2

HIGHEST FIFTH . " ; " 3 6 2' 1 0 1

TOTALS' . -, " " 16 16 9 11 4 12

P . = 0.24 ; , d == 0.)6

ESTIMATED DIFFICULTY -AVERAGE . ABILITY "2

50 .

A B Clf D J!: 0

LOWEST 'FIFTH' , I , 0 '. 0 1 .. 7 4 I

NEXT LOWEST 'FIFTH' , ' l' 0 4 4 5 MIDDLE' FIFTH' , " ! 1 0 4 7 1 1

NEXT HIGHEST 'FIFTH , , 0' 0 .6 7 1 I

HIGHEST FIFTH'.' , 0 0 10 3 0 . \ , ..

.

TOTALS' , ' ' 2 0 25 28 . 11 :2

•• p, - 0 •. 37 d - 0.64

, .

ESTITh'rATEDDIFFICULTY-AVERAGEI' ABILITY' 2' . .

"

8. , . A " B* C D E 0

LOWEST' FIFTH ',' , , , 0 1 2 4 1 5 NEXT LOWEST ,FIFTH , ,. 0 8 ,0 2 3 1

I ,

MIDDLE :FIFTH ' '. ' , " 1 8 ' 0 4 0 1 ,

NEXT HIGHEST FIFTH' . 1; 8 ,0 2 1 2 ••

HIGHEST FIFTH' , " , 1 10, 0 2 " 0

TOTALS' , '. . 3 35 2 14 5 9 . p,- D~5:j. " ,'d - 0.64

ESTIMATED.'DIFFICULTY :- AVERAGE I ABILITY ! 2

9. , , ' . A* B ,C D E 0 •

LOWEST 'FIFTH , ' , 5 0 5 0 1 2

NEXT LOWEST FIFTH , '9 1 2 0 2

MIDDLE, FIFTH' \,) , 9 1 3 1 0

NEXT HIGHEST FhTH' ... 0', 2 0 ,0 1 1 . .

HIGHEST' FIFTH " ' ! .LO 1 2 0 0

.. TOTALS' . _.', , . 43 5 12 . 1 4 3

ill - 0.63 d __ 0.)6 , . .

'ESTIMATED 'DIFFICULTY-'AVERAGE I' 'ABILITY' . , '2 e· ' .. .

10. ,

. A B C:>( D E .0

LowEST"FIFTH· " , , 1 1 4 2' 2 '3

NEXT LOWEST FIFTH' , '. 0 7 4 2 1

MIDDLE 'FIFTH . ' , . 2 3 4 2 1 ,'2

NEXT HIGHEST FIFTH ,6 ! 1 3 2 1 1 HIGHEST 'FIFTH . , " . 0 .1 11 0 1 TO'TALS', . . 9 13 26 8 6 . 6

, . . Pl = 0·38 ' d = 0.50

ESTIMATED 'DIFFICULTY - AVERAGE I' ABILITY :3 . "

51 .'

11. A B, , ' .. 6 .:r5¥.' ~. ·0 , ., 'LOlVEST'fF!FTH ' . , . 0 0 -1 2 8 2

NEXT LOVffiST'FIFTH . , 2 0 5 1 6 '.

,

MIDDLE FIFTH' , ' , '; 2, 0 0 2 7 3

NEXT HIGHEST 'FIFTH , 1~ 0 5 4 4

HIGHEST FIFTH ;; 1 0 3 '7 1 1

TOTALS' , . 6; , 0 '. 14 16 26 ,6

' .. ,p, - 0.24' , , d -:- 0036' , .

ESTIMATED'DIFFICULTY - HARD.' , TABILITY' '.' 3 .. , , ,

.

.

12. A B. C D* E 0 ,

LOVffiST· FIFTH' ': ' , 7 0 1 ' 1 1, 3 !

NEXT LmVEST FIFTH .' , 4 3 3· ',1 2 1

MIDDLE 'FIRTH ' . . ~ , . 7 1 1 2 2 1

NEXT HIGHEST FIFTH 2 3 4 1 1 3

HIGHEST -FIFTH' , 4 1 1 1 4 2

TOTALS ~4 8 10 6 10 10

p,'= 0.09 ,d = 0

ESTITlIATED-DIFFICULTY - HARD : I ABILITY . . 2 --c , ' .

.

13· , . A B C D E'!e 0

LOWEST -FIFTH' • 0 1 1 2 . 5 4

NEXT LOWEST FIFTH' , 4 1 1 '2 6

MIDDLE· FIFTH' , , ; 2 0 I, 2' 8 1

NEXT HIGHEST FIFTH . 0 1 0 2 11

HIGHEST FIFTH ' , 0 0 1 0 12

TOTALS' 6 3 4 8 42 5 p,-.0.62·"d - 0.50 ,

ESTIMATED DIFFICULTY- :AVERAGEI· 'ABILITY' 1 .

14. A Bt< C D E 0

LOWEST' FIFTH' 2 2 1 2 2 4

NEXT'LOWEST FIFTH, '3· 0 F 2 5 3 1

MIDDLE 'FIFTH 1 3 0 3 1 6

NEXT HIGHEST 'FIFTH, '0: . 7 0 '. 2 1 4 , HIGHEST]'IFrH ", • ' 0 1 1 '6 2 3

TOTALS· .' 6' 13 4, 18 9 18

PI = 0.19 'd ?, -0.07

ESTIMATED DIFFICULTY -HARD ABILITY 3 , .

52 ,

15. A ' B C~ D E 6 ,

LOWEST 'FIFTH' j: . , . 0 0 4 4 1 4 NEXT LOWEST! FIFTH: . ;1 3 9 0, 1

MIDDLE' FIFTH .,' O. 1 10 1 .1 1

NEx:T HIGHEST 'FIFTH , 0: 0 11 0 2 1

HIGHEST FIFTH , i o ,,' 1 11 0 1 ,

TOTALS" , 1 5 45 5 6 6.

p, = 0.66 d = 0.50

ESTIIVIATED'DIFFICULTY'- AVERAGE ! ' ABILITY 1 -:-. .

' , ,

16. , . , , " A . Bll: C D E 0

LOWEST'FIFTH 3 2 3 1 0 4

NEXT·LOWEST FIFTH, ·1 5 2 3 2 1

MIDDLE, FIFTH' ..

i. 0 7 0 0 3 4,

NEXT HIGHEST' FIFTH 3 7 0 1 1 ''2

HIGHEST'FIFTH O' 6 1 1 2 3 TOTALS' ... ; . 7 27 6 6 .' 8 14

p,'= 0.40 d = 0.29 •

ESTIMATED DIFFICULTY'-'AVERAGE I 'ABILITY' 1

17· A>f: B C D E 0

LOWEST FIFTH ... .. , 0 3 7 0 0 3

NEXT LOWEST FIFTH 2 2 6 2 2

. MIDDLE FIFTH ' . i. 11 0 7 2 1 3

NEXT HIGHEST FIFTH . 4 . . 2 5 0 2 1

HIGHEST' FIFTH , . 4 2 5 0 "1 1 . . TOTALS " .. lL 9 30 4 6 8

, . p, - 0.16 ' d - 0.29 ,

ESTHiATED DIFFICULTY - AVERAGE I ABILITY 3

18. A B C>f< D E 0

LOWEST FIFTH 0 2 ' 4 3 1 3

,NEXT LOWEST FIFTH 0 3 7 3 1

MIDDLE FIFTH' 0 4 6 2 1 1

'NEXT HIGHEST ]'IFTH 0 2 8 1 2 1

HIGHEST FIFTH 0 0 11 1. 0 1 -

TOTALS , . 0 ,11 . 36 ' 10 ' . 5 6

p, = 0.53 d = 0.50 ,

ESTIIVIATED DIFFICULTY - AVERAGE I ABILITY . 3

----------------------------~------------------~-----------

19. I A n*" C D E 0 I .

53 ~ . 'LOWEST I FIFTH 0 2 ' . 4 0 4 3

NEXT LOWEST FI:E'TH ,

2 2 2 1 7

MIDDLE FIFTH' ,1 •. 3 2 1 4 3 NEXT HIGHEST FIFTH 0 5 1 1 6 1

. HIGHEST FIFTH 1 8 0 , 0 3 1

TOTALS 4· 20 9 .' 3 24 8

p, - 0.29 d = 0.43

ESTIMATED DIFFICULTY -AVERAGE/ ABILITY, 2

.

20. A.1(- B C D E 0

LOWEST FIFTH , 0 1 2 1 2 7 NEXT LOWEST FIFTH 1 3 4 4 1 1

• MIDDLE FIFTH' . , 1· 1 4 1 0 7 NEXT HIGHEST FIFTH 1 1 4 2 4 2

0

HIGHEST FIFTH' 2 0 3 . 5 1 2

TOTALS 25 . 6. '17 13 8 19

Pl = 0.07 d = 0.14 . . '.' ." ,

ESTIMATED DIFFICULTY '- HARD I ABILITY ""·3 ".' .::' .. ' .. .

21. A B>t ·C D E 0

LOWEST FIFTH 1 2 4 3 1 2

NEXT LOw,EST FIFTH 2 1 4 6 1

MIDDLE FIFTH 0 6 2 2 2 2

NEXT HIGHEST FIFTH 0 ·2 3 4 3 2

HIGHEST FIFTH 1 6 4 O' 1 1

TOTALS . 4" 17 17 15 8 7

Pl -:-0.25 . d - 0.29

ESTIMATED DIFFICULTY - EASY I ABILITY 2 .

.'

'22. A B C D E* 0

LOWEST FIFTH . 2 1 2 2 2 4 .

NEXT LOWEST FIFTH 3 2 0 4 5 MIDDLE FIFTH ,2 2 2 0 6 2

.

NEXT HIGHEST FIFTH 0 3 3 3 4 1 . HIGHEST FIFTH' 0 1 1 2 9

TOTALS' 7. 9 8. 11 26 7 . j'.? . !

p]= 0038 d = 0.50.··· .

ESTIMATED DIFFICULTY - AVERAGEJ ABILITY 1

54

23. . '. ' . A B Cl£- D E 0 .

LOWEST FIFTH .

2 3 ·0 3 2 3

> , NEXT LOWEST FIFTH I 2 6 2 3 0 1

MIDDLE'FIFTH 5 3 1 1 4

NEXT HIGHEST FIFTH 3 5 2 1 2 1

HIGHEST FIFTH .' . 3 3 2 . 3 2 I

TOTALS. .

15 20 7 11 10 5

P, = 0.10 d = 0.14 .. .

ESTIMATED DIFFICULTY - AVERAGE I ABILITY 3

'.' .

A B .C D-\t' E 0

LOWEST FIFTH 2 4 5 0 2

NEXT LOWEST FIFTH 6 3 2 1 2

MIDDLE FIFTH .2 4 .6 1 1

NEXT HIGHEST FIFTH 2 8 O .. 3 ·1

HIGHEST' FIFTH' 4 7 2 0 0

TOTALS 16 26· 15 5 6

P, = 0~_07 d = 0 .

ESTIMATED DIFFICULTY - HARD I ABILITY 3

25. A Bl( C D E 0

LOWEST FIFTH . , 4 2 5 0 2

NEXT LOWEST -FIFTH 1 3 7 0 3 ~,uDDLE FIFTH 1 . 5 6 ·0 1 1

NEXT-HIGHEST FIFTH 1 7 3 0 2 1 .--,

HIGHEST ,FIFTH' 1 .. ' 3 ' 4 2 2 1

TOTALS ... . ' . 8 20 25 '. 2 10 3

. .

p, = 0.29 . d =0.07 . .

ESTIl\'lATED DIFFICUL~Y ~ AVERAGE I ABILITY 3 .'

26. . -- A B C D E~ 0

LOWEST FIFTH 5 2 2 1 2 1 · NEXT LOiVEST FIFTH 6 0 2 1 4 1

· '. MIDDLE FIFTH . 4 2 1 1 5 1

NEXT HIGHEST FIFTH 6 0 2 1 5

HIGHEST FIFTH 3 2 1 1 6

TOTALS 24 6 .. 8 5 22 3

P, = 0032 d = 0.29 . ,

I .. ,

· ESTIMATED DIFFICULTY - EASY ABILITY 3 . .

55

27· .

:A..f" B C D E 0 .

LO";,~ST FIFTH 1 1 0 5 4 2

NEXT LOWEST FIFTH 2 3 · 2 5 2 - .

IIIIDDLE FIFTH· 1 4 1 4 4 NEXT HIGHEST FIFTH ·3 O· 5 . 2. 3 1

I I· HIGHEST FIFTH 2. 0 2 .5 3 1

TOTALS 9 8 10 21 16 4 .. Il, = 0.13 . .d =0.07

ESTIIIIATED DIFFICULTY- AVERAGE I ABILITY· 1 . '. . -:-

• 28. A :B 'f;: .... .C D E '0

LOWEST FIFTH· 2 1 3 2 2 3-c-..

NEXT LO\VEST FIFTH .1 1 ·7 2 2 1 mIDDLE FIFTH' 3 '~ 3 3 1 2 1

NEXT HIGHEST FIFTH 1 3 9 0 1 - .

HIGHEST FIFTH 2 6 5 0 0 TOTALS

, 9 13 27 7 6 6

Il, = 0.19 d = 0-36 .

ESTIIlIATED DIFFICULTY - AVERAGE I ' ABILITY 3

., .

29. A1{ B C D 'E 0 LOWEST FIFTH 2 2 · 5 3 0 1 NEXT LOWEST FIFTH 4 2 5 0 2 1 IIIIDDLE FIFTH 6 2 4 0 2

,

,

NEXT HIGHEST FIRTH 10 1 2 ,

1 0

• HIGHEST FIFTH 11 0 0 1 1 TOTALS . .. 33 7 16 5 5 2

. Il, = 0.49 '. .. d = 0.64 ..

-:-

ESTIMATED DIFFICULTY .;. AVERAGE I ABILITY . 3 ,

. , .. 30. A' B · C~ D E 0

LOWEST FIFTH 1 1 8 0 2 1 NEXT LOWEST FIFTH 2·· '2 8 1 1 IIIIDDLE FIFTH ·1 ' ,3 8 1 ·1 NEXT HIGHEST FIFTH 0 l' 11, 1 1 HIGHEST FIFTH' 1 0 12 0 0" , TOTALS· .. ' 5 7 47 3 5 1

. Il, = 0;69 " . d = 0;29 ,

ESTIMA'rED DIFFICULTY - AVERAGE l' ABILITY 3

. , . 56

. 31. A B C*", D ..,. 0 l>

LOWES1' FIFTH 2 1 1. 4 2 3 NEXT LOWEST FIFTH. 1 6 0 4 3. MIDDLE FIFTH' 2 2 4· 1 3 2

. NEXT .. HIGHEST FIFTH 1 3 1 <

2 6 1

HIGHEST FIFTH 1 5 4 0 2 1

TOTALS ' . 7 17 10 11 16. 7

P, - 0.15 . : d = 0.21

ESTIMATED DIFFICULTY - AVERAGE I . ABILI1'Y· 1

32. . ' .. A '.' B){" C D E 0

LOWEST FIFTH 1 1 1 2 2 6 . , '.

2 NEXT LOWEST FIFTH 3 3 i 4 1

MIDDLE FIFTH· 0 5 1 3 1 4

NEXT HIGHEST FIFTH 1 8 1 1 1 2

HIGHEST FIFTH .' 1 10 0 0 2

TOTALS 6 27 4 '.10 7 14

P, = 0.40 d = 0.64

· ESTIMATED DIFFICULTY' - HARD I ABILITY ' .. 3

33. · A B>( C ·D E 0

LOWEST FIFTH 0 0 3 3 6 1

NEXT LOWEST FIRTH 1 4 0 3 6

MIDDLE FIF'm 1 2 1 1 8 1

NEXT HIGHEST FIFTH .' 2 1 0 1 10

.' HIGHEST FIFTH . l' 1 0 0 11·

TOTALS 5 8 4 8 41 2

P, = 0.12 d = 0.07

.' ESTIMATED DIFFICULTY- AVERAGE I ABILITY . 3 . . .

,

, . .

.

A' B C"t D' E 0 . 34. · LOWEST FIFTH 1 '1 2 3 2 4

NEXT LOWEST FIFTH 2 5 3 4 0

MIDDLE FIFTH 1 .4 5 0 3 1

NEXT HIGHEST FIFTH 0 ........

8 5 0 0 1

HIGHEST~IFTH 1 4 5 1 1 1

TOTALS' " . " ..•. 5 22 20 8 6 .' 7

-p, = 0.29 d = 0; 21 . .

ESTIMATED DIFFICULTY ~HARD j . ABILITY 3 .

57

35 • . A'k .. B C· D E ·.0 •.

LOW1:S T ' FIFTH I ' l' 1 1 2 7 1

NEXT LOWEST FIFTH 4 1 3 1 5 MIDDLE FIFTH' 3 0 1 0 10

.

NE'LT HIGHEST FIFTH 3 3' 1 2 5 HIGHEST FIFTH 9 l' .0 0 3 TOTALS ' . 20 6 6 " 5 30 1,

P, = 0.29 d = 0.57

ESTIlVIATED DIFFICULTY -AVERAGEf ,ABILITY .' 3

, ' .'

36. A B C. D* E 0

LOWEST FIFTH ' . 2 ' , 1 3 ," 1 " 3 3 NEXT LOWEST FIFTH 2 1 6 '1 4 MIDDLE FIFTH' 1 1 '2 5 4 NEXT HIGHEST FIFTH 0 1 3' 3 7 HIGHEST FIFTH 0 0 .1 5 " ,'6 1

TOTALS 5 4 15 ' 15 24 ;4 . .

PJ = 0.22 d = 0.29

ESTIMATED DIFFICULTY - HARD /illILITY 3 .

37 • , A B C . D E1 0

LOWEST FIFTH" 0 5 2 1 1 • 4 NEXT LOWEST FIFTH 2 2 2 5 2.

MIDDLE FIFTH 4 0 2 1 4 2

NEXT HIGHEST FIFTH 1 2 3 2 2 3 HIGHEST FIFTH 3 0 2 2 5 1

TOTALS . 10 9 11 11 ' . 14 .. 10

P, = 0.2l d = 0.29

ESTIMATED DIFFICULTY ~ AVERAGE I ABILITY 3 .

.

38 · A B C* D E 0

LOWEST FIFTH 1:' . ' 2 4 2 4 NEXT LOWEST FIFTH 0 5 5 0 3 MIDDLE FIFTH 1 2 3 2 4 NEXT HIGHEST FIFTH 1 3 8 1 l'

HIGHEST FIFTH ,0 0 11 1 1

TOTALS 3 12 , 31 6 13 . , ' ,

d = 0.50 .

P, = 0.46

ESTIMATED DIFFICULTY -AVERAG~ ABILITY 3 "

58 . . ' . . .

39. . A B· C . -It ·D

.., .

""' 0

LOWEST FIJi'TH ..

5 2 0 2 3 NEXT LOWEST FIFTH 5 1 2 1 .. 3 1 ,

. MIDDLE FIFTH 5 2· 1 1 2 .1 NEXT HIGHEST FIFTH 5 . 1. 1 2 '. 5

HIGHEST FIFTH 8 •... 0 0 1 2 2

TOTALS .. 28 6 4 • 7 15 4 p,= 0.10 d = -0.07 .

.

ESTIMATED DIFFICULTY - HARD I ABILITY 3 .

40 .' A . B-lt- C D E' 0

LOWEST FIFTH. 0 .3 1 5 3 NEXT LOWEST FIFTH 1 2 1 8 . 1

MIDDLE FIFTH 0 2 1 7 2

NEXT HIGHEST FIFTH 0 6 3 4 1

HIGHEST FIFTH 0 5 1 6 1

TOTALS 1 18 7 30 8

p, - 0.26 .d = 0.14

ESTnlATED DIFFICULTY - HARD J ABILITY 3

.. .. ..

41 A'>t. B C D E 0 LOWEST FIFTH . ,

1 0 0 8 0" 2

NEXT LOWEST FIFTH 6 0 3 2 1 1 1

MIDDLE FIFTH 1 3 3 3 1 I,

NEXT HIGHEST FIFTH 2 0 6 . ' ..

5 1

HIGHEST FIFTH ·9 "1. 1 2 0 -TOTALS 19 4 ·,13 20 3 4

p, = 0.28 d = 0.57 .

ESTnlAT;ED DIFFICULTY - AVERAGE I ABILITY' 3

,

42. A .B C DJ( E 0 LOWEST FIFTH 2 3 3 0 3 NEXT LOWEST FIFTH 1 6 0 1 4 1

IvIIDDLE FIFTH 2 2 1 3 2 2 NEXT HIGHEST FIRTH 3 2 2 2 1 4 HIGHEST FIFTH' , . '0' 1 4 2 4 2 TOTALS 8 14 10 8 14 9

I. p, = 0.12 d = 0.14 .

ESTIMATED DIFFICULTY - HARDj ABILITY 3

59

. , ,

A 13 C D ElE- 0

LOWEST' FIFTH ~ , ' , , 1 0 1 4 4 1

NEXT LOWEST 'FIFTH , 1 1 1 7 3 ,

l'rtIDDIJE FIFTH' ' ,

, ' 1 0 0 9 2,

NEX~l HIGHES~' ':b'IFTH 0 2 0 11 0 1

HIGHEST:b'IFTH' " 0' 0 0 13 0 I

'fOTALS ,

3 3 2 44 9 2 ,

" , 1', :0 0.13 d :0 -0.29

lOSTD!!.A'TED DH'FICULTY '- '.c;ASY IABILI'J:Y , 3 , , ,

" ' ,

"

44 ,

: A B,G~ D E '0 • , , '

'LOWEST FIF'rH , , 0 1 2 ' 4 3 1

NEXT LOWEST FIFTH 1 1 3 6 2

mIDDLE FIFTH, 0 0 1 8 3 NEXT HIGHEST FIFTH' 1 1 3 8 0

HIGHEST FIFTH 0 1 2 10 0

, 'TOTALS ," " 2 4 11 36 8 ' ,1 ,

,

P" - 0.16 d - 0 "

,

ESTDI'ATED DIYF'ICUUY - AVERAGBI ' , ABILITY' 3 ,

, ,

45· , '

A B 'c D E ''lE; 0

LOWEST FIF~~H 1 1 1 3 3 NEXT LOWEST FIFTH 0 2 0 6 5

, ,

MIDDLE FIFTH 2 2 0 2 6

NEXT'HIGHEST'FIFTH 1 2 0 3 7 HIGHEST FIFTH', . i 0 2 0 1 10

TOTALS " ' 4 9 1 15 31

", Pl - 0.46 'd- 0.50 " ' ... ~,

ESTIMATED'DH'FICULTY -AVERAGEI ' ABILITY' 3 , , , ,

60

The better items such as Nos. 1 and 2 stand out quite clearly

while ,those of more'doubtfu1 value ,'such as Nos~ 14 and 24 are

equally 'obvious. In the 18,tter case it seemed extraordinary at

first, that a11candidatest:r'ied the item and yet the facility

value was very, low'and' there was no, discrimination a tall. It

is reasonab1e:to'as8ume that most candidates felt that this was

a straightforward problem and the majority (26) ,chose option B'

implying that the first two'graphs are correct and the third one

wrong. ,Atota1 of 31 candidates realised that the third graph'

was wrong but only 5 of, these also saw that the second graph'

was incorrect. ,There'wasa1most complete failure to understand

what happened to the 'velOCity when the particle hit the 'smooth

horizontal surface at an angle, with the assumption of no

bouncing. The item is'poor because it is rather too much of a

"trick question". "This failure of an item does at least

emphasise the 'advantage of an 'objective-type presentation in that

the analysis so clearly and so simply enables the weakness to be

shown up. By looking through the tables of results for each item , '

it is possible to pick out options which are not operating

properly and from the p,and d va1ues the effectiveness of ' the '

items can be compared.

The total ,mark for the paper was 45, with the ,lowest mark l' '

and the highest 28. The spread is reasonable but i twou1dappear \

that the paper as a whole ,was rather too difficult 'or that the"

time ,given was too short. The fre''l.uency table :for the raw'

scores:i-s:-' ,

.

61

. , .

, . , . , , CUIilUL. , CUEen. CUMULo

SCORE FREQU, FREQU. SCORE FREQU. FREQU. SCORE FREQU~ FREQU. . ..

. .

1 'I , 1 11 :5 23 18 1 54

4 :1 '2 12 7 30 19 5 59

6 '2 4 13 '8 38 20 4 63

7 2 6 14 7 45 22 2 65

8 2 8 15 2 !t7 24 1 66 .

9 5 ,13 16 2 49 25 1 67 , . .

,

10 . , '5 18 , 17·· 4 53 " 28. 1 68 .

The mean. raw score = 13.63

Standard deviation = : 5.03 .' ,.

The distribution of the scores.:l.s also illustrated by the'

.graphs:-': "

(i) a' cumulative frequency graph and (page 61a)

(ii) a frequency graph in which the 'scores are divided'

into the' intervals 1-5, 6":'10, 11-15, '16-20, 21':"25 and 26+. (61b)

A more refined method of, comparing the effectiveness of the'

items is to calculate the point biserial coefficient for each

item using the formula' , (see ApPENDIX A)

.To compare items within the test, the point biserial coefficient

. is calculated from the. raw scores and' some examples of the

calculations are given:-

ITEM 1, ' , ' , Mp = 24% =14.d4 . .3 '

I

, . I

, (this, c01llpares the number of successes , with the number. of attempts) ,!'" . I

p = 0.576

q = 0.424' ,

MA can. be 'taken. approxima tely as the mean of " . I

all the1scores, since the number of omissions is small (2) I

i

i. e.

Similarly OA. can ,be ,taken to be the· standard deviation for· the

whole' = 5.03 '----------------------'------------ - ----

"

"

, 1, , , -.',. ,

j,~; 'I :"i,' / 'I "'1',1 ' ..

,\"t "1,,:"

,

, ,. :1' , ~,!; " .;.

:'", ":1,," '. ,,'

, , . , '," "'I " :"1 '-:1-":;1 . '1'1 'f:-"'J-r,--;t~i~rr" T'I"II'I'il'" I' - " .. ,., I",,!. , ,1.",l t ... \ ' :'1.tW~'I"'i'H''-''I.I''''--I+L'~'j''f'''' ·t,-:-: --, ~~j • - • I.. ~ tt, " . ,. L' I, I 'I L, ,',. 1;4. ....-t j , J -I I ,"j • ~ ~,",.j •• f . L L,. '-" r 1 1-'

-, .' I ,1 I T I I, I' 1 ' I-I' ,-1 - W- I· -1 ~ 1-'1 ,. 1'·- • ~-'-~ 1 t i. ~ ,., ,I -; I " I ~cl I , 'i" + "} I ~ ~ ,I t ~ ...... J • t I1 '1 • , •• , ... 1 .. I,.. ~. . '~" • ---- ': - --, L'-. .• : I ....... , h·· 1,+.,- I· .. j.rti i'-'C,~ ... t-- -s: 1-. :+ t -. --'-[t:- --'--'oO" -- ,

• ~-"' 0 i. or. ' ~,I ,~ -t '-~ . 'J r :.t+l- 1 ';-1''''! ,-', ... J-I-4-L+, . -oH-'" I r 1 _"L ~r"" r -~,.,. ~; ----'t' -"'- I--~ ,l,·~-;.-·- 'I· ri' -I ._L 4"f_~t_+_,..... ...... .p.~·L~l _ .• ~l_ , t~ -1-•• --<. r . ,. c.. ,,,., ... :, ,. ..,: 1. I,:t '1'\ i~ ri', 1 ,j1"H'l"l"""t-I-L ' r--.. L- --.;,:: ~ ... t, , . ,--, ~ ~ . f·~ ! . I I' I' ~ l' t I •• t 1 -I • 0 0 J 1 I ' - -- -, , 1++ I 'l \ '-1 ~ .- .. ',·1· .;.,' .•.• '-.~"1'~[ ',1,1, .. Jj~~I __ ~:r:, .. !_LtJ_,Lt-Lo~ . .J~".!-__ l."""Lr-Jll'1;_, -11 !-

l~' .... t • - , ... I l' I . . .. \... j' ' • ·1-· ,:" - ..,if: 1 1"1 - t..1 - Woo • -1-.• -1--.. • - _... I.

-, , . ! ·~I.t "', Il. '~I'rj~ ·fll ',1 I''''~..!. :_~.L.,-,.t ... .J--j-!OllJ,t·I'''' ~ ;....1.+ .••. - •. . , , 'or' ,I, _ ., 'I .'j I 'I , I' J"~.-j " .. , .,. --1' I I ,,' "I .,..... " ,

.... , • , .•. . ~ .. ,,....~'"" • ,:.~ 1,1' I. '" -11,,' 1-.1-.." ", L~ 1., ... ·:: .. t L. ""'"t"r" ! ~: _'M~'-"_ -'. • .... ·--r-- :-.-.+ ...

• _._' 11 I .p. "1-j~" tl·j~trt·II" 1~~l..~'I!..I+·;ll ri-I'tl"!! '~, '--;"','/f t. • I J II ~ - ,·f I r I"" f .' ~ I, I I I <_1.!-- ~. ttl -~ tit-'- r 1 ·1·' I;, "',"" ,.

,:" I 'r' "'I,,· .. tl"" ;·";j-I"J'f:f:,;--,,,-r1·t lf "lf.;. :j'rlt·"" .,1.,.".,:, .. -- ','1 . ',., ,I 1,1' .[tff+-'r··""'-"C'li'" 't11--' ,," ,.: '1 ,I I I I r,j' ,j- -11' l,.l' ,\.. h Il~t--tLi ~ ... ' et +_. -t." •• - . ""','- : 'I ,.,1,,11'11'111. 1 ,"I li;h"""'il [IWIII' ,., .. ,,11 1 '11-0 ,,1':""1'." .. I II ,~!, !-" '''Jr,'' .. J j ~ I! _._,...L~+_I.,- --Tt .... , 1'---" --r J_,_, •. -., •.

• I '11 I .. ) ... t· 1 I- ;\1,(1_·, ft !I~'+I'''- -tt 1"1! :.- , o+_j,(c.JI: .... j .. 1"'10:-, , I 1 ; 11 j • .. '1 I r l I I !. 'i I - ~ l ,I T' ~I 1 ~J. I· ,. I ~ I ,I- I ! t •• : '" ,_, ,I ': . --'~-:oO ,.----.• - .. I '1--1-"'C'!~'-~;""""1fLr-"T -.\..,W~,;- '. -T~ l-l-".f:+T.i- 0':." '.1"~; ":: ,,'I,· j':.: 1,.':"l!;·t,~c;l.l;~':~ t;'L~rt:'~f'~ :'T'lt:d~trdl:"', "":" d, '.' 1\' il, ,:.1',' : I·",' ,,'·t ,tl·"'l! I': I]: .. tJI':'·ll;1 ., :,i

-- I • 1 ' '! ., 'I hI' I 1'-,·· I " '" I' ~.j • I " I. I . , • I I t '1 11 ' 1 I· 1 . I · !... • -.! ,~.~-.-\-. ·f·' I' ~ .1 1" i--l".';'r!'"-~""-,! ·,'-Ir'~~-I·"I· I'~ .~: -.·t,

, \' ',1'1 1,1,' 11 1,L'1 "I" -"'" 1.tl'll·I' "I'I'·l,j '1,1 ' ~l' "".' ":, , ' 'I' , I " .• ' 1 i : ' I' t I I., 'I "1 .. 1 '! I't" J' It' , ,'" , " , , ... ~ , I I! '"' , • I I I 1,' l·-rI I " J I· . 1 ~' " t f I' . i , , I "I I' -I I. 1_ + \ t .. , I " I I I .. t I... 111'1-1 ,1·, 11 I . -.,..... ---, .. , , ·--tiIJ-t1j I'f-,t ,t •• I., ,·, .. 1. IfJ'I-I-~T l--t"1'tlc, +l~T' :":- "' .... (\

1 , 1 I I· :'!',' 'I I I t" 1'1 HI:' "1 1 I, , I I I! I i 1\ : i,I,. C. l' , '\ I I • I, 't ,- .1 , -!. 't 1 I 1- , 1 1 ' I I - I ,. I .

, " I ,. , '1'1 I I •· .. 'I'"I"L.I 1,11 1111 , '--", 'I '" 1",1,. :,~ I " , i I I, '" .'. ", ri' I ' I r' "I t, , I' 1 I f I """, l' ' I ' , I ,,. , : .:-"'- '\ ; .'~~v·'··I'~"""lrt-~;· 1'"1 -I--"j .,._.J ,;' Lf._.L"t.j.J..t J · '-'-1,,11,.,." <:,

'Oo' I' '.' .. ' "I'l ,,','. " ... , , /" lj " ~ 'r' .", ~ : ", ',1,1 11 l-l'L! 1 ! ,ll"lt'I' '_11' I-~~~Ifll':li ""l iOj'I" I •• ,'

I '" I " I I I' I'" . , r - l' / 'I, ,.1:' J', ,. , . I' I ,..,' I :l> 'I ..... ,1, . • ,jl -.'.1 '"' I j Irl~'l' j • J. --, "t.,~ J 1 1- till "

.,-'.,-... ,. , ,_. C' ._.., ..,--.It. T·,L ,-11"-,,,,,, --H+-. . -'+--!-r--- °C,,=" "", ·i, "I ·,t· ,'~l II.,_,~I 1,li~ \. '11 It, III 1--1,·1",1 .. : .. ,.:~I

'-', I"~ 'I """,J"11Ii--illl"I'I' 'rl'l rH"r"" t~I'~i-Jl "t'II', '.".'.'~ I : ' ,-I' !' L.. I ... , t·· ' 'I . I j' -I' j 1 '-"1 .+ I. I 1·1 • "i L I j I·, • I ... 1 1 I l '! 1 ~ ~ I I I I I ,. I I • " I ,~-I I I I , ! ~ I ~ "J

'i ." .1·-1,) 1'·1-,·, 1"1'1,,,,·1 ,.,\. 1'" 1 1'1'"' ·t·: It', "" " j. • 'I l" ,I I "r' 1":,1 ,1 ' • I ' • r': 1 1 I"~ i ,~ 1-·1 ': ·1 L ' " I· I '\ --."" '". I" , r' " ,-- .. t· t 1'1' "I .", t !. , ,.,-., ", ' . .. .• 1 • , , ,,-," I·' ". . j' '11 'I" ,. " ,,' [. , , IJ '

I j I I" I i ' 1 I • ~ I It L ~ t. I --t! . 1"1 ~ I, J 1.< t • I' I + ! I I • ~ 1 ' " .-... ~-- -' ... 1.. __ ,, __ ,0 ..... , .... -. __ .j '"-+ ,. ,~~ H--·r t-.. , -j.-,. -I-J-rr' .L.,J:I___ ,_l"_""r' oft --~ ----

" ,. I' " I, j,'" -,,' I'j" , ."" I " 1.1"11" I I '" / ,. r'l , I'" ': ., ' I I I • I I I , ' < 1 ~ ~ 1 I ~ I..... . J..,!, I 'f t ' • " :, !":' I , I I'!' '. r i I· I, ': 1.- " L : I,' I "11 I ' ! I i I I I I" i I " I '

-I • ~ 'i . I ~.' ',,1""'1' I ... ~~ .~ !! \ I" I, I-~., , I •

,·,· .. 1 " .. ".1"1" ., .. 1 :1-' t, 'IJ( l' ll 0 ;r;r,--~,.I· .. ,b":1' 1",.1-1. ... ", .. , , , . ,.. "I, 'I' "" 'I" 'I I , 'I r j, - ,. ,. , I r I J I 'I I [ .. 1'1' .. '" 'c. ' '1Ij"r I " I I·:~t.j... 11,1 I'-Ijl ..... :~I" ,',1 '111111.,lt'f i ., I '\-li ...... '. ~ I " " ~'r I 'MI'I ~ '1'1 .' I,:L'I-'''j ~,' r'" . .1 111 :'"

I ,1 I I I, > '. • I I j .\ j t j.. 1.. ~ , 'I j ; I 1'1 • 't '; It' : . •• I~'" .":' , ... 11 .. , .. " ,--- 1'4 ",]1_ .. ", ",+..1, 1 I +'1 r' .-- \ I, '1--' "I' "1-< 1"1- (;) .. ~ .....

, . I I ,I 1 1' t I I I 1 I I" I~" " I I I~·· l' , j t ( j ,I rt , I'" j " !. . j! r:- '. ; °1 ~. I ' ___ r 'rt I· I 11 -I I lot I I. "I' ,~, ",., ,1"'1" I 1·,·1:, I1I ',., 1,1 1'1' I'"'' ",'I '11, '.~! <, t·\ I ' I - I ' 1'"1 I I I I cD ., + ~ I .. ..1. j I I , ., '" 1 I I I ,'. "" ,I' .•

'·':I,'i·'·" .,,~ 'I' T"-'''' ",·-11,,: ,I "·'~IJ. ·tt." .. f .•• I""""rfl ~,; "" , "I ' 'I " ,.,' l' I ,I 11 I ')' 'I" '.'., , , , I 'I, I' . I I t 'I' .. ,,' " I' ,," , I ' I "" I ,1 ,,' ,"" I ' , "" I' 'I i 'I "J I' I ' \ ' I" ,," ri", 'I

i --' ~ 1 ~ "" 1.' : I , \ 1 ' , 1 ' 1'1 \ I \ I I" r~ I'" I I 1 1 ",., I I' -l I ' . I' .' ~. 1 I • I • ' ~ I • , • 1 : j I I. ,I I, • / • ! : I I, I • • '1 ' I I

·,·· .. ·"'1'" . I 'I .tL.J_.I".l_I"~" tjfJ,-+, .. '.7':r.r·n.j"T;-, .. · .... ·_·"--"[lT .... ..,. I " ," '" '1'1'·'·I"I·I·' l

rl l" ""1 11 1"1 ,I"" t··, '1, "\"1'" " " '. " ' .. , I·" .. ",. ,', ., ',I" ' '1" '" 1'1+'" l' .. "., '11'

.. I·, "1 I I.,~, ,. 11't 1I ,I IJ. '1°11' "·1'''.'1 I I., I 11-.L1j·:'I~ ,. . 111 11 I' l,li It'''! '1 '1'··~·11 I l'II"'-ttlill 'j ., ". "": "I, ",.r I' ) ""·I"I.IIIIW'i'·I"i'I'" 11,111 """""11,1,,:, , . I' 1" i' ",,", I, " ,.'" , !' I" "I. , . , , ~, ,1 'I'! ,'. I ,i : l I' I ,'i' i I i"' 1:' .j I -I 1') ',' ,t 'l I· ,

1 I -10, 11 I , I j ", I + 'of " f 1-1_ 'or "! "'" I .' .• -- -'re': ...:.~'.' :"11, -, L! .. ;.t._ .. ~ ~.ldJ.i 4'U I..!.-U: IHJ.t:L~IT' I :~:':·lL .. ;)j.~~.::l~l'':':; ~ ... I . ~ \, ! ' • I' - '~'I' ':1,-[1')11 I·' Ill., ~i·H-~-'t'i·h 't\' ,;~-~-L!'I-' 11:, ;"',

• " • t • I " r-r ".1 t I' 1,1,. 1,\. "I" I., 1 i I f I'·' l ., , , 'I I , t,' I"" '. I.",. '1IIj'I:1 ""'1,/'1. 1',11.1'1'. ,j.,II'tJ\j'illll :1. '.'" ""j , I '\ ....... '" I'" j" I"~' I .. I,', "" ',., "I I"'· .

... , " 'I" ,,',., '\·f·,,··/ '''' .. -I,'j,~~'1 l"'t"" '.--' .. '" ,." , " , ,I", "11 '·'''1','1 '.-f '11", 11 i1 1"'1 HI'IIL -1- •. 11'.1"1' . 1,,1 t' l •• ' " ,r;.,~ ..//1" I 'CI ''':'1,.~;- 1 I 'le,·'JI, I .'- """,1-,

".1"11 .-,of '11~("'111 ·'.'-I···'·····tJ~c·!'I. ". '.".' ,- I ~ I I Ill~ _,lllll ':r;' '. t, II·.\.:'.~ j III 11' ~ ,), lJ~I' pI, "1 ~;:"

.. ~ j .• ~ ,--'- .. I.U. --J ~ .. ~ ... -~ '>-e I"et)-, T ,., 't. + 'Ti ++ ~ +·...c"rrr· l' 'r +--- -J'H1~ .. , ~., , .... - '-", H" ,'" " , ' r' ·.:'11"11: ',:11 1 j el:, i, .< .• ~j."j"" "ll' I.'" ,. "·1"'- 'Ii':." .' 1 ' " ... , I 1 I _1. • .. 1 I • ~ .- I I ~ ,. ,J. ....... _ f"' ! ,t ,_I !..J I t l " .. f .' .', I " I· I I·, I. I j : r It~ 1 I I '1 I . ~ ~ f ! 1' 1 1-1 -r\. ~ , • 1'1 11 l I_ t.. -I "11 '.. (,I i' . r. ' , I 1. 1 . I! 1 i 1 [ ,)1 -11 1 , i· J.~ , ·1 'L. - III i·, 'f ' 1 i I ··i:, ; , .

". 'oO, .. " j L.1·""I.\tL ... ,',',1' .. ' .. 1. 1 .... ----.-.. [ IJ .. " .. ,,-, ... 1 H" , I lit ' " I \ ' 1 j , • ,~" • I . ,I I '" J • t ,I i \ , -!-.-, • I I , . I, I'~' " I J I I,:: '.

11' 'I" , '1"';'.11 11. 1"1" ."I" r l'i-J-j'HI"'II,llj,jlll,t 1'1 1 "/ I ·1":".' . .".. ",., .• ' I ., ;, ,I ., I "I ,,.h , ... , j I' 11" .. ,' I "',"'" 'I r . I • I: .' ,. -t ~ I j i 1 ~ ., <I- ~ '~j. t -1 .. t'r;T~ 'IJ ~t .. ~ ~I j 'I' ~r. I .' '< , ! , !. ' '

,.", .,' ,·, .. t ,",' '"-!.r- .. , 't r- r'1' ,-'+-i ~ "T "-, • ..,.. t· .. " .. "",· 'I ,,,,~, I:"i·' I' 111·'II]·'TiJjrl" ,."tr ,·1 IT "I.H.!I,,}! .[II"rIL'll,Nlt1'il.'::I::" "'1 If·!' I I I . r- .. I 111 1--- 1-I.-I-'I..~I-11- ~ .1 tj.L I. 1"f· I -", I i, '-1" ,1 I) _ .. \1, I:' .-,i_ ;',1- I_I 'rlJ·",.fr~·1 ,101,_, LI ri'I! .. ,

1',111',' 1 1:.1 11,;1'''1 I1 t·, ,If.h-r f - r..J~Ii,.,.1 ~l·~ lIlt ~ I If'I·!I,,- .. · .. ,. r'-'-' "I" .... J·J.·"l·_··_I !-;J.J. ._,_"<-1, .. ~'j:"r4·tL·TI .. f .. -.. J·"I+'~r~· .. ,'--""'-"."'1"'''' , , ' I'" I 'I I ,I I I I' t ~ I t I' I ' 'I ' ! 1 'I I 'I 'I I " "I , d ' j I " ' , I I .. ,f'; 1 I I I: i J ') )!'!' t· I , I, i I (I , 1 J : ( ~ '! I I ._ .... - • ",' I._~'" , .•. ''eI9''.':''>",. ..... ''''-,', :'~"'-", ,<::' ':'.;";':"'i'" ;,:1." '\:.:;',;',;:'~: "I'. ; .:

, .~_ -'1"".

: ,"~' " '

, , .< .

"

" . ," , <- .,

.,',

Hence r = 0171 " .576 ,I '~' 0.16 '57lij I :7424

62

Thus r :is rather 'low although PI . and ,d given, in the, analysis'

table 'for this item are of 'an acceptable, size. '

M ;" 507=15.84 P 32, ,

- 867 -1'3' "16 - b3, -, ·e

<lA == 4~92

p ::'¥ = 0~508and. 03, "

q = 0.492 then

= 2.08 /.508 r ,4.92 .492

= 0.43

A good item.

ITEM 5' "This is obviously poor with a low PI and no discriminattn

M = 167 =13.93 P ,12 '

P = 0.194 and,

, q = 0.806, ':

A very approximate r (all tM.tis necessary ,in this case)

= 13.93 '-'13.63' . J:J:9'4: I:;' 5.03 ' .;-:oot; , ,

, ' ,

- 0.03

ITEM 7 i • Acceptable Pi and d (quite ,high). '

M = 422 :'16.8& p 25 ,

p = 00379 .and:

q = 0.621'

,Ignoring the small number of omissions,

= 0.50

This is a good item except that option B has been inoperative.

I '

i

~fu-;r-9 Acceptalp:e IIll t Qqrti te ,high) and d. 'n, 6'36 FtP , r -, ,

~!' ~ =,43 " = ,14.79

p = 0.662 and

CJ.=0·338' . . .'. 1.16!7bb2 ~gnor~ngom~ss1.onS, 'r = '5:0'3 .j,733'iJ

'" 0032

A fairly good item.

.. I

ITEM 121 'A,very low Pl'and ,zero discrimination. '

A large number of omissions (10) and,the r is

found:to be ,-ve. A very poor item.

ITEM 29":Pl and dare,acceptable.

Mp ~ 5j~ =16:39 '

p = 0.50 and

ignoring the omissions (2),

r _2.76 - 5:0'3

= 0.55

A good item.

ITEM 30' 1 ' ' 'Pl and d are acceptable.', "

rlIp ;;'693 = 'i4;74 47 ,'",

A fairly good item.

~' ..

Pl and d acceptable.

A large number of omissions (14).

M = 43,6 =16.15 ,P 27 ,

p = 0.50 and q = 0.50

64

(which with"" A IIItlst here) ,

be separately calculated -

=790 =i4• 63 54

= 4·74 r = 1.52

,4.74

= 0·32

Since the' number of ~missions falls from the Lowest Fifth to

the Highest Fifth, this is a good item.

ITEM 39 A low Pl and a negative d.

This produces a negative r and is a poor item.

, ITEIVI 45 Pl and d are acceptable.

The 0 column shows ~ero, but in this case there are

8 omissions since 8 candidates did not reach the item.

Then

and

_ 488 _ Mp - 3T - 15·74

P = 0.517 and q = 0.483 M _ 852 '''A - ~

= 14.20

~A = 4·76 1. 54/517

r 4.76 :;nr.r

= 0033

Although only 60 attempted it, this is a fairly good item but

option C was not particularly useful.

These calculations of the point biserial coefficients have been

picked out because of their faults or qualities and they serve

to illustrate how all the items can be compare,d with each other

in efficiency and usefulness.

C""

65

Finally to give some ;measur~' of the' HHahiE ty' cif' the whole

test the;;:Kuder-Richards~n Formula 20 was applied as follows:":'

>['82 n J ~.

r KR -' n' - l Pj Clj n=r 2 .' s

where n is the number of items in the 'test, sis the. standard

'deviation ofthewhole'test, p is the successful proportion of

candidates who attempted each item and Cl = l-p. ':: "

Reliability is

a measure of the accuracy of the test as a "yardstick". It is a

measure of how far th~ Pilot Test could be relied upon to assess

the abilities of the candidates if the test was given at different

times but under the same conditions and on the assumption that

the abilities were unchanged.' The reliability could thus be.

measured by comparing the results of the' Pilot Test with the

results obtained in an exactly similar subseCluent test. The

obvious difficulties here are (i) it is most unlikely that an

exactly similar test could be constructed, (ii) it is assumed'

that the abilities of the candidates are unchanged either by , .

taking the first test (no effects of experience) or by any

developments in the interval between the tests and (iii) the

exact reproduction of the physical' conditions of the candidates

·themselves and of the external. arrangements would be most unlikely.

An alternative is the "split-half" method in which one part of

the Pilot Test is comparedwith another part, e.g. splitting the.

test by picking out alternate questions. The disadvantage here is

that different methods of splitting the test tend to produce

different reliabilities: The Kuder-Richardson 20 Formula was

easier to apply in this particular exercise and it has the

. advantage that it is not. based on the. division of the test in

one arbi tra.ry way. However, it is assumed when using this

formula/

66

formula that the 'test is as much as possible a "power"'test and

,there is no undue emphasis 'on speed so that almost every,

candidate has the chance to attempt every item. It is 'also

assumed 'that the items are 'of nearly equal ,difficulty and inter­

correlation~ The formula is based on the logical definition' of'

reliability, i.e.' for a set of measurements it is the proportion

of their, observed variance that is the true variance. 'It is . . '. 2

assumed that observed varia~ce (s ) consists of two parts true

variance (~t2) 'and 'error variance (~2), ,and that the error

components of the measurements vary at random, their mean is zero

and. they are uncorrela ted with true values or with other

measurewents. Then

s2 =0 2 :' + er' 2. e t

and coefficient.of reliability, a- 2 = t

7

, , '

. Guilford shows (page 411) that in the case of a test consisting of

items which are marked 0 or +1,

2 n . s = I: p. q. +

111

n 2 ~ r ij IPiqiPjqj'

where r ij is the correlation between 'item i and item j where the

subscript j is. numerically greater' than i. n .

This gives the numerator' s2 - ~i Pi qi of the K-R 20 Formula as

equal to the sum of the co-variances in which the' source of true

variance lies •. Thus the ratio of this tos2 is the basic

definition of reliability. The factor n is an attempt to' n-l

correct the result in order to take into account' the fact that

~PiCJ.i can never be zerCi so that the numerator can never become s2

to give a result of 1.00.

Applying the K-R 20.Formula to the Pilot Test gives the

following:-

and n =45

Then r KR = ~t' rl = 45 [1

44

= 45 (1 44 .

8.562J 5.03 2 .

8.582 ] 25·301

0.3384) = 0.68

The· standard error of the measurements

= s Ii r KR

= 5.03 x 0.32 = 2.9

The reliability is lower than is. desirable and is probably

because the test was too difficultJthus encouraging a certain

amount of guessing. The latter increases the error variance •

. On the other hand if the test had been made too easy the

67

dispersion of the results would have been reduced and this also 2

• reduces the reliability (coefficient of reliabili ty= 1 _ CT"e ).

7 The aim incons-tructing a test is to strike a mean between the

two extremes and in particular to reduce the effects of guessing

by providing a reasonable number of options for each item (e.g. 5),

68 ',Comparison of the results of the objectiy~"tf!li?:L_witB

, I 1 the, corresponding "'A' 'level :tesul ts' - ,:, , ,',' ' '.

The candidates were numbered 1 to 68 and the results were as 'fol1ows:-

RAW SCORE

1

,2

3

PILOT TEST CANDIDATE Nos.

66

4 63

5

6 59, 30

7 49, 7

8 ' 65, 2

9

10

54, 18,' 19, 26, 36.

46,' 50, 52, 32, 39

11 48, 51, 68, 34, 37 , ","

12 43, 56, 57, 64, 13, 22, 25

, 'A' LEVEL RESULTS RADECANDIDA'rE Nos •.

f

o

47, 49

12, 18, 59,63,,65, 66, 68,,48, 50, 56

:E 19, 26, 30, 34, 38, 39, 64, 42,,43, 46, 54

13 3, 6, 14, 21, 27, 28, 31, 35 D 4, 20, 21, 23, 25, 32, 36, 60, 45, 51

14 45, 62, 1, 5, 8, 16, 20,

15 '12, 29

16 47, 38

17

18

40, 42, 58, 60

4

19 . 53, 55, 9, 15, 23

,20 44, 61, 17,33

21

22

23

24

25

26

27

11, ,24

67

10

28 41

; ,

/

, J

/ .'

C

B

3, 6, 7, 13, 22, 27, 3 , 61, 44, 53, 55, 57

I, 2, 5, 8, 10, 14, 15, 16, 2'4, 28, 35, 37, 40, 52, 58

A '9, 11, 17, 29, 33, 67, 41

Candidate No. 62 did not sit the 'A' level examination •.

- - ",

69 , , , , These results were compared by means of the ,SIJearman ranl{-, " '1 di:lfeience 'Correlation meihod~ , The details are as folloVls:-

CANDIDATES PILOT ; I I A' LEVEL' 'D D2 ,RANK ORDER' , , RANK ORDER

1 26t ' 15 llt 132.25

2 6ot", 15 ' 45t 2070.25, '

3 33i 28t 5 25.00 '

4 15 A " 392 24-!a- 600.25,

5 26t 15 llt' 132.25

6 A' , 332" " 28t 5 25·00

7 62t 28t 34 1156.00

8 26! 15 llt 132.25

9 12 4 8 64~00'

, 10 2 15 13 169.00

11 H 4 t 0.25 '

12 ' i'

22'2 6o-il- 38 1444.00 "

13 41 28t 12t 156.25

'14 'A 332 15 18t 342.25

15 12 ' 15 3 9·00

16 26! 15 llt 132.25

17 7rr 4 3rr 11.25

,18 57 60t Jj! " 11.25

19 57 50 7 49.00

20 ' 26t 39t, 13 169·00

21 33! ,39t ~ 36.00 .' , .

, 22 41 28t 12! 156.25

23 ' 12 391.. " 2' 27t 756.25.

24 4t 15 lot ,110.25

25 41 39! It 2.25

26 57 50 7 49.00

27 .33! 28t 5 25.00

28 33t 15 18! 342.25

29 221.. 2 , 4 18! 342.25 '

64! ' .: -'

14f 30 50 210.25

D ·2 70

CANDIDATES l'ILOT 'A' LEVEL D RAI~r: ORDER RANK ORDER' , , r

3 . I 33" 28t ' I 5 ' . 25.00 ' #'"f'~' .... ,

32 52 39t 12t . 156.25

33 . ' 7t 4 3t 11.25 .

34 47 50 3 9.00 ' .

35 33! 15 18t 342.25

36 57 . 39t 18t 342.25

. 37 47 15 32 1024.00

38 20t 50 29! 870.25

39 52· 50 2 4.00 ' -1

40 . 17t'" . 15 2t 6.25

41 1 4 3 9·00 .' ,

42 17t 50 32t 1056.25

43 41 50 9 81.00

44 7! 28t 21 '441.00'

45 26t 39t 13 169·00

46 . 52 . 50 2 4·00

47 20t 66t 46 2116.00

48 47 60t 13t 182.25

49 62t 66-k 4 ' 16.00

50 52 60t 8t . 72.25

51 47 39t 7t 56.25

52 . - ·52. 15 37 1369.00 ' .

53 12 28t 16t 272.25

54 57 50 7 49·00

55 12 28t Ht 272· 25

56 41 60t 19t 380.25

57 . 41 28t 12t 156.25

58 17t 15 2t 6.25

59 64t 60t 4 '16.00

60 17t '., 39t 22 484·00 i .

C.AlmIDAT:>S' PILOT' , I ' A' LEVEL'

61

62

63

64

65

, 66

67

68

RANZ ORDER ',,' RAN:;;: Oil.:::lEil.

28t

66 6<*

41 50

6o! 60-}

67 ' i 6~

3 4

47 ' 6(}}

641)2, J "

N(N2_1) , then (> = 1

= 1 _ 6 x 19637"'" 67 x 4488

= 1 - 0.392'= 0.61

71

j) .'

21 441.00'

• 5t 30.25 i "

9 81.00, ' ; ;

0 0

,61; 42~25

1 1.00 . , ,

13t 182.25 '

Eo2 19,637.00

Guilford' (Appendix B,":," Page 528)·gives ,the rank-difference '

coefficient of correlation as 0.432; ;when N is 30, at 'the 0.02

level of significance (two-tail test). "Thus,with N =67, the'

above result 'of 0.61 is quite high.

72 COiWLUSIONS I I , : ': , '

For several reasons the conclusions which can be drawn are

'tentative and, perhaps, 'in 86me cases speculative. Some

reasons are:-

(i) the 'sample 'tested in the Pilot Test (68) was 'small 'enough ,"

to merit:caution when interpreting'the results;

(ii) the,schools involved 'co-operated '''by chance", but they were

not chosen with any precautions 'taken to try to ensure the

randomness:of the candidate sample;

(iii)'the'test, being constructed by one :writer, was : bound to '

contain ,faults many of which became lappa rent after it had been

used (there was no pre-test); ,

(iV) lack of familiarity with the rubric of the test clearly

caused some difficulties to the candidates and'reduced the'

amount of time available for the actual problems;

(v) some, candidates, when CJ.uestioned afterwards, admitted that

they found the 'prospect of 45 items in It hours much too

formidable a task and, in something ofa panic, they tended ,to

jump from oneitem'to'another at the first sign of difficulty.

Despite the faults of this particular test, the rank

correlation between it and the 'A' level examination is

sufficiently high to, suggest that the abilities reCJ.uiredin,the

objective test are 'concerned with the same traits as those which

are involved with the abilities ,used in the' 'A' level examination.

This is not to say that the abilities 'tested in the two

examinations are necessarily identical, but they either

considerably'overlap or have definite links. 'It is reasonable to

suppose that, give~a more perfected (and ire-tested) objective

test paper, the correlation would be higher. The combination of

the results of the two kinds of test appears to be well justified,

possibly/

73

possibly with the· objective. paper carrying as.lJluch weight as

each of the traditional style papers. In this case it is worth

considering increasing the time allowance for:the objective paper

from H· hours.,for45 items to at least 2t hours. This would.

help to ensure that the test was 'a "po'wer" ·test and 'not a "speed"

. ·test and would decrease the temptation .to resort to guesswork.

,It'is:recognised that the objective test requires much more

intensive ,thought than a traditional test, but ;thetime of ..

2{- hours ,for 18 year olds is .not likely to impose too much of 'an

examination strain.

In the same way that. equivalent successive tests of any kind'

can never be arranged in precisely the ,same conditions, 'pre-tests

are never likely:to'give a completely accurate picture of the' . '.

reactions of' the candidates, 'however co-operative they· are, but,

there will be sufficientevidencefor'the spot-light to be

thrown:on the more serious faults. It iS'not necessary; or

deSirable to use a pre~test to reject out of hand items which

appearltoo difficult. There is need to examine precisely why

an item is causing trouble before it is removed from the test.

The lack of the ability 'concerned may be just that weakness which

it is. desirable to pin-point.

One clear 'conclusion is that objective tests are better .

produced by small groups or teams rather. than by an individual!

However, one question whlch,.'lremains completely unanswered is'

whether objective tests at this level are better predictors of

future mathematical ability; Correlation between objective and

traditional papers could merely indicate that the former are as

unreliable as the latter, though it is fair to say that

traditional papers, which'are developing today in scope, style

. and content as curricula develop, have not yet been proved to

'be seriously wanting in predictability.

i3IBLIOGRAPHY

ATHERE'OLD, F.J.and.LAWRA:NCE; A.E. Handbook on Objective Testing - Mathematics

Methuen Educational, 1973

BLOOM, B.S. et aI, Taxonomy of· Educatiomil Objectives:. the Classification of

. Educational Goals~ Handbook I: Cognitive Domain Longmans, 1956 .

EDUCATIONAL TESTING SERVICE, PRINCETON, NEW JERSEY multiple Choice Questions - A Close Look, 1963·

. FRASER, W .G •. and GILLAM, J. N. The Principles of Objective Testing inlllathematics

He inemann , 1972

GUILFORD,. J. P. and FRUCHTER, B. Fundamental Statistics in Psychology and Education, 5th Ed.

. McGraw-Hill, 1973

74

INCORPORATED ASSOCIATION OF ASSISTANT MASTERS IN SECONDARY SCHOOLS The Teaching of Mathematics

. Cambridge University Press, 1957

PIDGEON, D. and YATES, A. An Introduction to Educational Measurement

Routledge and Regan Paul, 1969

UNIVERSITY OF LONDON·· G,C,E. EXAMINATION Mathematics Syllabus C, Ordinary level, Teacher's Booklet (for multiple choice objective test)

Uni versi ty of London, 1970

WE IT ZMAN , R. A. Ideal Multiple Choice Items

WILSON, N.··

Journal American Statistical Association, 65, 71-89, 1970

·Objective Tests and Mathematical Learning, 2nd Ed. Oliver and.Boyd, 1974 .

WOOD, R. Objectives in the Teaching of~futhematics

Educational Research (NFER);· Vol. 10, No • . 2, 83-98, Feb. 1968

75 APPBNTIIX A

. 1.·. Standard deviation of an item

no. of candidates answering the item correctly.

= no. of candidates attempting the item.

sum of the total scores of candidates answering the item correctly •

. p is· defined as ~ i.e. the proportion of."attempters" HA

who are correct with this item.

M = mean total score of correct "attempters" p

"

q.is defined as 1 - p ,.

Then for an item, the mean scores on that . . N 1

item = p x NA

Deviations of correct candidates on this item = 1 - p and deviations of ,vrong candidates = p • •

• • Sum of squares of deviations on the

,! ..

= Hp (1_P)2 + (NA-Np )p2

= Np - 2pNp +. N p2 .A

Standard deviation of the item

_ INp~2P:!+ NAP2

= ~p _ 2p2 +p2

= jp(kp)

= .,rw

item

2. Point Biserial Coefficient of an item.

= p.

The point biserial coefficient is a product moment coefficient.

Starting with the Pearson correlation coefficient = fx~ (i) . . . . . "i y

where x,y are the corresponding deviations from the means of the two variables, N is the total number of each variable andcr"x,<r"y are .the respective standard deviations, and using the abbreviations:-

76

X is the continuous variable, i.e. the total paper score of a candidate who attempted the item,

Y is the genuine dichotomy, with point values of 0 and +1, i.e. the score on the item of a candidate,

NA is the total number of candidates who attempted the item, Mp is the meEln of the total paper scores of the candidates who

who answered the item correctly, N is the number

p N of candidates who answered the item correctly,

P =..J2 and q = NA

I-p,

MA is"the mean of the total scores of the candidates who attempted the item,

r.y id) the mean of the scores on the item, which has been to be equal to p,

tr x is the standard deviation of the X values, and

tS"y is the standard deviation of the point values, which, shown in 0) is equal to .;pq.

Then in (i)

(Y is 0 or +1)

so that ~xy = 2:.(X - MA)(Y - My)

=~ - M~Y - ~ +NAMA~

but ~X = NAMA and :£Y = NA~

then :;.xy = J:.ll - N AlIlAMy

r =~ll - NAlIlAMy NA ()x (f" y

shown

as

in

where ~XY = NpMp since Y is 0 or +1 and L.lrr is then the the sum of the overall scores of the successful "attempters".

Also NAMA~ = NAMAP = NpMA and try = ;pg: r = NpMp - NpMA =

NA cr"A {W =(M~AMA) ~

---------APPENDIX B

i' I1 r N h'J!.r 1 ~I!B. (19'75) ,'i':"LEVEL PAPERS

Paper I

Section (1) (Half questions)

77

SI Evaluate the following, Significant figures:'

'giving'each answer correct to two

20 .' n (i) z= '.(1.1) ,

n=l

. 20 (ii) ~

n=l . .

S2 . Write down the expansion of (l+y)n in ascending powers. of. y, giving the first four terms. Given that x is so small that its cube and higher powers.are negligible compared with unity, find the constants a, band c in the approximate formula .

. . ' ...... (8~~~)t ~ a +bx + cx2•

S3 Establish the formula logba '= logca

log 11 c . .. , Find the first two terms in the expansion of 10gI0(10:t;;lt) in

ascending powers of x, and state the coefficient of r-. 84: Determine all pairs of values of the real numbers p,q for which I +i is a root of the equation

S5 The

32' Z + pz + qz - pq = O.

complex numbers

z~ - zl z2

zl and z2 satisfy the equation

+ z2 = O. , I

, Find the ratio z2/z" given that its imaginary part is positive. If zl =a+ ib, where a and b are real, show. that

~2=,-i;(a-bl3) + t(b+aj3)i.

In an Argand diagram, thepO:Lnts Pand Q represent zl and z2' respectively, and 0 is the origin; ,ShOW that the trIangle

.OPQ is equilateral. .

S6 Find, to the nearest minute, the acute angle a for which 4 cos 8':' 3 sine.::: 5 cos (e+a).·

Oalcula te the values ore in the interval -1800 ~ ()~ 1800 for' which the function

f(O) = 4: cos9- 3sin~:- 4: attains its greatest value, its least value and the value zero.

S? In the triangle ABO,.X, Y and Z are the mid-:-points of BC, CA and AB respectively •. Using the cosine rule orotherwise, prove that b2 +c2 = 2AX2 + 2BX2. Write down two other similar results, and hence' show that

~2 + By2 + CZ2 ~ ~(a2 + b2 + c2).

S8 Find all the values of e in the range 0 £. B ~ 2tr for which

., " sine'+' sin' 3C' = case +' cos 3 e. S9 'Prove that ,d .-1

, dx s~n' x = 1

:.:1..' (1_xL)2

Given that the variables x sin-l 2x +

andy satisfy the e~uation -1 -1(" ' sin y+ sin xy) = 0,

find dy/dx when x'= y = O. · (Note. 'Each of the inverse sine functions is defined as having·

i tsvalue in the range -~'iI' to in ~)

SlO find

With the help of a suitable substitution, . ,{eX+e 2X . " , dx •.

" ,l+e 2x .

. ' ,

Section' £2) '(Full ~uestions)

or otherwise,

Dl1 Find the x-coordinate of the turning point of the curve whose.e~uation is a

Y = - + log x . x e

· where x> 0 and a> 0, and determine whether ,this turning point is a maximu.m:or a minimu.m.Deduce the range of values of the constant 'a for which y~O for all x >0 •..

In the' case when a. = 1, find .the 'area and the x-coordinate of' the centroid of the regionb'ounded by the curve, the x-axis and the ordinates x = 1 and x = 2. Express both answers in terms of log~2.

e Dl2 A: curve jOining the pOints (0,1) and (0, -1) is represented' . parametrically by the e~uations .

. x = Sine, y =(1 + sin 0) cose " ", where 0~e41T . Find dy/dx in terms ofe, and' determinethex,y coordinates of the pOints on the curve at which the tangents are parallel to the x-axis and of the point at which the tangent is perpendicular to the x-axis. Sketch the curve.

· The region in. the ~uadrantx). 0, y ~ 0' bounded by the curve and . the coordinate. axes. is rotated about the. x-axis through an angle of 2tf. Show that the volume swept out is given by

V =lIf1 . (l+x)2(1~x2)dX. " 0

Evaluate V, ,leaving your result in terms·ofV. .' . ~ .

Dl3 'The l1ne , of ,gradient m(IO) through the 2Point A(a,O) isa tangent to the rectangular hyperbola xy = c at the point P. Find m in~ermsof aandc, and show that the coordinates of P are (ta,2c/a) ... · . '.' .

. . . .

The line through A parallel to the y-axis meets the hyperbola at · Q, and the line joining Q to the origin 0 intersects APat R.

Given that OQ and AP are perpendicular to each other, find the' numerical value ofc2/a2 and the numerical value of the ratio AR: RP.

79

, 1)14- 'The point p(a cos e, b sin e)' on the ellipse El whose equation is

is joined to ,the point A(a, 0), and M is the mid-point of AP. Find the 'cartesian equation, of the locus of M as e, varies. By a suitable change 'of origin, or otherwise,show that the locus is an;ellipseE" and 'find the coordinates of its centre and the lerigths'~f'its semi-major and 'semi-minor axes. Sketch 'El andE2 on one ' d~agram. , ',,', ',," ' "

Show that the equation of the tangent to ellipse E2 at M is'

"ay sine + bx cose' =tab(l + cose ),

and verify,that this tangent is 'parallel to the tangent to,' El. at P. Find the distance 'between the two tangents 'in terms or a in the case when b = t aJ3 and e = {tT.

Paper~II ,'"

Section I (1)' (Half' questions)

SI Three particles Ai B and 0 with masses lOm, 5m,'3m respectively, lie,'at'rest in a straight line on a smooth horizontal table. The particle A is projected 'horizontally , with sp§led u and undergoes a direct perfectly elastic collision' with B. Show that B acquires speed 4-u/3. ':' , , '

SUbsequently B is in direct perfectly' elastic rollision,wi th' O. Show thati=ediately after this collision the velocities, of A andB are' equal. ' " ,

S2 The ends of a lieht inextensible string-ABO 'of length 31 are attached to fixed pOints A and 0, 0 being vertically below A at a distance J31 from A. ' ,At a distance 21 along the string from A a particle B of mass ill is attached. When both portions of the string 'are taut,B'is given a horizontal veloci tyujandthen ' continues to move 'in a circle ,with

21

" C __ -~l.=---_-4.B

constant 'speed., Find the tensions in' the' two portions of the" string 'and show that the motion is possible only if '

2 ' .! "I u ~ 3 g v3;

S3 A uniform wire of weightW, bent 'into a semicircular arc, has: a particle of weight W attached .to one end and 'is 'suspended freely 'from the,' other end. 'Show that when the' system' is' in :equilibrium' the~ftraight line through the ends of the wire makes an angle tan (}rr/2) with the horizontal. ' ", " " • ",'

, , (The centre of ,mass of a uniform, semicircular arc of wire is at a, distance 2a/tT from' the centre, of ,the 'circle of -radius a of which it ,forms a part.)

80

S4 Arigid body is in' e'l.uilibriurri 'under 'the action cif three non-parallel coplanar forces.', Show that the lines' of action of the forces 'are "concurrent. , , ' Two smooth 'inclined planes 'are 'fixed ,perpendicular to each other with their 'line of, intersection 'horizontal: A uniform 'rod is in equilibrium'perpendicular to the 'line of intersection and with one end resting on each plane. 'ShOW that the 'mid-point of the rod is 'vertically above the line of intersection of the planes.

Section' (2 ), (FUll 'l.uestioris)

D5 A :bowl in:ihe;form of the .' surface generated by rotation of i ,

the parabola x = 4ay about its axis, which is vertical, is being 'filled 'with water at a constant rate r' ("rate"is 'volume per unit time). The height of the' water leveL above the vertex' o of the parabola is yat time t., Show that

EL dt

. , I = r.

41Ta y .. ! : , '

and3

deduce : that the time .. taken fory to increase ·froma ,to 2ais. ' Gtfa fr. " When-y= '2athe water supply is cutoff and a small plug at 0 is removed to allow the water to escape. The rate of 'escape , depends onyand'ise'l.ual to k,fy, where .kis'constant. Find the time taken for y to decrease from 2a to a.

I

Show that the two times are e'l.ual if k= ~(2i - l)a-ir;: ".

• I

D6 An' elastic string of natural length 2a and modulus Ahas its ends attached ,to two ·pointsA, Ban a smooth horizontal table. The distance AB is 4a,and Cis the mid-point:of AB. A 'particle of mass m is attached ,to the mid-point of the string. The particle is 'released from rest at D,. the mid-point :of CB •. Denotingbyx the,displacement of the particle from C,show that the equation of motion of the particle is

d2 '2~' , -..! +' -x = O. i

, dt 2 ma

, Find the maxillIUm speed taken'for the particle

of'the,particle and show that the time to move from D directly to the 'mid-point

:1. ,of CD is j(~r .i

(Standard fOrllIUlae for simple harmonic motion may be quoted'" without proof. )' ,

D7 The resistance to themotionofa lorry of mass m is kv at speed v, where k is a constant. (i) Find ,the acceleration of the lorry when it is climbing a hill,d'inclination a to the horizontal at speed v under full

,power S. Given that it can climb the same hill under full power at a steady speed u, show that "

2 . .'. ku =S - mgu s~n a.

D7 (H )I:rhe! lorry can traveia:t' steady' speed 1'1 on a level road undsr. full power and the same law of resistance. Show that the time taken for it to accelerate under full power from speed vI to speed v 2 (where Y l <.. v 2 <1'1) on the level road is

2 - 2 2 ~.log 1'1 VI ". 2S e w2 v 2 , 2

" ,

DB A ship A whose full speed is 40 kilometres per hour is

81

20 kilometres due west of a ship B which is travelling uniformly with speed 30 kilometres per hour in a direction due north. The ship A travels, at full speed on a course, chosen so ,as to interceptBas soon as possible. Find the direction of this course and calculate to the nearest minute the time A WOUld' take to reach B.,

'V/hen half of this time has elapsed the ship A has engine failure and thereafter proceeds at half speed. Find the course which A should then set in order to approach as close as possible to B, and 'calculate the distance of closest approach (in kilometres to 2 decimal places).

D9A point o lies on a horizontal plane, and the point A is at a height hvertically above O. A particle is projected from A with speed V at an angle a above the horizontal. Taking 0 as the origin and Oy vertically upwards, show that the equation of the path of the particle can be written in the form

, " 2 2 Y = h '~ gx '+ x tan a - ~ tan2a.

2V2 2V2

Thepa2ticle hits the plane at the point B(r,O). In the case when V = ghderive a quadratic equation for tan a in terms of hand r, and show that r~/3h. For the same value of V show that r = /3h when a = 30 .,

DIO A particle P of mass m, is at rest at a point' A on the smooth outer surface ofa fixed sphere ,P

'of centre 0 and radius a, OA being horizontal. ' The particle is attached to one end of a light inextensible string which is taut and passes

',over the sphere, its other end carrying a par~icle Qof mass 2m which hangs freely. The string lies in the vertical plane containing Q. OA; The -system is released from rest, and after time t the angle POA is e ,as shown in the diagram. Show that, while P 'remains in. contact with the sphere,

3a(~~I = 2g(28 - Sine ).

Find the tension in 'the string and the reaction of the sphere -on Pin terms of m, g and e .

Dll A uniform rod AB of length a and weight W is smoothly jointed at Bto a uniform rod. BC of length aJ2 and also of weight w • . The system is in equilibrium with C resting on a rough horizontal floor at a distance. 2a from a rough .... vertical wall, A resting against the' wall, and AB horizontal at a height a above the floor. . .The plane of the rods is perpendicular to 'the floor.' and the wall.

A I-__ ~'(J:B

a

(i) Find' the frictional force, exerted by the wall atA.

'(ii) Find the vertical rmd.ho,rizontal components of the force exerted' on BC- at C"

82

(iii) Show thatequilihrium is possible only if the coefficient of friction atC is at least 2/3. ,

(iv)· Find the least, value of the coefficient of friction at A for equilibrium to be possible. '

•

,

I