Test design, test writing KAYSERI, 29-31 JANUARY 2014

Aims

!  Procedures for test design !  Writing specifications !  Writing tests !  Effective instructions !  Good tests: the key ingredients

Procedures for test design

Objectives Specifications Item writing

Item trialling Item revision Assessment scales

Validating Handbook Training

Defining our test objectives

1)  What kind of test is it going to be? 2)  What decisions do we need information for? 3)  What abilities are tested? 4)  How much detail do we need? 5)  How accurate should our results be? 6)  Is there any washback effect? 7)  What are the practical constraints?

Writing specifications

Content

Structure

Timing

Medium

Techniques

Pass / fail levels

Marking and scoring

Administration procedure

Writing tests

WHAT IS THE DIFFERENCE? !  no support for students ! clearer !  simple, unambiguous instructions !  students familiar with task format and expectations !  if possible, more than one method to test a skill !  validation

Test writing vs exercise writing

Test writing: where do you begin?

!  Your own specifications, of course! !  NOT past papers

!  NOT coursebook exercises !  NOT other people’s tests

!  NOT tests for completely different objectives

Test writing

specs texts items edit key

Test writing: feedback and editing

Effective instructions

Let’s look at an example of test instructions.

Read the article about the future of electric cars. Then read the statements carefully.

Find the information about the statements in the article, and write the number of the paragraph (1-6) which contains the information.

There are statements which are confirmed by information in the article. Circle a for ‘true’ for these. There are statements which say something that is not confirmed by the article. Circle b for ‘false’ for these. There are statements about which there is information in the article. Circle c for ‘not given’ for these.

Effective instructions

example from the draft syllabus for a B1-level secondary school-leaving examination

Effective instructions

Which five of the following adjectives would you use to describe effective test instructions?

detailed

clear

familiar

brief

challenging

unambiguous

authentic

simple

!  simple, brief, clear and unambiguous !  should not test comprehension of instructions

!  use Turkish if you can’t make it simple and brief

!  that’s four adjectives - one more?

Effective instructions

!  simple, brief, clear and unambiguous !  should not test comprehension of instructions

!  use Turkish if you can’t make it simple and brief

!  task should be familiar to students !  use standardised instructions during practice

Effective instructions

Familiar?

extract from the Hungarian school-leaving examination 2009

!  simple, brief, clear and unambiguous !  should not test comprehension of instructions

!  use Turkish if you can’t make it simple and brief

!  task should be familiar to students !  use standardised instructions during practice

!  include an example where necessary ! with less familiar formats

! where students know different varieties of format

! where there is a danger of misunderstanding

Effective instructions

What’s the task here?

Read the sentence and decide which of the options: a, b, c, or d cannot be used to complete it.

Tourist can access the site of the castle by …

a) ferry from the sea terminal.

b) the regular local bus service.

c) foot across the bridge.

d) hired fishing boats from the village.

Will learners realise this in time?

!  simple, brief, clear and unambiguous !  should not test comprehension of instructions

!  use Turkish if you can’t make it simple and brief

!  task should be familiar to students !  use standardised instructions during practice

!  include an example where necessary ! with less familiar formats

! where students know different varieties of format

! where there is a danger of misunderstanding !  use no terminology if possible

Effective instructions

TO SUM UP !  simple, brief, clear and unambiguous !  familiar !  example !  no terminology AND CHECK your instructions with a colleague!

Effective instructions

GOOD TESTS

clear aims

thorough specs

items that work

effective instructions

useful feedback

balanced assessment

Thank you!

zoltan.rezmuves@digimedia.hu

consonantvoiced.blogspot.com