SpeakingSpeakingGraham HyattGraham Hyatt

Referent für EnglischReferent für Englisch

Referat QualitätssicherungReferat Qualitätssicherung

Ministerium für Bildung, Wissenschaft und Kultur, Ministerium für Bildung, Wissenschaft und Kultur, Mecklenburg-VorpommernMecklenburg-Vorpommern

based on a presentation bybased on a presentation by

Dr. Rita GreenDr. Rita Green

Test Development Training & Analysis Ltd.Test Development Training & Analysis Ltd.

www.testdevelopmenttrainingandanalysis.comwww.testdevelopmenttrainingandanalysis.com

OutlineOutline• Characteristics and types of “Speaking”Characteristics and types of “Speaking”• What should our learners be able to What should our learners be able to

do?do?– The “Standards” and the CEFThe “Standards” and the CEF

• What do we want to test and how?What do we want to test and how?– Test specificationsTest specifications– Examples of good practiceExamples of good practice– What makes a good interlocutorWhat makes a good interlocutor– Writing tasksWriting tasks– Assessing candidate performanceAssessing candidate performance

• Why do we find relatively few Why do we find relatively few speaking/oral tests/exams in Germany?speaking/oral tests/exams in Germany?

time-consumingtime-consuming

difficult to administerdifficult to administer

costly in terms of manpowercostly in terms of manpower

not easy to set appropriate not easy to set appropriate taskstasks

problems of assessment – problems of assessment – What? How?What? How?

Characteristics of Characteristics of Speaking Speaking

• Takes place in real timeTakes place in real time• We use different language in different We use different language in different

situationssituations– Some of it is planned Some of it is planned – Some of it is unplanned/spontaneousSome of it is unplanned/spontaneous– Formal versus colloquialFormal versus colloquial

• Speaking is a social and (usually) Speaking is a social and (usually) interactive interactive

process and skillprocess and skillSpeaking is often dependent on listeningSpeaking is often dependent on listening

• Spoken language contains generic words e.g. “thing”, “thingy” etc.• Spoken language contains fillers e.g. “you know”, “you see”, “kind of”, “as it were” etc • it employs set phrases and expressions e.g “I thought you’d never ask”, “Let’s not go there”, “You must be joking”, …• The speaker needs to choose the appropriate language for a particular context

• Speakers hesitate, add pausesSpeakers hesitate, add pauses• Speakers need time to process both input Speakers need time to process both input and output (speed depends on proficiency, and output (speed depends on proficiency, familiarity, perceived accuracy familiarity, perceived accuracy

requirements and requirements and penalties)penalties)• ““small words/lubricationssmall words/lubrications”” are employed are employed

e.g. e.g. ““reallyreally””, , ““ohoh””, “well”, “I , “well”, “I mean”, mean”, [Hasselgren, [Hasselgren, 2004]“yes but …”2004]“yes but …”

• Scientific studies …Scientific studies …““support the case that the more small words a learner uses, support the case that the more small words a learner uses, the better their perceived fluency” (Luoma, Assessing the better their perceived fluency” (Luoma, Assessing Speaking, page 19)Speaking, page 19)

• Planned:Planned:– the speaker has prepared and practised a the speaker has prepared and practised a

speech or speech or presentation (at home / before the presentation (at home / before the test)test)

– pronunciation and intonation have been pronunciation and intonation have been practisedpractised

– often tends towards formality of lexis and syntaxoften tends towards formality of lexis and syntax

• UnplannedUnplanned– e.g. everyday conversation – spontaneous e.g. everyday conversation – spontaneous

utterances; short sense units, incomplete utterances; short sense units, incomplete sentencessentences

– pronunciation and intonation less controlledpronunciation and intonation less controlled– lexis and syntax tend to be informal/colloquiallexis and syntax tend to be informal/colloquial

SituationsSituations

• ChattingChatting– Conversation, small talk Conversation, small talk

• purpose: to establish and purpose: to establish and maintain contact;maintain contact;

• the atmosphere is friendly, the atmosphere is friendly, tendency to agreement tendency to agreement rather than rather than argumentargument

• often found at the beginning of often found at the beginning of speaking tests i.e. speaking tests i.e. the warm-up the warm-up (Should the warm-up be assessed?)(Should the warm-up be assessed?)

Situations (2)Situations (2)

• Transmission / exchange of information on Transmission / exchange of information on a particular topic / transactionala particular topic / transactional– aim: to communicate a specific piece of aim: to communicate a specific piece of

information or information or statement successfully i.e. so statement successfully i.e. so that the “message” is that the “message” is understoodunderstood

– characteristics: establishing a common basis of characteristics: establishing a common basis of conversation, conversation, exchanging information in exchanging information in easily manageable and easily manageable and comprehensible comprehensible ““bites”; bites”; information is presented in a information is presented in a logical logical sequence; questions and answers, repetition sequence; questions and answers, repetition and and checking whether the message has been checking whether the message has been understoodunderstood

What we need to consider What we need to consider when developing speaking when developing speaking

tasks/tests.(1):tasks/tests.(1):• The purpose of the test:The purpose of the test:

– diagnostic (diagnostic (using a test to determine the using a test to determine the strengths and strengths and weaknesses of a test taker or test weaknesses of a test taker or test takerstakers / establishing the / establishing the level the test-taker level the test-taker has reached to determine future needs)has reached to determine future needs)

– assessing performance and awarding gradesassessing performance and awarding grades (classroom situation / year 10 examination?)(classroom situation / year 10 examination?)– placement (assigning to particular courses,placement (assigning to particular courses, admission to university)admission to university)– proficiency proficiency (determining the level of a test taker (determining the level of a test taker

against a against a descriptor of performance such as the descriptor of performance such as the CEFR)CEFR)

What we need to consider What we need to consider when developing speaking when developing speaking

tasks/tests.(2):tasks/tests.(2):• functions (please refer to the CEF):functions (please refer to the CEF):

– expressingexpressing (thanks, advice, opinions, (thanks, advice, opinions, information, information, apologies, wants/needs, apologies, wants/needs, complaints, attitudes,complaints, attitudes,

ideas, reasons, preferences)ideas, reasons, preferences)– requestingrequesting (advice, information, (advice, information,

permission, permission, reasons)reasons)– describing describing (people, places, objects)(people, places, objects)– comparing/contrastingcomparing/contrasting (people, (people,

places, objects, places, objects, situations)situations)– narratingnarrating (stories, events) (stories, events)

– agreeing/disagreeingagreeing/disagreeing– accepting / refusingaccepting / refusing

– persuading, warning, making persuading, warning, making suggestionssuggestions

– directing, ordering, giving permissiondirecting, ordering, giving permission– analysing, criticising, predicting, analysing, criticising, predicting,

arguing for/against, summarisingarguing for/against, summarising

• OutputOutput

The test taker should be able to:The test taker should be able to:– describe a picturedescribe a picture– give a presentationgive a presentation– make a speechmake a speech– initiate / maintain a conversationinitiate / maintain a conversation– negotiate a conclusionnegotiate a conclusion– give detailed directionsgive detailed directions– provide an explanationprovide an explanation– make an apology … [Banerjee, 2007]make an apology … [Banerjee, 2007]

CEFRCEFR• give a prepared talk and answer give a prepared talk and answer

follow-up questionsfollow-up questions• give straightforward descriptions, give straightforward descriptions,

reasons and explanations on a variety reasons and explanations on a variety of familiar subjectsof familiar subjects

• give short justifications and give short justifications and explanations for views, plans or explanations for views, plans or actionsactions

• give detailed accounts of experiences give detailed accounts of experiences describing feelings and reactionsdescribing feelings and reactions

• relate the plot of a book or film and relate the plot of a book or film and describe his/her reactionsdescribe his/her reactions

• narrate a simple story or eventnarrate a simple story or event

Factors which influence the Factors which influence the test-taker’s performancetest-taker’s performance

• Knowledge of the language – level of Knowledge of the language – level of competencecompetence• personal characteristics e.g. age, gender, personal characteristics e.g. age, gender, nationality, L1, education, preparation for nationality, L1, education, preparation for and familiarity with the test situation and and familiarity with the test situation and the test in generalthe test in general• general knowledgegeneral knowledge• emotional reaction to the taskemotional reaction to the task• interaction with other candidates (paired or group, interaction with other candidates (paired or group,

known / known / unknown, gender, level of proficiency of group unknown, gender, level of proficiency of group

members)members)• interaction with the interlocutorinteraction with the interlocutor

• Should there be one person who performs Should there be one person who performs two two

roles (interlocutor and assessor) or two roles (interlocutor and assessor) or two people?people?

• Personal characteristics of the interlocutor: Personal characteristics of the interlocutor:

gender, age, experience of testing, gender, age, experience of testing, accent, accent, speed of delivery, native v non-speed of delivery, native v non-native native speakerspeaker

• Inter-action with the test-taker(s): Inter-action with the test-taker(s): nature and scope – interlocutor framenature and scope – interlocutor frame

ExaminerExaminer

The tasksThe tasks

• level of difficulty /CEF levellevel of difficulty /CEF level– uni-level or multi-leveluni-level or multi-level

• task type: e.g. role-play, interview, task type: e.g. role-play, interview,

conversation, short presentation etc.conversation, short presentation etc.• material/stimuli: e.g. photographs, material/stimuli: e.g. photographs,

role-play cards,role-play cards,• length of the tasklength of the task

preparation timepreparation time

speaking timespeaking time

Tasks continuedTasks continued

• number of tasks within a test?number of tasks within a test?• instructions for the candidate instructions for the candidate andand the the

examiner / interlocutorexaminer / interlocutor

[Interlocutor Frame – essential [Interlocutor Frame – essential for fairness for fairness and standardisation]and standardisation]

• administration of the test:administration of the test:– planning, organisation, blockingplanning, organisation, blocking– will the test be recorded?will the test be recorded?– preparation, supervision, invigilation … ?preparation, supervision, invigilation … ?

Test Specifications Test Specifications and the and the

Common European Common European FrameworkFramework

Assessment of Speaking Assessment of Speaking Tasks/TestsTasks/Tests

Scales Scales • Holistic scalesHolistic scales = “ = “express an overall impression express an overall impression

of an examinee’s ability in one score” (Luoma, of an examinee’s ability in one score” (Luoma, pages 60/61)pages 60/61)

• Analytical scales =Analytical scales = assess a number of different assess a number of different aspects of the canidate’s performance e.g.aspects of the canidate’s performance e.g.– task fulfilment - content and communicationtask fulfilment - content and communication– pronunciationpronunciation– fluencyfluency– organisation (coherence/cohesion)organisation (coherence/cohesion)– grammar – range and accuracygrammar – range and accuracy– vocabulary - range and accuracyvocabulary - range and accuracy– interaction - e.g. with the interaction - e.g. with the

examiner/interlocutor, in pair or group tasksexaminer/interlocutor, in pair or group tasks

Speaking Assessment Speaking Assessment ScalesScales

Good PracticeGood Practice

careful test development procedure: careful test development procedure: constantly revised test specificationsconstantly revised test specifications

strictly controlled item production strictly controlled item production procedure (revision and editing of procedure (revision and editing of tasks)tasks)

piloting all tasks with studentspiloting all tasks with students training item-writerstraining item-writers the use of item-writer guidelinesthe use of item-writer guidelines

Good Practice continuedGood Practice continued

sstandardised examination procedure tandardised examination procedure (the use of(the use of an an Interlocutor Frame) Interlocutor Frame)

examiner training focusing on both examiner training focusing on both interlocutor & assessor traininginterlocutor & assessor training

the use of benchmarked the use of benchmarked performances in examiner trainingperformances in examiner training

Bibliography and Bibliography and SourcesSources

• Csépes, I & Együd, G., Csépes, I & Együd, G., Into Europe – Prepare Into Europe – Prepare for Modern for Modern English Exams, The Speaking English Exams, The Speaking HandbookHandbook, British , British Council – Teleki László Council – Teleki László Foundation (2003)Foundation (2003)

• Banerjee, J., Lancaster University, course Banerjee, J., Lancaster University, course Language Language Testing at LancasterTesting at Lancaster, 2007 , 2007

• Green, R., PowerPoint Presentation, Berlin, Green, R., PowerPoint Presentation, Berlin, 20082008

• Hasselgren, A., Hasselgren, A., Testing the Spoken English of Testing the Spoken English of Young Young NorwegiansNorwegians, CUP (2004), CUP (2004)

• Luoma, S., Luoma, S., Assessing SpeakingAssessing Speaking, CUP (2004), CUP (2004)• Schuh-Fricke, U., PowerPoint Presentation, Schuh-Fricke, U., PowerPoint Presentation,

Braunschweig, 2008Braunschweig, 2008

INTO EUROPEINTO EUROPESeries Editor: J. Charles AldersonSeries Editor: J. Charles Alderson

The Speaking HandbookThe Speaking HandbookIldikó Csépes & Györgyi EgyüdIldikó Csépes & Györgyi Együd

The The HandbookHandbook is accompanied by a 5-hour is accompanied by a 5-hour DVDDVD

Published by Teleki László Foundation Published by Teleki László Foundation & The British Council& The British CouncilDistributor: Libro TradeDistributor: Libro TradeInfo:Info: books@librotrade.hubooks@librotrade.hu

The The Into EuropeInto Europe series can now be downloaded from series can now be downloaded fromhttp://www.lancs.ac.uk/fass/projects/examreform/http://www.lancs.ac.uk/fass/projects/examreform/