Mobile Phones in Language Education

Using mobile phones for the assessment of and for oral skills development in

secondary education.

Ton Koenraad

Hogeschool Utrecht, University of Applied Sciences, Faculty of Education

TELLConsult

Overview

Project context & goals

Pre-Pilot: set-up & research questions

Data collection

Results & conclusions

Next phase

English for Kids Foundation

“Voice for children”

A, not -for-profit organisation

" Voice for children "

Every kid has a right to communicate.

English is a good tool to communicate internationally.

The English for Kids Foundation wants to promote the use of English

to communicate internationally.

We want to give children 'a voice'.

Projects are to meet the

following requirements:

> long term target (3-5 yrs)

> aimed at children

> close co-operation with

local organisations in the

project countries so that

activities can be embedded

and be allowed to proceed

EFKF projects:

India

Nicaragua Very Young

Learners

South Africa Elandsdoorn

Kenia Teacher support

Surinam EDS

Gobabis Namibia

Cambodia

www.efkf.org

www.englishforkidsfoundation.org

Anglia Network Europe www.anglianetwork.eu

15 European countries

International context

Step by step

- 28 January - 15 April - 20 May - 24 June - Speaking Tests

Examinations

Why this Project?

Anglia: flexibility / assessment of - sharing innovative speaking practice materials - delivery of formal language assessments

English for Kids: mobile as infrastructure, & schools assessment for: washback effect of testing

Issues to be researched

general organisation

task and test design,

teacher competences

face & content validity aspects

system usability

Research design / instrument development

Why use a phone?

Natural interface. Widely available. Familiar to students. Technology used as a tool – not “accessorizing education”. Also available on iPod Touch & Skype.

20 20

How it works

Teachers:

Personal online workspace to set spoken

questions/tasks.

Questions are easy to set online by using a

microphone or uploading audio files.

Audio player allows teachers to review spoken

work and leave feedback.

Over time teachers establish a digital portfolio

of student work.

21

How it works

Students:

Connect using:

Mobile phones

iPod Touch

Skype

Landline

Computer

Access spoken exercises & leave voice responses.

Connect with other students for role play.

Personal online workspace to store work, listen & receive

feedback.

Listen to exemplar and sample questions posted by teachers.

22

Pilot

2 secondary ed. EFL teachers

2 Anglia member schools

Volunteer students (n= 20)

Assessment: asynchronous, interview format

Oral presentation skills

Data Collection

Instruments Pupils Teachers

Pre-Questionnaire

+ +

Recordings + Test scores

+ +

Post-Questionnaire

+

Structured Interview

N=2 +

Reflections Developers

Pre-Questionnaire: Learner Profiles (1)

Aspect Group A Group B

Group size 12 8

Girls 5 4

Boys 7 4

Avg. Age 13.5 13.5

Years of English 3 3

Attitude to Learning English

Fairly positive

Positive

Average score at Secondary

6.37

7.6

Pre-Questionnaire: Learner Profiles (2)

Aspect Group A Group B

Speaking Skills: (Self reported)

Fairly good Good

Like speaking in class

So, so Definitely

Actual speaking hardly Very frequently

Telecollaboration at school

n/a Slightly more than once

Tel. Experience in projects

n/a

Very occasionally

Tel. Experience IRL

seldom seldom

Pre-Questionnaire: Learner perceptions: L2 in class & IRL

Aspect Group A Group B

L2 in lessons Once in 3 lessons

Practically every lesson

Answers /Discussion Only now and then

(very) frequently

Pairwork Hardly ever sometimes

Use of English IRL

1. Chat in games

2. Holidays 3. Skype

1. Holidays, 2. Chat in

games 3. Txt chat

Assessment of Oral skills

Aspect Group A Group B

Assessed Tasks in 2010

All: 1 All: 3.5

School reports: Oral skills included?

35% ? 70% ?

Expectations: Is tele-testing valid?

Yes: 35% ?: 55%

Yes: 60% ?: 40 %

Post: Valid Yes: 60% 40%

Topics

Introduction/warming up

Your holiday this year

A good school

Social networks

A million euros

The climate

Post-Questionnaire (1)

Aspect Group A Group B

Technically OK? Yes So, so

Read Instructions

Yes Sure

Different from expectation

Yes Yes

Questions: complexity,

speed, loudness,

Hard to remember,

Speed bit fast, Not loud enough

Idem, but Speed OK

Post-Questionnaire (2)

Aspect Group A Group B

Answer time left Yes Yes

Expected Mark Just sufficient O.K

Problem Topics Networks; 1M Euros Good School; Climate

Test Location School School + home

O.K. to do Yes, quite Yes

More pleasant without teacher

No No

Pupils’ Comments

Time constraint is unnatural

Was interrupted: new session needed

Retries: worries about costs

Questions could be louder

Questions: peer voice is more inviting

Teacher Perceptions

System usability -System: fairly user-friendly

Topics - More alignment with pupils‟ interests might be needed

Validity -Content measured in time is less suitable as criterion when no interaction is possible - computer-based testing, as such, not perceived as unusual or unfriendly. - Retry option?

Reviewing work online

34

Teacher Perceptions

Teacher competencies: - Knowledge of CEFR -> difficult, training / practice needed - Evaluation categories (content, accuracy, complexity, fluency.) useful; scoring doable in one session. But …would prefer a grading scale that results in a CEF-level: better match to Dutch current grade system

Teacher perceptions

Implementation - Use as practice material and preparation for speaking test. Actual testing: rather face-to-face - Gives students the opportunity to practice outside the classroom, extra practice - Chances for providing individualised feedback - May help reduce anxiety of insecure & shy students:

Conclusions / next steps

Improve briefing (demo, online tutorial) + raise awareness implications of re-tries

Redesign questions (granularity)

Try-out alternatives: - system access (landlines, computers) - content aligned to syllabus / textbook

Conclusions

Pupils, teachers & management have concerns about costs

Also found in other projects: […] cost to the end user is a major consideration and can be a barrier to successful uptake when using mobile devices (Kukulska-Hulme & Shield, 2007)

Thank you for your attention.

Comments, Questions?

www.koenraad.info

Ton.Koenraad@gmail.com

Annex

Literature Selection Research data

Learnosity

Literature selection

Collins, T. (2005). „English Class on the air: Mobile Language Learning with CellPhones‟, Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT‟05).

http://csdl2.computer.org/comp/proceedings/icalt/2005/2338/00/23380402.pdf

Fallahkair, S., Pemberton, L. & Griffiths, R. 2007. „Development of a cross-platform ubiquitous language learning service via mobile phone and interactive television‟. Journal of Computer Assisted Learning, 23 (4), 312-325.

Kukulska-Hulme, Agnes; Sharples, Mike; Milrad, Marcelo; Arnedillo-Sanchez, Inmaculada and Vavoula, Giasemi (2009). Innovation in Mobile Learning: A European Perspective. International Journal of Mobile and Blended Learning, 1(1), pp. 13–35.

Literature selection (2)

Naismith, L., Lonsdale, P., Vavoula, G. & Sharples, M. (2004). „Literature Review in Mobile Technologies and Learning‟. FutureLab Report 11. http://www.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative, University of Nottingham, UK.

Shield, Lesley and Kukulska-Hulme, Agnes (2008). Special issue of ReCALL on Mobile Assisted Language Learning. Cambridge University Press.

Thornton, P. & Houser, C. (2005). „Using mobile phones in English education in Japan‟. Journal of Computer Assisted Learning, 21, (3): 217-228.

Pre-Questionnaire: Learner Profiles (1)

Aspect Group A Group B

Group size 12 8

Girls 5 4

Boys 7 4

Years of English 3 3

Positive Attitude to Learning English

2.42 STD: 0.51

3.11 STD: 0.33

Average score at Secondary

6.37 STD: 0.9

7.6 STD: 1.7

Speaking Skills: (Self reported)

6.58 STD: 0.9

7.55 STD 0.68

Pre-Questionnaire: Learner Profiles (2)

Aspect Group A Group B

Like speaking in class

2.67 STD: 0.78

3.78 0.44

Actual speaking 1.5 STD: 0.52

3.56 STD: 0.53

Telecollaboration at school

n/a 2.0 0.0

Tel. Experience in projects

n/a

1.38 STD: 1.0

Tel. Experience IRL

1.83 STD: 1.19

1.44 STD: 0.73

Pre-Questionnaire: Learner perceptions: Oral L2 in class

Aspect Group A Group B

L2 in lessons 2.1 STD: 0.50

5.0 0.0

Answers /Discussion 2.2 / 1.6 3.9 / 3.2

Pairwork 1.6 1.9

English IRL

1. Chat in games

2. Holidays 3. Skype

1. Holidays, 2. Chat in

games 3. Txt chat

Assessment of Oral skills

Aspect Group A Group B

Assessed Tasks in 2010

All: 1 All: 3.5

Formal reports: Oral skills included?

35% ? 70% ?

Is tele-testing valid?

Yes: 35% ? : 55%

Yes: 60% ?: 40 %

Post: Valid Yes: 60% 40%

Post-Questionnaire (1)

Aspect Group A Group B

Likert scale Disagree 1 Agree 4

Technically OK? 2.43 STD: 1.13

1.8 0.84

Read Instructions

2.8 STD: 0.7

3.0 0.7

Different from expectation

2.7 STD: 0.76

2.8 1.3

Questions: complex, speed,

loudness,

Not loud enough Hard to

remember

idem Speed OK

Post-Questionnaire (2)

Aspect Group A Group B

Likert scale Disagree 1 Agree 4

Answer time left 3.4 STD: 0.5

3.2 1.1

Expected Mark Just sufficient STD: 1.4

O.K 1.1

Problem Topics Networks; 1M Euros Good School; Climate

Fun to do 2.8 STD: 0.4

2.4 0.9

More pleasant without teacher

2.07 STD: 0.6

2.0 1.0

www.learnosity.com

Twitter @learnosity

Speak. Listen. Learn.