PRINCIPLES OF CALL

EVALUATIONTSL 641

Chapelle, C. (2001). Computer Applications in SLA. UK: Cambridge

1

PRINCIPLES FOR CALL

EVALUATION2

Three needs must be addressed when

evaluating CALL materials:

Materials include

Courseware – CD ROMs etc

Websites – ESL website

Mobile Apps -

Tasks & – assignments including use of WWW, CMC

Activities – Chat, Blog, Podcast, etc

PRINCIPLES FOR CALL

EVALUATION3

Evaluation criteria should incorporate

findings and theory-based speculation about

ideal conditions for SLA

Findings from research on SLA

Theories of SLA

PRINCIPLES FOR CALL

EVALUATION

Criteria should be

accompanied by

guidance as to

how they should

be used i.e. a

theory of

evaluation needs

to be articulated

4

PRINCIPLES FOR CALL

EVALUATION5

Both criteria and theory need to apply not only

to software, but also to the task that the

teacher plans and that the learner carries out

PRINCIPLES FOR CALL

EVALUATION6

According to Chapelle (2001)

5 principles of CALL evaluation criteria

Evaluation of CALL is a situation-specific argument.

CALL should be evaluated through two perspectives: judgmental analysis of software and planned tasks, and empirical analysis of

learners’ performance.

Criteria for CALL task quality should come from theory and research on instructed SLA.

Criteria should be applied in view of the purpose of the task.

Language learning potential should be the central criterion (most critical) in evaluation of CALL.

CALL EVALUATION:

PRINCIPLE 17

Evaluation of CALL is a situation-specific argument.

The outcome of task evaluation for any L2 tasks including those for CALL cannot be adecision about effectiveness.

Instead, an evaluation has to result in an argument indicating in what ways a particularCALL task is appropriate for particular learners at a given time, the use at a particularsetting

Implication: CALL developers need to be familiar with criteria for evaluation whichshould be applied relative to a particular context.

CALL EVALUATION:

PRINCIPLE 28

CALL should be evaluated through two perspectives: judgmentalanalysis of software and planned tasks, and empirical analysis oflearners’ performance.

Judgmental analyses - examine characteristics of the software andtask in terms of criteria drawn from research on SLA.

Empirical analyses - address the same criteria but throughdata gathered to reveal the details of CALL use and learningoutcomes.

Implication: CALL developers need to be familiar with criteriafor evaluation which should be applied relative to a particularcontext.

LEVELS OF ANALYSIS FOR CALL EVALUATION

9

Level of

Analysis

Object of Evaluation Example Question Method of

Evaluation

1 CALL software

Does the software provide

learners the opportunity for

interactional modifications to

negotiate meaning?

Judgmental

2 Teacher-planned

CALL activities

Does the CALL activity

designed by the teacher

provide learners the

opportunity to modify

interaction for negotiation of

meaning?

Judgmental

3

Learners’

performance

during CALL

activities

Do learners actually interact

and negotiate meaning while

they are working in a chat

room?

Empirical

CALL EVALUATION:

PRINCIPLE 310

Criteria for CALL task quality should come from theory and research on instructed SLA.

Language learning potential

Learner fit

Meaning focus

Authenticity

Positive impact

Practicality

Implication: CALL evaluators need to keep up with and make links to research on instructed SLA.

CRITERIA FOR CALL TASK APPROPRIATENESS

11

Language Learning Potential

•The extent to which the activity can be considered to be a language learning activity rather than simply an opportunity for language use.

•The degree of opportunity present for beneficial focus on form.

Learner Fit

•The individual differences in linguistic ability level and non-linguistic characteristics.

•The amount of opportunity for engagement with language under appropriate conditions given learner characteristics.

Meaning Focus

•The learner’s primary attention is directed toward the meaning of the language that is required to accomplish the task.

•The extent to which learners’ attention is directed toward the meaning of the language.

Authenticity

•The degree of correspondence between an L2 learning task and tasks that the learner is likely to encounter outside the classroom

•The degree of correspondence between the CALL activity and target language activities of interest to learners out of the classroom.

Positive Impact

•Its effects beyond its language learning potential. E.g. – help build metacognitive strategies, willingness to seek out opportunities to use TL.

•The positive effects of the CALL activity on those who participate in it.

Practicality

•How easy it is for the learners and teachers to implement a CALL task within the particular constraints of a class or language program.

•The adequacy of resources to support the use of the CALL activity.

CRITERIA FOR CALL TASK

APPROPRIATENESS12

Language Learning Potential

• They learn e.g. how to use modals

Learner Fit• Task appropriate at their level, taking into also consideration age, learning style, willingness to communicate

Meaning Focus

• Meaning of language to accomplish a task. e.g. using language to make decision, exchanging information to accomplish a goal

Authenticity• What they would actually encounter outside of the classroom, in the real world

Positive Impact• E.g. Seek opportunities to use what they have learned in class outside.

Practicality• Easy to implement CALL despite lacking facilities

CALL EVALUATION:

PRINCIPLE 413

Criteria should be applied in view of the purpose of thetask.

Implication: CALL tasks should have a clearlyarticulated purpose.

CALL EVALUATION:

PRINCIPLE 514

Language learning potential should be the centralcriterion (most critical) in evaluation of CALL.

Implication: Language learning potentialshould be one aspect of the purpose of CALLtasks.

JUDGEMENTAL ANALYSIS OF

CALL APPROPRIATENESS15

Qualities Questions

Language learning

potential

Do task conditions present sufficient opportunity for beneficial focus on form?

Learner fit Is the difficulty level of the targeted linguistics forms appropriate for the learners to

increase their language ability?

Is the task appropriate for learners with the characteristics of the intended learners?

Meaning focus Is learner’s attention directed primarily toward the meaning of the language?

Authenticity Is there a strong correspondence between the CALL task & second language tasks of

interest to learners outside the classroom?

Will learners be able to see the connection between the CALL task and tasks outside the

classroom?

Impact Will learners learn more about the target language and about strategies for language

learning through the use of the tasks.

Will instructors observe sound second language pedagogical practices by using the task?

Will both learners and teachers have a positive learning experience with technology

through the use of the task?

Practicality Are hardware, software, and personnel resources sufficient to allow the CALL task to

succeed?

Empirical Analysis of CALL

Appropriateness16

Qualities Questions

Language learning

potential

What evidence suggests that the learner has acquired the target forms that

were the focused on during the CALL tasks/

What evidence indicates that learners focused on form during the CALL task?

Learner fit What evidence suggests that the targeted linguistic forms are at an

appropriate level of difficulty for the learners?

What evidence suggests that the task is appropriate to learners’ individual

characteristics (age, learning styles, computer experience)

Meaning focus What evidence suggests that the learner’s construction of linguistic meaning

aids language learning?

What evidence indicates that the learners use the language during the task

for constructing and interpreting meaning?

Authenticity What evidence suggests that learners’ performance in the CALL tasks

corresponds to what one would expect to see outside the CALL tasks?

What evidence suggests that the learners see the connection between CALL

task and tasks outside the classroom?

Impact What evidence suggests that the learners learn more about the target

language and about strategies for language learning through the use of the

tasks?

Practicality What evidence suggests that hardware, software and personnel resources

prove to be sufficient to allow the CALL task to succeed?

THE END17