Language Testing 2

7/23/2019 Language Testing 2

http://slidepdf.com/reader/full/language-testing-2 1/13

1



TABLE OF CONTENT

Preface i

TABLEOF CONTENT ii

CHAPTER 1INTRODUCTION 1

A. Background of Problem 1

B. Problem Statement 1

CHAPTER IICONTENT 2

A. Difining The Construct, Language Abilit 2

a. Sllabus!Based Construct Definition 2

b. Theor "Based Construct Definition #c. Strategic Com$etence %

B. The &ole of To$ical 'no(ledge in Construct Definitions )

C. &ole og Language Skill in Construct Definitions *

CHAPTER III CLOSING 1+

Conclusion...1+

&eferences....11

2



CHAPTER 1

INTRODUCTION

A. Background of Pro!e"

Another essential acti-it in the design stage is that of defining the construct to be

measurment. The (a (e do this (ill be determined, largel, b the kinds of

inferences (e (ant to make on the basis of score from language test. n this $a$er,

(e discuss different a$$roaches to defining the construct to be measured,

corres$onding to the different kinds of inferences (e (ant to make.

n defining the construct , the test de-elo$er needs to make a conscious and

deliberate choice to s$ecif $articular com$onents of the abilit or abilities to be

measured in a (a that is a$$ro$riate to a $articular testing situation. S$ecific

definitions of the abilities, or construct, are needed for three $ur$ose/

10 To $ro-ide a basis for using test scores for their intended $ur$ose

20 To guide test de-elo$ment effort, and

#0 To enable the test de-elo$er and user to demonstrate the construct -alidit of

these inter$retations.

B. Problem Statement

1. hat is definition of construct to be measured

2. hat $lan for the language of the test

CHAPTER II

BOD#

1



A. Def$n$ng T%e Con&'ruc'( Language A$!$')

e defined language abilit as including language kno(ledge and

strategic com$onent, or metacogniti-e strategies. 3ne consideration is thus to

decide (hich s$ecific com$onent of language abilit are to be included in the

construct definition. n man testing situations the test user (ill (ant to make

inferences about s$ecific com$onent of language abilit, and ma thus define the

construct in terms of those com$onent. This might be the case if the test is to be

used in an instructional setting and its $ur$ose is to diagnose areas of strength and(eakness, or to asses the achie-ement of s$ecific sllabus ob4ecti-es. The test

de-elo$er (ill then most likel base the construct definition on the s$ecific

com$onents of language abilit that are included in the course sllabus. n other

case, such as the use of tests for determining admission into an academic $rogram,

or for making decisions about em$loment, (here there ma not be an

instructional sllabus, the test de-elo$er (ill most likel base the definition of the

construct on the com$onent described in a theor of language abilit.

hen (e design test tasks that engage test takers in acti-ities that generate

a sam$le of language 5s$oken or (ritten te6t0, (e ha-e a $roduct (hich (e can

analseand use to tease out s$ecific com$onents of language abilit. n other

situation the test de-elo$er ma $lan to de-elo$ test tasks that do not engage test

takers in acti-ities that generate e6tended sam$les of s$oken or (ritten te6t, et

ma nonetheless ield res$onses that can be used to tease out s$ecific com$onents

of language abilit.

a. Sllabus!based construct definitions

Sllabus!based construct definitions distinguish among the s$ecific com$onents

of language abilit that are included in an instructional sllabus. the are likel to

be most useful (hen (e need to obtain detailed information on students master

of s$ecific areas of language abilit. for e6am$le su$$ose (e (ere teaching a set

of s$ecific grammatical structures, and (anted to de-elo$ an achie-ement test to

2



measure student abilit to use them, so as to $ro-ide feedback on master of these

s$ecific teaching $oints. e make $re$are a definition of construct abilit to use

grammatical structures accuratel (hich included a list of the structures (e had

taught, such as article usage, use of the $ast tense, sub4ect!-erb agreement, and so

forth.

b. Theor!based construct definitions

Theor!based construct definitions are different from sllabus!based definitions in

that the are based on theoretical model of language abilit rather than the

contents of a language teaching sllabus. 7or e6am$le, su$$ose (e (anted to

screen a$$licants for bilingual secretar8s 4ob for (hich one re9uirement (as the

abilit to $roduce letters and memos in the a$$ro$riate register. e might make a

list of s$ecific $oliteness formulae used for greetings, lea-e taking, introducing

contradictions, introducing clarifications, and so forth, and use these as a basis for

the construct definition.

f (e define the construct to include more than one s$ecific area of language

abilit, (e ha-e the $otential of $ro-iding se-eral scores, or a $rofile of language

abilit. 7or e6am$le, if (e define our construct broadl to include kno(ledge of

grammatical organi:ation, te6tual organi:ation, functions, and sociolinguistic

characteristic, for each test taker (e could re$ort a $rofile of scores in each of

these areas of language abilit. ;o(e-er, it is im$ortant to understand that 4ust

defining the com$onents is not sufficient 4ustification for making com$onential

inter$retations. e need to $ro-ide e-idence of the construct -alidit of these

inter$retations e-idence of the -alidit of the inferences about the different

com$onents of the construct definition before re$orting scores for the different

com$onents include in the construct definition.

c. Strategic com$etence

#



Strategic com$etence, or (e ha-e called metacogniti-e strategies, is al(as

in-ol-ed in language use, e-en in tasks that are not -er interacti-e. Thus the

strategies are al(as im$lied in our construct definitions of language abilit and

can al(as be assumed to be $art of the construct. ;o(e-er, in defining a

construct for $ur$oses of testing, (e need to decide (hether or not (e (ant to

make s$ecific inferences about as$ects of strategic com$etence.

Situation in which strategic competence is likely not to be included in construct

definition

n most language testing situations, (e (ill $robabl not (ish to make

s$ecific inferences about strategic com$etence, so (e (ould not be likel to

include this in our definition of the construct. n de-elo$ing this e6$erimental test,

one of authors noted that $erformance on the test seemed to be affected to a

considerable e6tent b the strategies the test taker used. &ecall that the test task

re9uired the test taker to describe one out of four similar $ictures so that the

e6aminer could identif (hich had been described. The test (as scored on the

basis of the amount of time taken to com$lete the task. Some test takers (ith -er

little language kno(ledge (ere able to com$lete the test task -er 9uickl b

focusing onl on the rele-ant differences among the $ictures and then describing

those differences using as sim$le language as $ossible. 3ther test takers (ith

greater language kno(ledge too longer to com$lete the task because the didn8t

use that the strateg.

Since the test de-elo$er (anted to use this test $rimaril as a test of languagekno(ledge, he did not include strategic com$etence in the construct definition. n

order to reduce the effect of differences in strategic com$etence on test

$erformance and allo( the scores to more accuratel reflect differences in

language kno(ledge, the test de-elo$er could ha-e included instructions to be

sure to use com$lete, grammaticall (ell!formed in their res$onses.

%



Situation in which strategic competence is likely to be include in the construct

definition.

n other testing situation, (e ma (ant to make s$ecific inferences about

strategic com$etence, and (ill need to include it in construct definition. 7or

e6am$le, if in the situation describe abo-e, the test de-elo$er had (anted to

measure not onl language kno(ledge but also the test takers fle6ibilit in

ada$ting their language use to different situations, he could ha-e included

strategic com$etence in the construct definition, $erha$s b defining it as abilit

to assess the content of the $ictures and de-elo$ a $lan for the minimi:ing the

amount and com$le6it of information to be describe. n this case, the test

de-elo$er could ha-e instructed the test takers to focus onl on rele-ant

differences among the $ictures and to describe these in the sim$lest language

$ossible, (hile still using language correctl. This might ha-e ser-ed to increase

the effect of differences in strategic com$etence on test $erformance and thus

allo(ed the scores to more accuratel reflect differences in both language

kno(ledge and strategic com$etence.

hen (e include strategic com$etence in construct definition, (e need to decide

u$on the le-el of s$ecificit (ith (hich this needs to be defined. f (e (anted to

assess com$etence in the use of s$ecific strategies (e (ould use a com$onential

definition of language abilit, as described abo-e.

B. T%e Ro!e of To*$ca! +no,!edge $n Con&'ruc' Def$n$'$on&

To$ical kno(ledge of language users is al(as in-ol-ed in language use. t

follo(s that if language test tasks are authentic and interactional, and elicit

instances of language use, test takers to$ical kno(ledge (ill al(as be a factor in

their test $erformance. ;istoricall, language testers ha-e -ie(ed to$ical

kno(ledge almost e6clusi-el as a $otential source of test bias, or in -alidit, and

the traditional $ractice in de-elo$ing language test is to design test tasks that (ill

minimi:e, or control, the test takers to$ical kno(ledge on test $erformance. e

take a slightl different -ie( and (ould argue that this is not a$$ro$riate for all

)



situation. Although to$ical kno(ledge ma, in man situation,be a $otential

source of bias, there are other situations in (hich it ma, in fact, be $art of the

construct the test de-elo$er (ants to measure.

(e belie-e that there are essentiall three o$tions for defining the construct to be

measured, (ith res$ect to to$ical kno(ledge/

1. Define the construct solel in terms of language abilit, e6cluding to$ical

kno(ledge from the construct definition,

2. nclude both to$ical kno(ledge and language abilit in the construct

definition, or

#. Define to$ical kno(ledge and language abilit as se$arate constructs.

O*'$on 1- Define the construct solel in terms of language abilit, e6cluding

to$ical kno(ledge from the construct definition,

T$ical situations/

here test takers are e6$ected to ha-e relati-el (idel -aried to$ical kno(ledge/

• Language $rograms, (here inferences about language abilit are to be

used to make decisions about indi-iduals.

•Academic, $rofessional, or -ocational training $rograms, or for

em$loment, (here inferences about language abilit (ill be one of

se-eral factors considered in the section $rocess

• &esearch in (hich language abilit is include as a construct.

t is im$ortant to remember that e-en if the test de-elo$er has decided not to

include to$ical kno(ledge in the construct definition, it ma still be in-ol-ed in

test takers $erformance. for $articular testing situation, the test de-elo$er ma

sim$l choose to focus on language abilit and not attem$t to make inferences

about to$ical kno(ledge.

ntended inferences/ com$onent of language abilit onl

O*'$on - to$ical kno(ledge include in the construct definition.

<



T$ical situation/

here test takers are e6$ected to ha-e relati-el homogeneous to$ical

kno(ledge/

• Language for s$ecific $ur$ose $rogram, (here the language is being

learned in con4unction (ith to$ical information related to s$ecific

academic disci$lines, $rofessions, or -ocations, and (here inferences from

test score are to be used to make decisions about indi-iduals

• Selection for $rofessional or -ocational training $rogram, or for

em$loment, (here score from language test (ill be the ma4or factor

considered in the selection $rocess. this in-ol-es using the inferences to

make $redictions about test takers ca$abilit to $erform future task or 4ob

that re9uire the use of language.

ntended inferences/ Abilit to $rocess5inter$ret or e6$ress0 s$ecific to$ical

information through language.

O*'$on /- language abilit and to$ical kno(ledge defined as se$arate construct

T$ical situation/

here test de-elo$er ma not kno( (hether test tekers ha-e relati-el

homogeneous or relati-el (idel -aried to$ical kno(ledge, and (ants to

measure both language abilit and to$ical kno(ledge.

• Language for s$ecific $rograms, (here the language is being learned

in con4unction (ith to$ical kno(ledge related to s$ecific academic

disci$lines, $rofessions, or -ocations.

• Selection for -ocational training $rogram, or for em$loment, (here

scores from the language test (ill be the ma4or factor considered in the

section $rocess. these inferences (ill be used to make $redictions

about test takers ca$abilit to $erform future task or 4ob that re9uire

the use of language

• &esearch in (hich language abilit and to$ical kno(ledge are include

as construct.

=



ntended inferences/ Com$onent of language abilit and areas of to$ical

kno(ledge

Dealing ith The Problem of To$ical 'no(ledge in Language Test

;o( to deal (ith $otential effect of differing le-els of to$ical kno(ledge

on language test scores is a $roblem fundamental to all language test. The

$articular solution that the test de-elo$er arri-es at (ill be a function of the

-arious factors discussed abo-e. hat is clear is that the rest de-elo$er cannot

sim$l assume, either e6$licitl or b default, that to$ical kno(ledge need to be

addressed sim$l because the focus of the test is on language abilit. t is e9uall

clear that in all situations the language test de-elo$er needs to obtain as much

information as $ossible about $otential test takers areas and le-els of to$ical

kno(ledge, and should consult (ith content s$ecialist in determining the areas of

to$ical content to include in the test and the accurac of the information that is

included. 7urthermore, in situations (here the test de-elo$er>user (ants to make

inferences about test takers language abilit and areas to$ical kno(ledge, t is

crucial for the test de-elo$ment team to include s$ecialists in both language and

the content areas to be assessed.

C. Ro!e of Language Sk$!! $n Con&'ruc' Def$n$'$on&

e ha-e taken the $osition that the familiar language skill 5listening,

reading, s$eaking and (riting0 should not include in the construct definition. This

is because, if (e distinguish among the language skill onl in terms of mode

5$roducti-e or rece$ti-e0and channel 5audio or -isual0, (e end u$ (ith skill

definitions that miss man of the other im$ortant distinctions bet(een language

used in $articular tasks.

Another reasons for not including a s$ecific skill in the construct

definition is that, as (iddo(son 51?=*0 has $ointed out, man language use tasks

in-ol-e more than one skill. hat he call the communicati-e abilit of

con-ersation in-ol-es listening and s$eaking, (hile (hat he call corres$ondence

*



in-ol-es reading and (riting. Therefore, rather than defining a construct definition

include onl the rele-ant com$onents of language abilit, and that the skill

element be s$ecified as characteristics of the tasks in (hich language abilit is

demonstrated.

C;APT@&

C3CLS3

?



The $rocess of test de-elo$ment end (ith define construct to be measured

so that (e can kno( ho( to inter$ret test score. During the design stage,

(e define the construct, language abilit, abstractl, and (e can base this

definition on either the content of a language sllabus or a theor of

language abilit. Although strategic com$etence (ill be in-ol-ed in test

$erformance, the test de-elo$er ma or ma not (ant to make s$ecific

inferences about this com$onent of language abilit, and thus must decide

(hether or not to include it e6$licitl in the definition of construct. Test

taker to$ical kno(ledge (ill also in-ol-ed in test $erformance, and thus

the test de-elo$er must also decide on the nature of the inferences to be

made, and define the construct accordingl. There are essentiall three

o$tions for this/

1. Define the construct solel in terms of language abilit, e6cluding to$ical

kno(ledge from the construct definition,

2. nclude both to$ical kno(ledge and language abilit in the construct

definition, or

#. Define to$ical kno(ledge and language abilit as se$arate constructs.

Reference&

1+



7.Lle,Bachman S.Adrian,Palmer.Language Testing n

Practice/Designing and De-elo$ing seful Languge Test. (e e(

ork/36ford ni-ersit Press 51??<0.

7.Lle,Bachman.7undemental Considerations of Language Testing.(e

e( ork/36ford ni-ersit Press 51??+0.

11

Documents

Language Testing 2