7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 1/13
1
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 2/13
TABLE OF CONTENT
Preface i
TABLEOF CONTENT ii
CHAPTER 1INTRODUCTION 1
A. Background of Problem 1
B. Problem Statement 1
CHAPTER IICONTENT 2
A. Difining The Construct, Language Abilit 2
a. Sllabus!Based Construct Definition 2
b. Theor "Based Construct Definition #c. Strategic Com$etence %
B. The &ole of To$ical 'no(ledge in Construct Definitions )
C. &ole og Language Skill in Construct Definitions *
CHAPTER III CLOSING 1+
Conclusion...1+
&eferences....11
2
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 3/13
CHAPTER 1
INTRODUCTION
A. Background of Pro!e"
Another essential acti-it in the design stage is that of defining the construct to be
measurment. The (a (e do this (ill be determined, largel, b the kinds of
inferences (e (ant to make on the basis of score from language test. n this $a$er,
(e discuss different a$$roaches to defining the construct to be measured,
corres$onding to the different kinds of inferences (e (ant to make.
n defining the construct , the test de-elo$er needs to make a conscious and
deliberate choice to s$ecif $articular com$onents of the abilit or abilities to be
measured in a (a that is a$$ro$riate to a $articular testing situation. S$ecific
definitions of the abilities, or construct, are needed for three $ur$ose/
10 To $ro-ide a basis for using test scores for their intended $ur$ose
20 To guide test de-elo$ment effort, and
#0 To enable the test de-elo$er and user to demonstrate the construct -alidit of
these inter$retations.
B. Problem Statement
1. hat is definition of construct to be measured
2. hat $lan for the language of the test
CHAPTER II
BOD#
1
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 4/13
A. Def$n$ng T%e Con&'ruc'( Language A$!$')
e defined language abilit as including language kno(ledge and
strategic com$onent, or metacogniti-e strategies. 3ne consideration is thus to
decide (hich s$ecific com$onent of language abilit are to be included in the
construct definition. n man testing situations the test user (ill (ant to make
inferences about s$ecific com$onent of language abilit, and ma thus define the
construct in terms of those com$onent. This might be the case if the test is to be
used in an instructional setting and its $ur$ose is to diagnose areas of strength and(eakness, or to asses the achie-ement of s$ecific sllabus ob4ecti-es. The test
de-elo$er (ill then most likel base the construct definition on the s$ecific
com$onents of language abilit that are included in the course sllabus. n other
case, such as the use of tests for determining admission into an academic $rogram,
or for making decisions about em$loment, (here there ma not be an
instructional sllabus, the test de-elo$er (ill most likel base the definition of the
construct on the com$onent described in a theor of language abilit.
hen (e design test tasks that engage test takers in acti-ities that generate
a sam$le of language 5s$oken or (ritten te6t0, (e ha-e a $roduct (hich (e can
analseand use to tease out s$ecific com$onents of language abilit. n other
situation the test de-elo$er ma $lan to de-elo$ test tasks that do not engage test
takers in acti-ities that generate e6tended sam$les of s$oken or (ritten te6t, et
ma nonetheless ield res$onses that can be used to tease out s$ecific com$onents
of language abilit.
a. Sllabus!based construct definitions
Sllabus!based construct definitions distinguish among the s$ecific com$onents
of language abilit that are included in an instructional sllabus. the are likel to
be most useful (hen (e need to obtain detailed information on students master
of s$ecific areas of language abilit. for e6am$le su$$ose (e (ere teaching a set
of s$ecific grammatical structures, and (anted to de-elo$ an achie-ement test to
2
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 5/13
measure student abilit to use them, so as to $ro-ide feedback on master of these
s$ecific teaching $oints. e make $re$are a definition of construct abilit to use
grammatical structures accuratel (hich included a list of the structures (e had
taught, such as article usage, use of the $ast tense, sub4ect!-erb agreement, and so
forth.
b. Theor!based construct definitions
Theor!based construct definitions are different from sllabus!based definitions in
that the are based on theoretical model of language abilit rather than the
contents of a language teaching sllabus. 7or e6am$le, su$$ose (e (anted to
screen a$$licants for bilingual secretar8s 4ob for (hich one re9uirement (as the
abilit to $roduce letters and memos in the a$$ro$riate register. e might make a
list of s$ecific $oliteness formulae used for greetings, lea-e taking, introducing
contradictions, introducing clarifications, and so forth, and use these as a basis for
the construct definition.
f (e define the construct to include more than one s$ecific area of language
abilit, (e ha-e the $otential of $ro-iding se-eral scores, or a $rofile of language
abilit. 7or e6am$le, if (e define our construct broadl to include kno(ledge of
grammatical organi:ation, te6tual organi:ation, functions, and sociolinguistic
characteristic, for each test taker (e could re$ort a $rofile of scores in each of
these areas of language abilit. ;o(e-er, it is im$ortant to understand that 4ust
defining the com$onents is not sufficient 4ustification for making com$onential
inter$retations. e need to $ro-ide e-idence of the construct -alidit of these
inter$retations e-idence of the -alidit of the inferences about the different
com$onents of the construct definition before re$orting scores for the different
com$onents include in the construct definition.
c. Strategic com$etence
#
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 6/13
Strategic com$etence, or (e ha-e called metacogniti-e strategies, is al(as
in-ol-ed in language use, e-en in tasks that are not -er interacti-e. Thus the
strategies are al(as im$lied in our construct definitions of language abilit and
can al(as be assumed to be $art of the construct. ;o(e-er, in defining a
construct for $ur$oses of testing, (e need to decide (hether or not (e (ant to
make s$ecific inferences about as$ects of strategic com$etence.
Situation in which strategic competence is likely not to be included in construct
definition
n most language testing situations, (e (ill $robabl not (ish to make
s$ecific inferences about strategic com$etence, so (e (ould not be likel to
include this in our definition of the construct. n de-elo$ing this e6$erimental test,
one of authors noted that $erformance on the test seemed to be affected to a
considerable e6tent b the strategies the test taker used. &ecall that the test task
re9uired the test taker to describe one out of four similar $ictures so that the
e6aminer could identif (hich had been described. The test (as scored on the
basis of the amount of time taken to com$lete the task. Some test takers (ith -er
little language kno(ledge (ere able to com$lete the test task -er 9uickl b
focusing onl on the rele-ant differences among the $ictures and then describing
those differences using as sim$le language as $ossible. 3ther test takers (ith
greater language kno(ledge too longer to com$lete the task because the didn8t
use that the strateg.
Since the test de-elo$er (anted to use this test $rimaril as a test of languagekno(ledge, he did not include strategic com$etence in the construct definition. n
order to reduce the effect of differences in strategic com$etence on test
$erformance and allo( the scores to more accuratel reflect differences in
language kno(ledge, the test de-elo$er could ha-e included instructions to be
sure to use com$lete, grammaticall (ell!formed in their res$onses.
%
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 7/13
Situation in which strategic competence is likely to be include in the construct
definition.
n other testing situation, (e ma (ant to make s$ecific inferences about
strategic com$etence, and (ill need to include it in construct definition. 7or
e6am$le, if in the situation describe abo-e, the test de-elo$er had (anted to
measure not onl language kno(ledge but also the test takers fle6ibilit in
ada$ting their language use to different situations, he could ha-e included
strategic com$etence in the construct definition, $erha$s b defining it as abilit
to assess the content of the $ictures and de-elo$ a $lan for the minimi:ing the
amount and com$le6it of information to be describe. n this case, the test
de-elo$er could ha-e instructed the test takers to focus onl on rele-ant
differences among the $ictures and to describe these in the sim$lest language
$ossible, (hile still using language correctl. This might ha-e ser-ed to increase
the effect of differences in strategic com$etence on test $erformance and thus
allo(ed the scores to more accuratel reflect differences in both language
kno(ledge and strategic com$etence.
hen (e include strategic com$etence in construct definition, (e need to decide
u$on the le-el of s$ecificit (ith (hich this needs to be defined. f (e (anted to
assess com$etence in the use of s$ecific strategies (e (ould use a com$onential
definition of language abilit, as described abo-e.
B. T%e Ro!e of To*$ca! +no,!edge $n Con&'ruc' Def$n$'$on&
To$ical kno(ledge of language users is al(as in-ol-ed in language use. t
follo(s that if language test tasks are authentic and interactional, and elicit
instances of language use, test takers to$ical kno(ledge (ill al(as be a factor in
their test $erformance. ;istoricall, language testers ha-e -ie(ed to$ical
kno(ledge almost e6clusi-el as a $otential source of test bias, or in -alidit, and
the traditional $ractice in de-elo$ing language test is to design test tasks that (ill
minimi:e, or control, the test takers to$ical kno(ledge on test $erformance. e
take a slightl different -ie( and (ould argue that this is not a$$ro$riate for all
)
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 8/13
situation. Although to$ical kno(ledge ma, in man situation,be a $otential
source of bias, there are other situations in (hich it ma, in fact, be $art of the
construct the test de-elo$er (ants to measure.
(e belie-e that there are essentiall three o$tions for defining the construct to be
measured, (ith res$ect to to$ical kno(ledge/
1. Define the construct solel in terms of language abilit, e6cluding to$ical
kno(ledge from the construct definition,
2. nclude both to$ical kno(ledge and language abilit in the construct
definition, or
#. Define to$ical kno(ledge and language abilit as se$arate constructs.
O*'$on 1- Define the construct solel in terms of language abilit, e6cluding
to$ical kno(ledge from the construct definition,
T$ical situations/
here test takers are e6$ected to ha-e relati-el (idel -aried to$ical kno(ledge/
• Language $rograms, (here inferences about language abilit are to be
used to make decisions about indi-iduals.
•Academic, $rofessional, or -ocational training $rograms, or for
em$loment, (here inferences about language abilit (ill be one of
se-eral factors considered in the section $rocess
• &esearch in (hich language abilit is include as a construct.
t is im$ortant to remember that e-en if the test de-elo$er has decided not to
include to$ical kno(ledge in the construct definition, it ma still be in-ol-ed in
test takers $erformance. for $articular testing situation, the test de-elo$er ma
sim$l choose to focus on language abilit and not attem$t to make inferences
about to$ical kno(ledge.
ntended inferences/ com$onent of language abilit onl
O*'$on - to$ical kno(ledge include in the construct definition.
<
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 9/13
T$ical situation/
here test takers are e6$ected to ha-e relati-el homogeneous to$ical
kno(ledge/
• Language for s$ecific $ur$ose $rogram, (here the language is being
learned in con4unction (ith to$ical information related to s$ecific
academic disci$lines, $rofessions, or -ocations, and (here inferences from
test score are to be used to make decisions about indi-iduals
• Selection for $rofessional or -ocational training $rogram, or for
em$loment, (here score from language test (ill be the ma4or factor
considered in the selection $rocess. this in-ol-es using the inferences to
make $redictions about test takers ca$abilit to $erform future task or 4ob
that re9uire the use of language.
ntended inferences/ Abilit to $rocess5inter$ret or e6$ress0 s$ecific to$ical
information through language.
O*'$on /- language abilit and to$ical kno(ledge defined as se$arate construct
T$ical situation/
here test de-elo$er ma not kno( (hether test tekers ha-e relati-el
homogeneous or relati-el (idel -aried to$ical kno(ledge, and (ants to
measure both language abilit and to$ical kno(ledge.
• Language for s$ecific $rograms, (here the language is being learned
in con4unction (ith to$ical kno(ledge related to s$ecific academic
disci$lines, $rofessions, or -ocations.
• Selection for -ocational training $rogram, or for em$loment, (here
scores from the language test (ill be the ma4or factor considered in the
section $rocess. these inferences (ill be used to make $redictions
about test takers ca$abilit to $erform future task or 4ob that re9uire
the use of language
• &esearch in (hich language abilit and to$ical kno(ledge are include
as construct.
=
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 10/13
ntended inferences/ Com$onent of language abilit and areas of to$ical
kno(ledge
Dealing ith The Problem of To$ical 'no(ledge in Language Test
;o( to deal (ith $otential effect of differing le-els of to$ical kno(ledge
on language test scores is a $roblem fundamental to all language test. The
$articular solution that the test de-elo$er arri-es at (ill be a function of the
-arious factors discussed abo-e. hat is clear is that the rest de-elo$er cannot
sim$l assume, either e6$licitl or b default, that to$ical kno(ledge need to be
addressed sim$l because the focus of the test is on language abilit. t is e9uall
clear that in all situations the language test de-elo$er needs to obtain as much
information as $ossible about $otential test takers areas and le-els of to$ical
kno(ledge, and should consult (ith content s$ecialist in determining the areas of
to$ical content to include in the test and the accurac of the information that is
included. 7urthermore, in situations (here the test de-elo$er>user (ants to make
inferences about test takers language abilit and areas to$ical kno(ledge, t is
crucial for the test de-elo$ment team to include s$ecialists in both language and
the content areas to be assessed.
C. Ro!e of Language Sk$!! $n Con&'ruc' Def$n$'$on&
e ha-e taken the $osition that the familiar language skill 5listening,
reading, s$eaking and (riting0 should not include in the construct definition. This
is because, if (e distinguish among the language skill onl in terms of mode
5$roducti-e or rece$ti-e0and channel 5audio or -isual0, (e end u$ (ith skill
definitions that miss man of the other im$ortant distinctions bet(een language
used in $articular tasks.
Another reasons for not including a s$ecific skill in the construct
definition is that, as (iddo(son 51?=*0 has $ointed out, man language use tasks
in-ol-e more than one skill. hat he call the communicati-e abilit of
con-ersation in-ol-es listening and s$eaking, (hile (hat he call corres$ondence
*
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 11/13
in-ol-es reading and (riting. Therefore, rather than defining a construct definition
include onl the rele-ant com$onents of language abilit, and that the skill
element be s$ecified as characteristics of the tasks in (hich language abilit is
demonstrated.
C;APT@&
C3CLS3
?
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 12/13
The $rocess of test de-elo$ment end (ith define construct to be measured
so that (e can kno( ho( to inter$ret test score. During the design stage,
(e define the construct, language abilit, abstractl, and (e can base this
definition on either the content of a language sllabus or a theor of
language abilit. Although strategic com$etence (ill be in-ol-ed in test
$erformance, the test de-elo$er ma or ma not (ant to make s$ecific
inferences about this com$onent of language abilit, and thus must decide
(hether or not to include it e6$licitl in the definition of construct. Test
taker to$ical kno(ledge (ill also in-ol-ed in test $erformance, and thus
the test de-elo$er must also decide on the nature of the inferences to be
made, and define the construct accordingl. There are essentiall three
o$tions for this/
1. Define the construct solel in terms of language abilit, e6cluding to$ical
kno(ledge from the construct definition,
2. nclude both to$ical kno(ledge and language abilit in the construct
definition, or
#. Define to$ical kno(ledge and language abilit as se$arate constructs.
Reference&
1+
7/23/2019 Language Testing 2
http://slidepdf.com/reader/full/language-testing-2 13/13
7.Lle,Bachman S.Adrian,Palmer.Language Testing n
Practice/Designing and De-elo$ing seful Languge Test. (e e(
ork/36ford ni-ersit Press 51??<0.
7.Lle,Bachman.7undemental Considerations of Language Testing.(e
e( ork/36ford ni-ersit Press 51??+0.
11