1

CHAPTER 1

INTRODUCTION

A. Background of Problem

Another essential activity in the design stage is that of defining the construct to be

measurment. The way we do this will be determined, largely, by the kinds of

inferences we want to make on the basis of score from language test. In this paper,

we discuss different approaches to defining the construct to be measured,

corresponding to the different kinds of inferences we want to make.

In defining the construct , the test developer needs to make a conscious and

deliberate choice to specify particular components of the ability or abilities to be

measured in a way that is appropriate to a particular testing situation. Specific

definitions of the abilities, or construct, are needed for three purpose:

1) To provide a basis for using test scores for their intended purpose

2) To guide test development effort, and

3) To enable the test developer and user to demonstrate the construct validity of

these interpretations.

B. Problem Statement

1. What is definition of construct to be measured ?

2. What plan for the language of the test ?

2

CHAPTER II

BODY

A. Defining The Construct, Language Ability

We defined language ability as including language knowledge and

strategic component, or metacognitive strategies. One consideration is thus to

decide which specific component of language ability are to be included in the

construct definition. In many testing situations the test user will want to make

inferences about specific component of language ability, and may thus define the

construct in terms of those component. This might be the case if the test is to be

used in an instructional setting and its purpose is to diagnose areas of strength and

weakness, or to asses the achievement of specific syllabus objectives. The test

developer will then most likely base the construct definition on the specific

components of language ability that are included in the course syllabus. In other

case, such as the use of tests for determining admission into an academic program,

or for making decisions about employment, where there may not be an

instructional syllabus, the test developer will most likely base the definition of the

construct on the component described in a theory of language ability.

When we design test tasks that engage test takers in activities that generate

a sample of language (spoken or written text), we have a product which we can

analyseand use to tease out specific components of language ability. In other

situation the test developer may plan to develop test tasks that do not engage test

takers in activities that generate extended samples of spoken or written text, yet

may nonetheless yield responses that can be used to tease out specific components

of language ability.

a. Syllabus-based construct definitions

Syllabus-based construct definitions distinguish among the specific components

of language ability that are included in an instructional syllabus. they are likely to

be most useful when we need to obtain detailed information on students mastery

3

of specific areas of language ability. for example suppose we were teaching a set

of specific grammatical structures, and wanted to develop an achievement test to

measure student ability to use them, so as to provide feedback on mastery of these

specific teaching points. We make prepare a definition of construct ability to use

grammatical structures accurately which included a list of the structures we had

taught, such as article usage, use of the past tense, subject-verb agreement, and so

forth.

b. Theory-based construct definitions

Theory-based construct definitions are different from syllabus-based definitions in

that they are based on theoretical model of language ability rather than the

contents of a language teaching syllabus. For example, suppose we wanted to

screen applicants for bilingual secretary’s job for which one requirement was the

ability to produce letters and memos in the appropriate register. We might make a

list of specific politeness formulae used for greetings, leave taking, introducing

contradictions, introducing clarifications, and so forth, and use these as a basis for

the construct definition.

If we define the construct to include more than one specific area of language

ability, we have the potential of providing several scores, or a profile of language

ability. For example, if we define our construct broadly to include knowledge of

grammatical organization, textual organization, functions, and sociolinguistic

characteristic, for each test taker we could report a profile of scores in each of

these areas of language ability. However, it is important to understand that just

defining the components is not sufficient justification for making componential

interpretations. We need to provide evidence of the construct validity of these

interpretations evidence of the validity of the inferences about the different

components of the construct definition before reporting scores for the different

components include in the construct definition.

4

c. Strategic competence

Strategic competence, or we have called metacognitive strategies, is always

involved in language use, even in tasks that are not very interactive. Thus the

strategies are always implied in our construct definitions of language ability and

can always be assumed to be part of the construct. However, in defining a

construct for purposes of testing, we need to decide whether or not we want to

make specific inferences about aspects of strategic competence.

Situation in which strategic competence is likely not to be included in construct

definition

In most language testing situations, we will probably not wish to make

specific inferences about strategic competence, so we would not be likely to

include this in our definition of the construct. In developing this experimental test,

one of authors noted that performance on the test seemed to be affected to a

considerable extent by the strategies the test taker used. Recall that the test task

required the test taker to describe one out of four similar pictures so that the

examiner could identify which had been described. The test was scored on the

basis of the amount of time taken to complete the task. Some test takers with very

little language knowledge were able to complete the test task very quickly by

focusing only on the relevant differences among the pictures and then describing

those differences using as simple language as possible. Other test takers with

greater language knowledge too longer to complete the task because they didn’t

use that the strategy.

Since the test developer wanted to use this test primarily as a test of language

knowledge, he did not include strategic competence in the construct definition. In

order to reduce the effect of differences in strategic competence on test

performance and allow the scores to more accurately reflect differences in

language knowledge, the test developer could have included instructions to be

sure to use complete, grammatically well-formed in their responses.

5

Situation in which strategic competence is likely to be include in the construct

definition.

In other testing situation, we may want to make specific inferences about

strategic competence, and will need to include it in construct definition. For

example, if in the situation describe above, the test developer had wanted to

measure not only language knowledge but also the test takers flexibility in

adapting their language use to different situations, he could have included

strategic competence in the construct definition, perhaps by defining it as ability

to assess the content of the pictures and develop a plan for the minimizing the

amount and complexity of information to be describe. In this case, the test

developer could have instructed the test takers to focus only on relevant

differences among the pictures and to describe these in the simplest language

possible, while still using language correctly. This might have served to increase

the effect of differences in strategic competence on test performance and thus

allowed the scores to more accurately reflect differences in both language

knowledge and strategic competence.

When we include strategic competence in construct definition, we need to decide

upon the level of specificity with which this needs to be defined. If we wanted to

assess competence in the use of specific strategies we would use a componential

definition of language ability, as described above.

B. The Role of Topical Knowledge in Construct Definitions

Topical knowledge of language users is always involved in language use. It

follows that if language test tasks are authentic and interactional, and elicit

instances of language use, test takers topical knowledge will always be a factor in

their test performance. Historically, language testers have viewed topical

knowledge almost exclusively as a potential source of test bias, or in validity, and

the traditional practice in developing language test is to design test tasks that will

minimize, or control, the test takers topical knowledge on test performance. We

take a slightly different view and would argue that this is not appropriate for all

6

situation. Although topical knowledge may, in many situation,be a potential

source of bias, there are other situations in which it may, in fact, be part of the

construct the test developer wants to measure.

we believe that there are essentially three options for defining the construct to be

measured, with respect to topical knowledge:

1. Define the construct solely in terms of language ability, excluding topical

knowledge from the construct definition,

2. Include both topical knowledge and language ability in the construct

definition, or

3. Define topical knowledge and language ability as separate constructs.

Option 1: Define the construct solely in terms of language ability, excluding

topical knowledge from the construct definition,

Typical situations:

Where test takers are expected to have relatively widely varied topical knowledge:

Language programs, where inferences about language ability are to be

used to make decisions about individuals.

Academic, professional, or vocational training programs, or for

employment, where inferences about language ability will be one of

several factors considered in the section process

Research in which language ability is include as a construct.

It is important to remember that even if the test developer has decided not to

include topical knowledge in the construct definition, it may still be involved in

test takers performance. for particular testing situation, the test developer may

simply choose to focus on language ability and not attempt to make inferences

about topical knowledge.

Intended inferences: component of language ability only

7

Option 2: topical knowledge include in the construct definition.

Typical situation:

Where test takers are expected to have relatively homogeneous topical

knowledge:

Language for specific purpose program, where the language is being

learned in conjunction with topical information related to specific

academic disciplines, professions, or vocations, and where inferences from

test score are to be used to make decisions about individuals

Selection for professional or vocational training program, or for

employment, where score from language test will be the major factor

considered in the selection process. this involves using the inferences to

make predictions about test takers capability to perform future task or job

that require the use of language.

Intended inferences: Ability to process(interpret or express) specific topical

information through language.

Option 3: language ability and topical knowledge defined as separate construct

Typical situation:

Where test developer may not know whether test tekers have relatively

homogeneous or relatively widely varied topical knowledge, and wants to

measure both language ability and topical knowledge.

Language for specific programs, where the language is being learned

in conjunction with topical knowledge related to specific academic

disciplines, professions, or vocations.

Selection for vocational training program, or for employment, where

scores from the language test will be the major factor considered in the

section process. these inferences will be used to make predictions

about test takers capability to perform future task or job that require

the use of language

8

Research in which language ability and topical knowledge are include

as construct.

Intended inferences: Component of language ability and areas of topical

knowledge

Dealing With The Problem of Topical Knowledge in Language Test

How to deal with potential effect of differing levels of topical knowledge

on language test scores is a problem fundamental to all language test. The

particular solution that the test developer arrives at will be a function of the

various factors discussed above. What is clear is that the rest developer cannot

simply assume, either explicitly or by default, that topical knowledge need to be

addressed simply because the focus of the test is on language ability. It is equally

clear that in all situations the language test developer needs to obtain as much

information as possible about potential test takers areas and levels of topical

knowledge, and should consult with content specialist in determining the areas of

topical content to include in the test and the accuracy of the information that is

included. Furthermore, in situations where the test developer/user wants to make

inferences about test takers language ability and areas topical knowledge, It is

crucial for the test development team to include specialists in both language and

the content areas to be assessed.

C. Role of Language Skill in Construct Definitions

We have taken the position that the familiar language skill (listening,

reading, speaking and writing) should not include in the construct definition. This

is because, if we distinguish among the language skill only in terms of mode

(productive or receptive)and channel (audio or visual), we end up with skill

definitions that miss many of the other important distinctions between language

used in particular tasks.

9

Another reasons for not including a specific skill in the construct

definition is that, as widdowson (1978) has pointed out, many language use tasks

involve more than one skill. What he call the communicative ability of

conversation involves listening and speaking, while what he call correspondence

involves reading and writing. Therefore, rather than defining a construct definition

include only the relevant components of language ability, and that the skill

element be specified as characteristics of the tasks in which language ability is

demonstrated.

10

CHAPTER III

CONCLUSION

The process of test development end with define construct to be measured

so that we can know how to interpret test score. During the design stage,

we define the construct, language ability, abstractly, and we can base this

definition on either the content of a language syllabus or a theory of

language ability. Although strategic competence will be involved in test

performance, the test developer may or may not want to make specific

inferences about this component of language ability, and thus must decide

whether or not to include it explicitly in the definition of construct. Test

taker topical knowledge will also involved in test performance, and thus

the test developer must also decide on the nature of the inferences to be

made, and define the construct accordingly. There are essentially three

options for this:

1. Define the construct solely in terms of language ability, excluding topical

knowledge from the construct definition,

2. Include both topical knowledge and language ability in the construct

definition, or

3. Define topical knowledge and language ability as separate constructs.