Uses Of Language TestsBachman (1990)

By Mahsa FarahanyniaPhD studentAllameh Tabataba'I University 2015

Major uses of language tests 1 . Education: As sources of information for making decisions and evaluation within the context of educational programs justify the usefulness and quality of tests for educational settings meet certain assumptions and considerations

2. Research: As indicators of abilities or attributes that are of interest in research on language, language acquisition, and language research

Uses of language tests in educational programs (Bachman, 1990)

The main use making decisions and evaluation

Components of Evaluation (a) information (either qualitative/non-measurement or quantitative/ measurement)(b) Value judgements or decisions

Assumptions1 . Information regarding outcome is essential to effective formal education. Essential mechanisms for the continued effectiveness of any educational program are: a) Accountability: “Being able to demonstrate the extent to which we have effectively and efficiently discharged responsibility” (Bachman & Savignon, 1986)

b) Feedback: Information that is provided to teachers, students, and other interested persons about the effects of the educational programs

Cont. Assumptions 2. It is possible to use feedback to improve teaching and learning via appropriate changes in the program.

3. The educational outcomes of a program are measurable (a highly debatable assumption) Bachman states what can be defined can be measured

Considerations1. The amount and type of testing depends on a) decisions b) type of information need to make proper decisions 2. The quality of information the tests must provide effect on individual lives information must be as reliable and valid as possible to reduce the probability of errors in making decisions and potential costs these errors may occur

Cost-effectiveness: Striking a balance between the costs of assuring reliability and validity and the potential costs of making wrong decisions

In nation-wide examination high costs of making wrong decisions

1. eventual fail of students and lots of drop outs 2. The vain expenditure of educational resources 3. The misdirection of students’ time and effort 4. The potential loss to society

In classroom tests the costs of making wrong decisions is low

The principal educational uses of language tests (Harris)

1. To determine readiness for instructional program

2. To classify or place individuals in appropriate language tests

3. To diagnose the individual’s specific strengths and weaknesses

4. To measure aptitude for learning 5. To measure the extent of student

achievement of the instructional goals6. To evaluate the effectiveness of instruction

Types of decisions (Bachman) Decisions about individuals (micro-evaluation) 1) Decisions about students (selection/entrance/readiness – placement – diagnosis - Progress and grading ) 2) decisions about teachers

Decisions about programs (macro-evaluation)

Selection/ entrance/ readiness

Utilized in conjunction with measures of other abilities (In North America, TOEFL is used as a complement to other types of information such as academic achievement or recommendation letters for the acceptance of non-native English-speaking students)

Selection/ entrance to decide whether or not they should enter the program

Readiness to decide whether or not they are ready for instruction

Placement tests To place students into appropriate groups (based on language proficiency level, language aptitude, language use need, etc.

Two considerations:In the design of placement tests, test content may be based on either: a) A theory of language proficiency (where students enter from a wide variety of language backgrounds and it is hard to specify a syllabus encompassing a full range of competences) a norming procedure

b) Learning objectives of the syllabus (where the objectives are clearly specified and sequenced

Placement tests2) The relative stability of enrollments a) Having the same number of students in different

levels of a program from one term to the next NRT teachers’ adjusting their teaching to accommodate possible differences in levels of proficiency from term to term.

b) Having large changes in the number of students enrolled at different levels from term to term CRT without changing the cut-offs teachers’ adjusting for the differences in numbers

Diagnosis To detect students’ area of strength and

weakness to determine appropriate types and levels of teaching and learning activities

Providing detailed information about the

specific content domains that can be either theory-based or syllabus-based

Almost all tests are diagnostic in nature

Progress and grading

Formative evaluation mostly based on qualitative feedback students and teachers obtain in the process of learning and teaching activities

Achievement or attainment tests Quizzes at the end of each instructional unit/units to provide additional information on student progress

Mastery tests whether or not students have mastered the course content Summative evaluation mostly quantitative, in the form of grades, administered at the end of the instructional course

To provide feedback on the effectiveness of student learning

syllabus-based

Decision about teachers Language proficiency tests for hiring teachers Qualitatively and quantitatively different from proficiency

tests of students

Teachers must be proficient in a) the complete range of language abilities b) a variety of pedagogically oriented communication strategies and metalinguistic competences talk with students about language in a way that will be instructionally useful

Decisions about programs

Evaluating specific components of programs in terms of its appropriateness, effectiveness, and efficiency to improve the program

Formative assessment for the programs under development using achievement tests based on the content of the syllabus

Summative evaluation to see whether it is better than other programs both achievement tests and measures of language proficiency

Functions of language tests (Farhady, Jafarpour,& Birjandi )

Functions Prognostic Evaluation of attainment

Selection Placement Aptitude Achievement Proficiency Knowledge

General Diagnostic

Purposes of assessment (Cohen)

Classifications (Bachman)

Intended use Admission decisions selection/entrance/readiness

Identifying appropriate instructional levels

placement and diagnostic tests

knowing about the students’ progress

progress, achievement, attainment, or mastery tests

Content Theory-based tests Language proficiency tests Aptitude tests

Syllabus- based tests achievement tests

Frame of reference

NRT Standardized testsCRT  

Scoring procedure

Subjective oral interviews and writing

Objective multiple choice items, dictation, close tests

Testing method translation, dictation, composition, scored interview, multiple-choice, short-answer items 

 

Classifications Based on intended use Admission decisions

selection/entrance/readiness Identifying appropriate instructional levels or

specific areas required greater instruction placement and diagnostic tests

knowing about the students’ progress progress, achievement, attainment, or mastery tests

Classifications Based on content

Theory-based tests

a) Language proficiency tests (the use of language)

b) Aptitude tests (language acquisition is related to the cognitive abilities such as rote memorization, phonetic coding, sensitivity to grammatical analogy) Syllabus- based tests (achievement tests)

Classifications Based on the frame of reference A) Norm-referenced interpretation with reference to the

particular group of individuals (norms) NRT/psychometric tests

-Norm group: A large group of individuals who are similar to the individuals for whom the test is designed - Properly designed NR normally distributed

scores - mean (as central tendency) & SD (as dispersion)- Maximizing variations among individuals’ scores- Discriminating strong and weak students

Classifications The quintessential NR test standardized tests Three characteristics: 1) based on a fixed or standardized content which doesn’t vary from one form of the test to another 2) Standard procedures for administering and scoring the test3) Being well tried out and being empirically developed and studied

B) Criterion- or domain- based referenced interpretation with respect to a specific level or domain of ability degree of mastery

- Criterion the level of ability or domain of content not a cut-off score

Classifications Based on scoring procedure a) Objective tests Correctness of the test taker’s response is determined entirely by predetermined criteria so that no judgment is required on the part of scorers multiple choice items, dictation, close tests

b) Subjective tests The scorer must make a judgement about the correctness of the response based on her subjective interpretation of the scoring criteria oral interviews and writing

Classifications Based on testing method Multiple choice, completion, dictation, etc.

Harris: the principal language-testing techniques are: translation, dictation, composition, scored interview, multiple-choice, short-answer items

Thanks for Your Attention