Technology In Standardized Language Assessments

E4C Vera

Una

Alley

Introduction

• The emergence of computers represents a major turning point in our civilization.

• Computer have had, and will have , a large and even defining impact on second language assessment as well.

• CBT ( Computer-Based-Tests) has become an appealing and viable medium for the administration of standardized L2 tests in academic and nonacademic institutions.

Introduction

• Type I (Maddux) and sustaining technology (Christensen) refer to innovations intended to simplify and facilitate current practices and products. Ex. The use of the computer to administer learning drills.

• It is more flexible and individualized test administration, tracking of student performance, immediate test feedback, new item/task types, and enhanced test security.

Introduction

• Type II (Maddux) and disruptive technology (Christensen) lead to the accomplishment of something that previously had not been considered plausible, such as utilizing the computer as part of new models of instruction, such as providing distance learning.

• CAT (Computer Adaptive Testing) enables tailoring item difficulty to test-takers’ performance, allowing a more accurate assessment of test-takers’ performance, allowing a more accurate assessment of test-takers’ L2 ability.

Introduction

• Areas covered include the representation of the L2 construct, overall test design and measurement issues, item/task construction, and test purpose.

Second Language CBT Instruments

• The following presents a brief description of representative sample of these operational CBT and CAT projects that have been developed in academic and nonacademic institutions.

• The first CBTs developed in the L2 field-the placement tests in French, Greman, Spanish, Russian and ESL in item response theory model. (IRT)

• French placement test is the Computer Adaptive Proficiency Test (CAPT) in a three- parameter IRT and graded-response models.

• Dunkel at Georgia State University, an ESL and a Hausa CAT.

• Southern Illinois University also has an ESL placement CAT.

• TOEFL has administered as CBT in the United States of America and in other countries around the world.

Discussion in tests• The tests employ largely paper-pencil thinking in t

heir approach to assessment in using IRT models mainly multiple-choice and generate an overall scores used for selection and placement purposes.

• The various advantages of L2 CBTs functions mainly to help make assessment more efficient and serviceable.

• Advances that can help make this assessment become Type II or disruptive are still needed.

Representation of The Construct

• The construct comprises a complex constellation of components that include knowledge of the language ues, and a variety of strategies and processing skills needed to access, plan, and execute the communicatve intents.

• Test-takers at different ability levels differ in their command of linguistic and nonlinguistic concepts and knowledge, the richness of the connections among knowledge structures, the kinds of processes they employ, and the degree of automaticity governing these processes.

• Applied linguists need to make explicit the nature of the interactive components and processes involved in L2 performances in order to utilize the information to help explicate test performances.

• The integration of skills provides for a more meaningful and appropriate depiction of language use.

Overall Test Design

• Unidimensional measurement models are at odds with the multidimensional representation of the L2 construct.

• Extensions of and alternatives to current models have been put forth in the measurement literature.

• Technological innovations and measurement advances can help CBT move beyond conventional test design practices.

• Information generated by the use of alternative measurement models and an integrated test design can greatly enhance the information and quality of inferrences made about test-takers’ L2 abilities.

Item/Task Construction

• In standardized test it is more of an intuitive process that represents the complex L2 construct and provides clear link between task and the abilities needed to perform these tasks.

Item/Task Construction

• Bachman and Palmer: distinctive task characteristics

• The setting

• The characteristics of the input and the expected response

• The relationship between input and expected response

Item/Task Construction

RGM: response generative modeling.

it can be used to generate many kinds of task types, such as selected response multiple-choice or more complex types, such as simulations.

• Simulations: used in instructional settings and to some extent in assessment(e.g. role plays, interview, and other authentic tasks).

Test purpose

• The main purpose of most standardized assessments, including CBT, is to help test user make accept/ reject or classification decisions.

• L2 CBT purposes reflect the “gatekeeping” practices.

Test purpose

CM(continous measurement) CBT: learner centered, with an emphasis on meeting individual needs.

• A close relationship between assessment and the curriculum.

• It is integrated into instructional plans in the form of exercises and activities.

• It is intended to produce a rich profile of learners’ strengths and weaknesses.