Build Bright UniversityBuild Bright UniversityLanguage Testing and Language Testing and

AssessmentAssessment

Chapter- 4 Chapter- 4 Standardized TestingStandardized Testing

Prepared by Kheang SokhengPrepared by Kheang SokhengPh.D Candidate and MEd in TESOLPh.D Candidate and MEd in TESOL

What is standardization?What is standardization?

A standardized test presupposes certain A standardized test presupposes certain standard objectives or criteria that are held standard objectives or criteria that are held constant across one form of the test to constant across one form of the test to another. another.

A good standardized test is the product of A good standardized test is the product of a thorough process of empirical research a thorough process of empirical research and development.and development.

It measures children’s mastery of the It measures children’s mastery of the standards or competencies that have been standards or competencies that have been prescribed for specific grade levels. prescribed for specific grade levels.

What is standardization?What is standardization? Scholastic Aptitude Test (SAT) is designed Scholastic Aptitude Test (SAT) is designed

for college entrance exam of many high for college entrance exam of many high school seniors seeking further education.school seniors seeking further education.

The Graduate Record Exam (GRE) for entry The Graduate Record Exam (GRE) for entry into many graduate school programs like into many graduate school programs like Graduate Management Admission Graduate Management Admission Test(GMAT) and the Law School Aptitude Test(GMAT) and the Law School Aptitude Test(LSAT) specialize in particular Test(LSAT) specialize in particular disciplines.disciplines.

Test of English as a Foreign Language Test of English as a Foreign Language ((TOEFLTOEFL)produced by the Educational Testing)produced by the Educational Testing

What is standardization?What is standardization?

Test for entry into many graduate school Test for entry into many graduate school programsprograms

Graduate Management Admission Test Graduate Management Admission Test (GMAT) & Law School Aptitude Test (LSAT): (GMAT) & Law School Aptitude Test (LSAT): tests that specialize in particular disciplinestests that specialize in particular disciplines

Test of English as a Foreign Language Test of English as a Foreign Language (TOEFL): It is produced the Educational (TOEFL): It is produced the Educational Testing Service (ETS) in the United States Testing Service (ETS) in the United States and/or its British counterpart, the and/or its British counterpart, the International English Language Testing International English Language Testing System(IELTS) System(IELTS)

What is standardization?What is standardization?

in the U.S and/or British counterpart, the in the U.S and/or British counterpart, the International English Language Testing International English Language Testing SystemSystem(IELTS)(IELTS)The tests are standardized because they The tests are standardized because they specify a set of competencies for a given specify a set of competencies for a given domain and through a process of construct domain and through a process of construct validation they program a set of tasks that validation they program a set of tasks that have been designed to measure those have been designed to measure those competencies. In general standardized test, competencies. In general standardized test, items are in the form of MC. items are in the form of MC.

What is standardization?What is standardization?

They provide ‘objective’ means for They provide ‘objective’ means for determining correct and incorrect determining correct and incorrect responses. responses.

However, MC is not the only test item However, MC is not the only test item type in standardized test.type in standardized test.

Human scored tests of oral and written Human scored tests of oral and written production are also involved.production are also involved.

Advantages of Standardized TestAdvantages of Standardized Test It is a ready-made previously validated It is a ready-made previously validated

product that frees the teacher from having to product that frees the teacher from having to spend hours creating a test.spend hours creating a test.

Administration to large groups can be Administration to large groups can be accomplished within reasonable time limits.accomplished within reasonable time limits.

In the case of multiple-choice formats, In the case of multiple-choice formats, scoring procedures are streamlined for either scoring procedures are streamlined for either scannable computerized scoring or hand-scannable computerized scoring or hand-scoring with a hole-punched grid for fast scoring with a hole-punched grid for fast turnaround time.turnaround time.

Disadvantages of Standardized Disadvantages of Standardized TestTest

Inappropriate use of such tests.Inappropriate use of such tests. Example: using an overall proficiency test Example: using an overall proficiency test as an achievement test simply because of as an achievement test simply because of the convenience of the standardizationthe convenience of the standardization Potential misunderstanding of the difference Potential misunderstanding of the difference

between direct and indirect testing. between direct and indirect testing.

Characteristics of a standardized test

DEVELOPING A STANDARDIZED TESTDEVELOPING A STANDARDIZED TEST

Knowing how to develop a standardized Knowing how to develop a standardized test can be helpful to revise an existing test can be helpful to revise an existing test, adapt or expand an existing test, test, adapt or expand an existing test, create a smaller-scale standardized testcreate a smaller-scale standardized test

(A) The Test of English as a Foreign (A) The Test of English as a Foreign Language (TOEFL)Language (TOEFL) ‘general ability or ‘general ability or proficiency’proficiency’

DEVELOPING A STANDARDIZED TESTDEVELOPING A STANDARDIZED TEST

(B) The English as a Second Language (B) The English as a Second Language Placement Test (ESLPT), San Francisco Placement Test (ESLPT), San Francisco State University (SFSU)State University (SFSU) ‘placement test ‘placement test at a university’at a university’

(C) The Graduate Essay Test (GET), (C) The Graduate Essay Test (GET), SFSUSFSU ‘gate-keeping essay test’‘gate-keeping essay test’

1. Determine the purpose and 1. Determine the purpose and objectives of the testobjectives of the test..

Standardized tests are expected to be valid Standardized tests are expected to be valid and practical.and practical.

TOEFL: TOEFL: To evaluate the English proficiency To evaluate the English proficiency of people whose of people whose NL NL is not English.is not English.

Colleges and universities in the US use the Colleges and universities in the US use the score TOEFL score to admit or refuse score TOEFL score to admit or refuse international applicants for admissioninternational applicants for admission

ESLPT: ESLPT: To place already admitted sts at To place already admitted sts at SFSU in an appropriate course in academic SFSU in an appropriate course in academic writing and oral productionwriting and oral production

1. Determine the purpose and objectives 1. Determine the purpose and objectives of the testof the test..

To provide teachers some diagnostic To provide teachers some diagnostic information about studentsinformation about students

GET: GET: To determine whether their To determine whether their writing ability is sufficient to permit writing ability is sufficient to permit them to enter graduate-level courses in them to enter graduate-level courses in their programs. It is offered at the their programs. It is offered at the beginning of each term)beginning of each term)

2.2. Design test specificationDesign test specification TOEFL:TOEFL: This is the step of laying the This is the step of laying the

foundation stones of the test.foundation stones of the test. Example:Example: TOEFL – Specifications: 1. Listening SectionTOEFL – Specifications: 1. Listening Section

2. Structure Section2. Structure Section3. Reading Section3. Reading Section4. Writing Section4. Writing Section

Each specs are not just stated that way, it Each specs are not just stated that way, it should include what does it measures, what should include what does it measures, what does it covers, and what material it uses.does it covers, and what material it uses.

..

2.2. Design test specificationDesign test specification

Listening section focuses on a Listening section focuses on a particular feature of language or overall particular feature of language or overall listening comprehensionlistening comprehension

Reading section aims to test Reading section aims to test comprehension of long/short passages, comprehension of long/short passages, single sentence, phrases or wordssingle sentence, phrases or words

3. 3. Design, select and arrange test Design, select and arrange test tasks/itemstasks/items

Once specifications for a standardized test Once specifications for a standardized test have been stipulated, the task of designing, have been stipulated, the task of designing, selecting and arranging test tasks/items selecting and arranging test tasks/items begins. begins.

The specs act much like a blueprint in The specs act much like a blueprint in determining the number and types of items to determining the number and types of items to be created.be created.

4. Make appropriate evaluations of 4. Make appropriate evaluations of

different kinds of itemsdifferent kinds of items.. Item facility (IF) – % of people who give the Item facility (IF) – % of people who give the

right answerright answer Item Discrimination (IDis) – indicates the Item Discrimination (IDis) – indicates the

extent to which success on an item extent to which success on an item corresponds to success on the whole test.corresponds to success on the whole test.

Item Difficulty (ID) - finding out the % of Item Difficulty (ID) - finding out the % of people who get the item right in the try-out people who get the item right in the try-out group.group.

4. Make appropriate evaluations of 4. Make appropriate evaluations of different kinds of items.different kinds of items.

Performing them may not be practical, Performing them may not be practical, especially if the classroom-based test especially if the classroom-based test is a one-time test. But for a is a one-time test. But for a standardized multiple-choice test that is standardized multiple-choice test that is designed to be marketed commercially, designed to be marketed commercially, or administered a number of times, and or administered a number of times, and administered in different form, these administered in different form, these indices are a must.indices are a must.

4. Make appropriate evaluations 4. Make appropriate evaluations of different kinds of itemsof different kinds of items

There are different form of evaluation There are different form of evaluation for other types of response formats. for other types of response formats. (e.g, Production responses)(e.g, Production responses)

Practicality - Practicality - Clarity of directionsClarity of directionsTiming of the testTiming of the testEase of administrationEase of administrationTime required to score responses.Time required to score responses.

4.4. Make appropriate evaluations Make appropriate evaluations of different kinds of itemsof different kinds of items..

Reliability – is the degree to which an Reliability – is the degree to which an assessment tool produces stable and assessment tool produces stable and consistent results.consistent results.

Facility –Facility – Unclear directionsUnclear directions Complex languageComplex language Obscure topics Obscure topics Fuzzy dataFuzzy data Culturally biased informationCulturally biased information..

5. Specify scoring procedures or 5. Specify scoring procedures or formatsformats

A systematic assembly of test items in pre-A systematic assembly of test items in pre-selected arrangements and sequences, all selected arrangements and sequences, all of which are validated to conform to an of which are validated to conform to an expected level of difficulty, should yield a expected level of difficulty, should yield a test that can then be scored accurately and test that can then be scored accurately and reported back reported back to test-takers and to test-takers and institutions efficiently.institutions efficiently.

6. Perform on going construct 6. Perform on going construct validation studiesvalidation studies

No standardized instrument is expected to No standardized instrument is expected to be used repeatedly without a rigorous be used repeatedly without a rigorous program of on going construct validation.program of on going construct validation.

Any standardized test, once developed, Any standardized test, once developed, must be accompanied by systematic must be accompanied by systematic corroboration of its effectiveness and by corroboration of its effectiveness and by steps towards its improvement.steps towards its improvement.

Standardized Language Proficiency Standardized Language Proficiency TestsTests

Standardized oral production testsStandardized oral production tests1.1. TSE – Test of Spoken EnglishTSE – Test of Spoken English2.2. OPI – Orals Proficiency InventoryOPI – Orals Proficiency Inventory3.3. TWE – Test of Written EnglishTWE – Test of Written English

Standardized Language Proficiency TestsStandardized Language Proficiency Tests Commercially produced standardized tests Commercially produced standardized tests

of English language proficiency:of English language proficiency:1.1. TOEFL – The Test Of English as a Foreign TOEFL – The Test Of English as a Foreign

LanguageLanguage2.2. MELAB – Michigan English Language MELAB – Michigan English Language

Assessment BatteryAssessment Battery3.3. IELTS – International English Language IELTS – International English Language

Testing SystemTesting System4.4. TOEIC – Test of English for International TOEIC – Test of English for International

CommunicationCommunication