November 4-5, 2002 Test Coordinator's Training 1

Testing 101

Dr. Nick Stayrook

Fairbanks North Star Borough

School District

November 4-5, 2002 Test Coordinator's Training 2

What is a test?

A test is a collection of questions designed to evaluate a student’s knowledge

Student’s responses to test questions are scored to determine level of knowledge Right or wrong (multiple choice) Quality of answer (rubric/scoring guide)

Tests generate scores for students

November 4-5, 2002 Test Coordinator's Training 3

Tests Used in State Assessments

Benchmark Exams in Grades 3, 6, and 8

TerraNova Exams in Grades 4, 5, 7, and 9

Alaska High School Qualifying Exam (beginning in grade 10)

Tests measure subject area knowledge Alaska Performance Standards Reading, Writing, and Mathematics

November 4-5, 2002 Test Coordinator's Training 4

Test Maps

Every test has a map of the content measured on the test Benchmark Exams use the performance standards TerraNova Exams use common objectives across

curricula used in the United States

Benchmark Test Maps on www.eed.state.ak.us

TerraNova Test Maps in Teacher’s Guide

November 4-5, 2002 Test Coordinator's Training 5

Determining Student Performance

Students receive points for each test question and total test points are computed

The total points are converted into a scale score which ranges from 100 to 600 for Benchmark Exams and HSGQE

The total points (raw score) are converted into a percentile score for TerraNova Exams which ranges from 1 to 99

November 4-5, 2002 Test Coordinator's Training 6

Performance Levels & Quartiles

For Benchmark Exams and the HSGQE the State has established performance levels Advanced, Proficient, Below Proficient, Not

Proficient for Benchmark Exams Proficient and Not Proficient for the HSGQE

For TerraNova Exams we report the percent of students in each quartile: Top Quartile is 99-76 percentile, bottom quartile is 1-25.

November 4-5, 2002 Test Coordinator's Training 7

Test Reports

Student Reports provide information to parents and students

School Reports provide summary information to schools and teachers

District Reports are summaries at the district level and for each school

November 4-5, 2002 Test Coordinator's Training 8

Uses of TestsTo assess individual student performance In relation to a specific “cut score” – Criterion

Referenced Tests (CRT) In relation to a “norm” group – Norm Referenced Tests

(NRT)

The difference in CRTs and NRTs is not the test questions, but how you use the resultsThe Benchmark Exams could be NRTs if we compared student results to a norm groupTerraNova could be a CRT if we established a cut score for proficiency

November 4-5, 2002 Test Coordinator's Training 9

Test Administration 101

Based on the work of

Dean Arasmith

Northwest Regional Educational Laboratory

November 4-5, 2002 Test Coordinator's Training 10

Test Scores from Assessments

A student’s observed test score consists of the student’s true score plus error.

Assumption: Error is random and not systematic

Systematic error can be understood as assessment bias

November 4-5, 2002 Test Coordinator's Training 11

Characteristics of Assessments

Reliability of Test – AccuracyReliability is the concept that if a test is given repeated times the scores would be similarTypes of Reliability Parallel Forms & Split Half Repeated measures (Test/Re-test) Internal Consistency

November 4-5, 2002 Test Coordinator's Training 12

Characteristics of Assessments

Validity of Assessment ToolValidity is the concept that an assessment tool measures what it is intended to measure. An assessment is validated based on the use to which it is put.Types of Validity Content Validity Predictive Validity Concurrent Validity Construct Validity Consequential Validity

November 4-5, 2002 Test Coordinator's Training 13

Sources of Errors in Test Scores

Errors due to the assessment tool

Errors due to student factors

Errors due to testing environment

November 4-5, 2002 Test Coordinator's Training 14

Errors Due to the Assessment Tool

Lack of high reliability or validityToo much reading on the math testUnclear or novel instructionsUnnecessarily difficult vocabulary used in directions or test itemsIrrelevant clues to right answersNot appropriate for the purpose intended

November 4-5, 2002 Test Coordinator's Training 15

Errors Due to Student Factors

Student fatigue, illness, or hunger

Lack of motivation

Lack of test wise-ness

Not understanding test directions

Not understanding purpose and importance of the test

November 4-5, 2002 Test Coordinator's Training 16

Errors Due to Testing Environment

Physical Room Conditions Too hot, dim lighting, no ventilation Noisy, distractions inside and outside room

Test Proctor Behaviors Does not adhere to test directions Gives assistance to students Negative attitude toward test

November 4-5, 2002 Test Coordinator's Training 17

To Eliminate or Reduce Errors in Test Scores

Strictly follow test administration procedures contained in: District Test Coordinator Manuals Teacher Administration Directions

Review your testing tasks for: Test preparation Testing day After Testing

November 4-5, 2002 Test Coordinator's Training 18

Test Preparation

Know who, when, where, and by whom.

Communicate test information to parents, students and teachers.

Provide training to test proctors and teachers.

Prepare students for the test (use practice tests).

Be familiar with procedures.

Establish and use a Hotline number if people have questions.

November 4-5, 2002 Test Coordinator's Training 19

Testing Day

Minimize disturbances.Check that proctors/teachers are familiar with materials, directions, and procedures.Put up “Do Not Disturb” signs.Verify identity of examinees.Check answer sheets/booklets for accuracy.Monitor breaks and going to the bathroom.Record all irregularities (test proctor logs)Account for all materials when finished.

November 4-5, 2002 Test Coordinator's Training 20

After Testing

Examine all answer sheets/booklets for tampering, excessive erasures, or other signs of cheating.Investigate all reports of breaches of test security.Account for ALL test materials – answer sheets, booklets, scrap paper, rulers, etc.Follow return directions precisely and return by due date.Know the Hotline number or contact for questions. Mark Leal – EED 907-465-8691

November 4-5, 2002 Test Coordinator's Training 21

Questions & Answers