Assessment TrainingAssessment TrainingNebo School DistrictNebo School District

Assessment LiteracyAssessment Literacy

Test AcronymsTest Acronyms CRT -CRT -Criterion Referenced TestCriterion Referenced Test

1-111-11

IOWA IOWA –Iowa Test of Basic Skills and Iowa Test of Educational –Iowa Test of Basic Skills and Iowa Test of Educational DevelopmentDevelopment3,5,8,&113,5,8,&11

UBSCT -UBSCT -Utah Basic Skills Competency TestUtah Basic Skills Competency Test10-1210-12

DWA -DWA -Direct Writing Assessment Direct Writing Assessment 6&96&9

UAA –UAA –Utah Alternate AssessmentUtah Alternate Assessment1-12 with severe cognitive disabilities1-12 with severe cognitive disabilities

UALPA -UALPA -The Utah Academic Language Proficiency Assessment The Utah Academic Language Proficiency Assessment 1-12 ELL1-12 ELL

Norm-Referenced TestsNorm-Referenced Tests Standardized TestsStandardized Tests Scores interpreted in terms of comparison Scores interpreted in terms of comparison

to a specific groupto a specific group Percentile scores are most common Percentile scores are most common

measurement of achievementmeasurement of achievement Percentile scores range from 1st to 99Percentile scores range from 1st to 99thth

with the 50with the 50thth percentile being used to percentile being used to represent the national averagerepresent the national average

ITBS and ITED (IOWA) tests are the state ITBS and ITED (IOWA) tests are the state adopted Norm-Referenced Assessmentsadopted Norm-Referenced Assessments

Criterion-Referenced TestsCriterion-Referenced Tests

Standardized TestsStandardized Tests Every question/item is aligned to an Every question/item is aligned to an

explicitly stated educational objectiveexplicitly stated educational objective Used to identify which standards and Used to identify which standards and

objectives have been mastered by the objectives have been mastered by the examineeexaminee

CRT or End-of-Level tests in Language CRT or End-of-Level tests in Language Arts, Math, and ScienceArts, Math, and Science

Summative AssessmentSummative Assessment

Used to determine the students’ final Used to determine the students’ final understanding of materialunderstanding of material

State CRT tests are an exampleState CRT tests are an example

Formative AssessmentFormative Assessment

Used to identify the students’ Used to identify the students’ understanding of material, to provide understanding of material, to provide feedback for teachers and learning feedback for teachers and learning experiences for studentsexperiences for students

Benchmarks, UTIPS, Running Records, Benchmarks, UTIPS, Running Records, and Student Interviews are all included in and Student Interviews are all included in this categorythis category

Raw scoreRaw score

The number of correct responses on a testThe number of correct responses on a test A student answered 48 questions correctlyA student answered 48 questions correctly

Percent Correct ScorePercent Correct Score

The number of correct responses divided The number of correct responses divided by the total number if itemsby the total number if items

49 out of 70 = 70%49 out of 70 = 70%

Percentile ScorePercentile Score

The percent of students who performed The percent of students who performed worse on a testworse on a test

7575thth percentile – 75% of examinees scored percentile – 75% of examinees scored lower on the test than this examineelower on the test than this examinee

Scaled ScoreScaled Score

The students performance is based on an The students performance is based on an arbitrary numerical scale (can be arbitrary numerical scale (can be alphabetical)alphabetical)

A scaled score correctly provides A scaled score correctly provides comparable information on student comparable information on student performance for different years on different performance for different years on different teststests

ACTACT

What is 36?What is 36? What is 28?What is 28? What is 12?What is 12? These numbers represent the value we place on These numbers represent the value we place on

numbers in a scale numbers in a scale Often we have the help of others such as Often we have the help of others such as

colleges in setting valuecolleges in setting value Utah State University and University of Utah say Utah State University and University of Utah say

you must have at least a score of 18you must have at least a score of 18

Scaled ScoresScaled Scores

Act Scores range from 10-36Act Scores range from 10-3618-28 is considered proficient depending 18-28 is considered proficient depending on schoolon school

Advanced Placement tests range from 1-5 Advanced Placement tests range from 1-5 3 is proficient3 is proficient

UBSCT and CRT range from 100-200 UBSCT and CRT range from 100-200 160 is proficient160 is proficient

Scaled ScoresScaled Scores

Scaled scores offer the advantage of Scaled scores offer the advantage of simplifying the reporting of resultssimplifying the reporting of results

There can be common score reporting for There can be common score reporting for each level and for each testeach level and for each test

No more specific percentages for cut No more specific percentages for cut scores for each subjectscores for each subject

Far greater comparability between tests Far greater comparability between tests and yearsand years

Scaled ScoresScaled Scores

CRTs and UBSCT use a cut score of 160CRTs and UBSCT use a cut score of 160 Each proficiency level has its own cut Each proficiency level has its own cut

scorescore Proficiency levels range from 1-4 in NCLB Proficiency levels range from 1-4 in NCLB

and 1a-4 in UPASS (We will discuss this in and 1a-4 in UPASS (We will discuss this in the next session)the next session)

ExampleExample

If john has a raw score of 65 in 2004, and If john has a raw score of 65 in 2004, and a raw score of 58 in 2005,does this show a a raw score of 58 in 2005,does this show a decrease in performance?decrease in performance?

If john has a scaled score of 165 in 2004, If john has a scaled score of 165 in 2004, and a scaled score of 155 in 2005, does and a scaled score of 155 in 2005, does this show a decrease in performance?this show a decrease in performance?

Why Not Raw ScoresWhy Not Raw Scores

Most states do not release raw scoresMost states do not release raw scores Looking at raw scores can lead to an Looking at raw scores can lead to an

incorrect assumptionincorrect assumption It is incorrect to compare raw scores from It is incorrect to compare raw scores from

one year to those of the nextone year to those of the next It is incorrect to compare the raw scores of It is incorrect to compare the raw scores of

one test to those of anotherone test to those of another

Career Home Runs

EQUATING

• Individually AbilityStrength

Skill

Technique

Knowledge

• Difficulty of the gameTightly Wound Baseballs

Improved Bats

Higher Pitchers Mound

Changes in Season Length

Steroids .

Who Is The Greatest?

ComparisonsComparisons

Impossible to compare Barry Bonds with Impossible to compare Barry Bonds with Babe RuthBabe Ruth

Impossible to compare a game in 1914 to Impossible to compare a game in 1914 to a game in 2006a game in 2006

ComparisonsComparisons

Possible to compare johns ability on the Possible to compare johns ability on the 2005 language arts CRT with johns ability 2005 language arts CRT with johns ability on the 2006 language arts CRT on the 2006 language arts CRT (Scaling)(Scaling)

Possible to compare the difficulty of the Possible to compare the difficulty of the 2005 language arts CRT to the 2006 CRT2005 language arts CRT to the 2006 CRT(Equating)(Equating)

EquatingEquating

Statistical process that takes different tests Statistical process that takes different tests and makes them equal in difficultyand makes them equal in difficulty

Disentangles differences between test Disentangles differences between test difficulty and student abilitydifficulty and student ability

EquatingEquating

Common (anchor) items between test Common (anchor) items between test formsforms

Statistical comparison of common items Statistical comparison of common items for equivalent difficulty levelfor equivalent difficulty level

This statistical process ensures that This statistical process ensures that results from test to test are accurately results from test to test are accurately comparable and not subject to fluctuations comparable and not subject to fluctuations due to unintentional changes in item due to unintentional changes in item difficultydifficulty

EquatingEquating

Anchor Items

Form XForm X

Anchor Items

Form YForm Y

Anchor ItemsAnchor Items It is the performance of the two sets of anchor It is the performance of the two sets of anchor

items across years that allow us to make items across years that allow us to make interpretations about the relative difficulty of the interpretations about the relative difficulty of the non-anchor itemsnon-anchor items

If student performance on the anchor items is If student performance on the anchor items is the same, we conclude that the student the same, we conclude that the student achievement is the sameachievement is the same

If student performance on the anchor items If student performance on the anchor items increases we can interoperate that student increases we can interoperate that student achievement increasedachievement increased

If student performance on the anchor items If student performance on the anchor items decreases we interoperate that student decreases we interoperate that student achievement decreasedachievement decreased

We use this information to judge the difficulty of We use this information to judge the difficulty of the non-anchor itemsthe non-anchor items

Why EquateWhy Equate

One test is more difficult than anotherOne test is more difficult than another One group of examinees may be more One group of examinees may be more

intelligent than anotherintelligent than another BothBoth