Scale Scoring

A New Format for Provincial Assessment Reports

http://www.ella.ednet.ns.ca

Beginning in the 2007-2008 school year, all

provincial assessments will be reported on a

common scale.

Why Use a Common Scale?

Imagine trying to accurately compare a student’s or a school’s achievement when given the following raw scores:

letter grades numerical marks rubric levels percentage grades

What is a Scale?

A scale is an arbitrarily established set of numbers used for measurement.

Different scales may be used to measure the same thing, but they are calibrated differently. (e.g. rulers, thermometers, speedometers, bathroom scales)

What is a Scale?

A scale, simply put, is a mathematical conversion of raw scores to a

common scale or a derived score.

A scaled score is a conversion of a student's rawscore on a test to a common scale that allows for a

numerical comparison between students.

Provincial assessments use multiple versions of a testover the years. The scale is used to control slightvariations from one version of a test to the next.

Scaled scores, when equated, are particularly useful for comparing test scores over time.

Equating: A Statistical Process

Equating measures the difficulty of each assessment (content,

cognition, difficulty) and adjusts the cut score to

account for differences among assessment forms.

The difficulty level of tests in different years can be

equated and this enables us to compare assessment

results from year to year.

Advantages of Scale Scores

Scaled scores allow better comparison of assessmentresults:

• from year to year

• from one grade level assessment to another

• of student achievement in different subject areas

Future Advantages

Because each successive year’s assessments are statistically equated with a baseline reference year, the scaled scores can also be used to reflect growth.

For example, if a future scaled score is higher than the reference year scale score (2007-2008), it can be interpreted as an improvement from that base year.

Cautions for Comparisons

Many factors impact upon a school’s results.

No single assessment result tells the whole story of a student’s or school’s achievement.

Sample size does matter! Schools with fewer students will show more variability in scores from one year to the next.

Scale Scoring

500400 600

65 %

of students who wrote this assessment

469300200 800700

Scale Scores

Provincial Mean One

standard deviation above

Cut score

One standard deviation below

To review:

Distribution of Scores:- Scale ranges from 200 to 800- Provincial mean is set to 500- Provincial standard deviation is set to 100- On most provincial assessments, about 65% of

students will place between 400 and 600- The cut score is 469

Cut ScoreA cut score defines the point at which acertain required level of achievement has

been demonstrated on the assessment.

Cut scores are set by a panel of educatorswho look at the actual test questions todetermine levels of acceptable performance.

A cut score can be above or below the mean.

Cut Score

A cut score differentiates between students who are

meeting early stage expectations

or

meeting some early stage expectationsnot yet meeting early stage expectations

Interpreting Results Overall Score

Cut Score:

How close or how far away from the cut score (469)did the student perform?

Did the student meet expectations for this assessment or hasthe student not yet met expectations for this assessment?

REMEMBER: The cut score only applies to the overall score(section 1 and 2A) the student received.

Interpreting ResultsOverall Score

Provincial Mean (500)

How close or how far away from the mean did the student perform?

How did the student perform relative to others that wrote the assessment?

Using the Results

Student’s Performance in Silent Reading:• Provides information on student performance in realistic

fiction and nonfiction/multimodal text and cognitive levels.

• Allows for comparison to the provincial mean of 500. • May help teachers isolate concerns and

develop strategies to address these indicators.• is intended to provide to teachers information

for planning purposes only.

Using the Results Overall Score

Students Meeting Expectations (470 and above)If the overall score is above the cut score (470) the student is

meeting early stage expectations.

If the overall score is within the 351-469 range, the student is meeting some early stage expectations.

For every student who is meeting some early stage expectations teachers will be expected to:

• Collect classroom assessment data• Develop a Literacy Development Record (LDR)• provide on-going, in-class support• contact parents/guardians to discuss support options

Using the Results Overall Score

Students Not Yet Meeting Expectations (Overall Score below 350)

For every student who is not yet meeting early stage expectations (overall score is below 350) teachers will be expected to:

• Develop a Literacy Development Record (LDR)• Support student within the class structure• Contact parents/guardians to discuss support options

The LDR will be on-going for students, with transition meetings at the end of each grade.

Students with the following performance in writing, will require an LDR:

• Students who met some early stage expectations • Students who did not yet meet early stage expectations • Insufficient Evidence

Using the Results

The Writing Performance section:

• Provides information on student performance in the elements of writing: ideas, organization, language use and conventions.

• May help teachers identify concerns and trends in student writing.

Determining Monitored Status

• In some instances, a student who ‘met some of the early stage expectations’ may be placed on monitored status.

• If current classroom assessment indicates that a student’s literacy development matches with current curriculum expectations, he/she may be placed on monitored status.

Criteria for determining monitor status:

• Student met some early stage expectations or has insufficient evidence on ELLA.

• Classroom/school/board assessment information collected within the past 4-6 weeks suggest that the student meets grade level expectations.

• Classroom assessment information must be attached to the LDR.

Classroom /school/ board assessment requirements:

Reading - monitoring statusRequirements: Details: Instructional Level of

Expectation:

At least 2 current Oral Reading Records (ORR)

Unseen fiction/non-fiction texts reflecting curriculum expectations

Accompanying comprehension questions and responses

90%+ accuracy and 80% + with

comprehension questions and retelling

At least 2 current silent reading passages

Unseen fiction/non-fiction texts reflecting curriculum expectations

Accompanying comprehension questions (completed in oral, visual or written presentation)

80% + with comprehension questions and retelling

Writing - monitoring statusRequirements: Details: Instructional Level of

Expectation:

At least 2 current student writing samples

Two different topics or genres Written , revised, edited and

proofread independently

Assessed using an analytic writing rubric

Student performance must be consistent with current classroom expectations

Monitored status

• An LDR is developed. • Complete “to be monitored” in Section C of the

LDR.• Classroom/school/board assessment data is to

be attached to the LDR.• Specific Instructional Practices/ Interventions

need not be completed on the LDR when classroom/school/board assessment data demonstrates that the student meets current curriculum expectations.

For more information on developing LDRs please view the Podcast:

Developing

Literacy Development Records

Using the Results

Schools are responsible for tracking the LDRs and for monitoring support.

Teachers and Principals are expected to:• review LDRs and monitor support• ensure parent/guardian signatures on the LDR • ensure that the completed LDR is kept in the

student’s cum file

Using Your Data: Classroom Level

Teachers are encouraged to use the student reports to identify trends and patterns in literacy development.

Class / grade profiles may be used for classroom instructional decision making and for targeting support to individual students.

Using the Results: School Level

In May, The Minister of Education will release the provincial, board and school results in the Minister’s Report to Parents.

Next Steps:• Print Report for School Administration and Teachers and place them in the

cum file.

• Print Report for Parents/ Guardians and send them home by May 9th.

• Gather classroom based assessment information to support LDR writing.

• Commence LDR writing using the electronic template.

• Arrange a meeting with parents/guardians of students who require an LDR to review results and support.

EXPECTED IMPLEMENTATION DATE: Mid May 2008

• Late May / June: complete the Student Progress and Transition sections of the LDR.

• Print a hard copy of the updated LDR and place it in the cum file.

http://hrsbstaff.ednet.ns.ca/lsp/

For more information contact:

Emilie LivelyHRSB literacy department

elively@hrsb.ns.ca

464-2000 (4431)