District Assessment Coordinator Meeting May 16, 2007

District Assessment Coordinator Meeting

May 16, 2007

Research & Assessment Division

Test Administration Review

Research & Assessment Division

3

Test Administration Review

Fall Field Test (October 2 – October 27) Mathematics Test for English Language

Learners (MTELL) N = 8,825

Test of Emerging Academic English for Listening and Speaking (TEAELS)

N = 16,012

4

Test Administration Review

March Administration Test of Emerging Academic English (TEAE)

GradesNumber

of Students

3-4 17,120

5-6 15,946

7-8 13,049

9-12 21,355

5

Test Administration Review

March Administration Minnesota Student Oral Language

Observation Matrix (MN SOLOM)

GradesNumber of Students

K-2 19,156

3-12 42,441

6

Test Administration Review

Spring Administration Minnesota Comprehensive Assessment –

Series II (MCA-II) Reading and Mathematics Minnesota Test of Academic Skills (MTAS) Mathematics Test for English Language

Learners (MTELL) MN Writes! Minnesota Comprehensive Assessment –

Series II (MCA-II) Science Field Test

7

Test Administration Review

Spring Administration Minnesota Comprehensive Assessment –

Series II (MCA-II) Reading and Mathematics

Math Reading

3 82,048 76,617

4 84,004 76,147

5 84,725 76,851

6 84,687 77,421

7 87,005 80,081

8 88,652 82,944

11/10 96,582 89,900

Ordered

8

Test Administration Review

Spring Administration Minnesota Test of Academic Skills (MTAS)

Math Reading

3 741 805

4 721 793

5 745 793

6 806 841

7 798 825

8 793 803

10/11 857 820

9

Test Administration Review

Spring Administration Mathematics Test for English Language Learners

(MTELL)3 4,660

4 4,504

5 3,904

6 3,699

7 3,442

8 3,442

11 2,493

Total 26,144

10

Test Administration Review

Spring Administration MN Writes!

62,117 ordered

11

Test Administration Review

Spring Administration Minnesota Comprehensive Assessment – Series II

(MCA-II) Science Field Test 281 districts participating

GradeNumber

of Students

5 29,936

8 33,418

10 29,660

11 6,688

12 1,603

Total 101,305

12

Test Administration Review

Basic Skills Test (BST) and Graduation-Required Assessment for Diploma (GRAD) September BST Reading Retest January/February BST Reading, Mathematics

and Writing Retests Grade 10 BST Written Composition

April Seniors Only BST Reading, Mathematics and Writing

Grade 9 GRAD Written Composition July BST and GRAD Reading, Mathematics and

Writing Retests

Overview of District

Assessment & Accountability Work Plan

Research & Assessment Division

14

District Assessment & Accountability Work Plan

Results Release Calendar Work Plan Overview Posttest Edit Window Discrepancy Reports

15

2007 Results Release

Work Plan Overview

Research & Assessment Division

2007 Posttest Edit Window

May 30, 2007 – June 15, 2007

Dawn Brant and Kaitlynn LewisPearson Educational

Measurement

18

General Information Posttest edits collected via Pearson’s

Schoolhouse website www.ncsschoolhouse.com

Similar process to the Pretest Edit Window that occurred in January 2007

There is one window to do all updates. There will not be a separate AYP update window.

Posttest Edits include information collected in MCA-II - TEAE MTELL - MN SOLOM MTAS - MN Writes!

19

Information Available Posttest Edit Window/Record Change

Instructions for 2007 MN Administrations User’s Guide Available May 29, 2007

WebEx Trainings May 30: 9AM, 11AM, 2PM May 31: 9AM, 11AM, 2PM June 1: 9AM June 4: 2PM

20

Supplemental Information Supplemental reports provided on

Schoolhouse website (under REPORTS tab)

1. Unknown Status of Student Report

2. New Students Report3. Discrepancy Report

IMPORTANT! Please review these reports prior to making student edits

21

Unknown Status of Student Report

The students in this report were on your MARSS submission for December 2006 but no test was submitted.

Use this report to confirm that these students are no longer in your school district. Students who appear on this report and are no longer enrolled in the school (after confirming with MARSS coordinator) should be given a test code of NE (Not Enrolled).

Report is available on May 30 and is a one time only report.

22

New Students Report These students were not in your

December 2006 MARSS submission, but they were in the school district during the testing window and an answer book or electronic test was returned for scoring.

Use the New Students Report to confirm that these students are reported on MARSS with accurate MARSS status start and end dates and submitted to MDE by the July 11, 2007 submission deadline.

Report is available on May 30 and is a one time only report.

23

Discrepancy Report This report can include approximately 30 different

types of discrepancies that indicate something may be in error with this record

You will need to download (from SchoolHouse) the discrepancy definitions – discrepancies will be identified by a number.

Use the Daily Discrepancy Report to verify the correct student information for identified discrepancies.

Update the MARSS system and SchoolHouse so both systems reflect the correct student data for these students.

This report will be re-run nightly

24

Reviewing Reports Will Help To...

Identify students who may have missing answer books

Identify students that need to have a test code applied

Identify student records that may need to be merged

Aid in planning edits ahead of time

25

Editing Student Data Functions within the Posttest Edit Window

Search fields Edit student demographic data NO adding or deleting of students are allowed in

posttest edit window Download Excel Roster – Original and Current

Versions Cannot upload Excel changes to SchoolHouse

Update test codes and accommodation codes Move students within a district (at district level) Merge student records

26

Posttest Edit Search Screen

27

Posttest Edit Screen

28

Posttest Edit Screen

29

Validation Rules SchoolHouse will perform many validations

to help with the accuracy of the edits you are making. Grey/Locked Fields – Certain conditions will

cause the test codes to become grey and not allow edits to be made

Warning Messages – These will alert you that something may be wrong and ask you to confirm. You can still save and continue

Errors – When you try to save, you will get a message that certain fields are missing data or that there is conflict in the information provided

30

Score Codes Score Codes are good indicators who

took each test VS=Valid Score NA=Not Attempted NC=Not Completed

If no Score Code exists, no record was returned or record indicated a test code

Non-editable – reference only

31

Wrong Grade Indicator WG=Wrong Grade Tested Grade did not match Enrolled

Grade Cannot change Tested Grade Enrolled Grade can be changed if incorrect A Wrong Grade (WG) indicator will prevent

you from making any test or accommodation code edits to a student’s record

32

Merging Student Data User’s Guide will have detailed

instructions on merging Indicator on Search Screen

T=Tested Student (Received a student test/record)

NT=Not Tested Student (Did not receive a student document/record)

Rules for Merging Students

33

Merge Rules At most only 2 student records can be merged at a

time Only one merge operation can be done per student Cannot merge 2 “Not Tested” students

For example, 2 students showing on the Unknown Status of Student Report

Cannot merge 2 “Tested” students with the same test/assessment

For example, 2 student records that both tested MCA-II Reading etc.

Most records needing merge will likely be students that appear on both the Unknown Status of Student Report and the New Student Report

34

Merging - Primary Will need to indicate which of the 2

student records you want to merge as primary

The demographic information from the “Primary” will then become the demographic information for both records you are merging

35

Recommended Process

1. Review Reports Use to find students to merge Look for students who need test codes

(i.e. NE-Not Enrolled, ABS-Absent, etc) Potential of student documents not

returned

2. Merge Students3. Update demographic or flag data4. Search for any WG-Wrong Grade

indicators and correct, if necessary

36

Recommended Process (cont.)

5. Use Discrepancy Report to correct all other potential discrepancies

Remember - the Discrepancy Report is run nightly; continue correcting discrepancies until the Discrepancy Report contains no discrepancies

If you do not have any edits, you still need to go in to SchoolHouse and select “Finish Edits”

37

Contact Information 1-800-627-7990 x824 mnmca@pearson.com

38

Questions

Overview of Progress Score

Research & Assessment Division

40

Progress Score What is it?

Measure of a student’s learning over time Allows comparison of an individual

student’s performance from one grade to the next

Other terms used to describe Progress Score Growth Score Vertical Scale Score

41

Why a Progress Score? Why not: 2007 Scale Score - 2006 Scale Score?

Scale scores across grades not on the same metric

Grade

Partially Meets the Standards

Meets the

Standards

Standard Deviation

Math Reading

3 340 350 12.1 19.1

4 440 450 13.4 16.0

5 540 550 14.7 14.7

6 640 650 15.4 13.1

7 740 750 15.6 14.5

8 840 850 15.0 13.7

42

Grade3 4 5 6 7 8

8

8 8

8 8

8 8

8 8

8

Linking Design of MCA-IIs

43

Show Changes in Achievement Across Grades

GradeScale Score Difference

4 450

100

5 550

100

6 650

44

Show Changes in Achievement Across Grades

Example: MCA-II Mathematics: Meets the Standards

GradeScale Score Difference

Progress Score Difference

4 450 3641

100 150

5 550 3791

100 51

6 650 3842

45

MCA-II Mathematics Progress Score Across Grades

3450

3550

3650

3750

3850

3950

4050

3 4 5 6 7 8Grade

Pro

gre

ss S

core

Meets the Standards

46

Change Between Achievement Levels

Grade

Partially Meets the Standards

Meets the Standards

4 440 450

10

5 540 550

10

6 640 650

10

47

Change Between Achievement Levels Not Equal

Example: MCA-II Mathematics

Grade

Partially Meets the Standards

Meets the Standards

Partially Meets the Standards

Meets the Standards

4 440 450 3480 3641

10 180

5 540 550 3550 3791

10 192

6 640 650 3630 3842

10 216

48

MCA-II Mathematics Progress Scores by Achievement Levels

3300

3400

3500

3600

3700

3800

3900

4000

4100

4200

3 4 5 6 7 8Grade

Pro

gre

ss

Sc

ore

Partially Meets the Standards Meets the Standards Exceeds the Standards

49

Progress Score on InsideIndividual Student Report

50

Sample Progress Score onIndividual Student Report

51

Progress Score Characteristics

One scale for grades 3 to 8 Based on vertical linking design using

Item Response Theory (IRT) Range from 2200 to 5200 Not equal intervals between all

grades or across levels Expect individual student progress

scores to increase overtime

52

Progress Score Cautions Use at individual level only Do not aggregate to classroom,

school or district level MDE looking at other methods to

measure school and district level impact on learning

Legislative and Policy

Updates

Research & Assessment Division

54

Legislative and Policy Updates

Status of Testing Policies Bill Science content added for MCA-III GRAD Construction and Operation Value-added Model for Minnesota English Language Proficiency Test MN Peer Review Status

MTELL MTAS

Looking Forward/Comments

Research & Assessment Division

56

Annual Assessment Conference

Register now for theThird Annual MinnesotaAssessment ConferenceThursday, August 9, 2007

www.mngts.orgclick on:

57

Contact us:

mde.testing@state.mn.us