Educational Assessment

Stephen Parks, Learning Specialist - Mathematics

Assessment and Evaluation Branch (Anglophone Division)

Department of Education and Early Childhood Development

Atlantic Education International

July 22, 2014

0

0 .2

0 .4

0 .6

0 .8

1 .0

-3 -2 -1 0 1 2 3

b

Ab i l i ty

Pro

bab

ilit

y

Item Characteristic Curve: ITEM 0001

a = 0 .5 3 9 b = 0 .2 0 2

1. Educational Assessment in General

~Balanced Assessment

Large-Scale Assessment

2. The New Brunswick Provincial Assessment Program

~Mission

~Provincial Assessments (mandate, operational standards,

processes)

•Protocols for Accommodations and Exemptions

•Provincial Assessment Schedule

•ELPA/R

•Realignment of Provincial Assessment Program

•Mock Coding Session – G5 Mathematics

TOPICS

© 2005, Dynamic Measurement Group 3

GPS On Where are we?

You are Here

Port

Where do we want to be ?

How do we get there?

How are we doing?

ActualCourse

DesiredCourse

Assessment is the GPS for Educators

10

20

30

40

Dec.Scores

Feb.Scores

Jan.Scores

M archScores

AprilScores

MayScores

JuneScores

60

50

Rea

ding

Aimline

Balanced Assessment

• No single assessment approach is 'better' than another as each serves a slightly different role.

• All types of assessment must be planned with its purpose in mind.

Standardized testing, particularly, is justified as one resource to aid learning and should not be mistaken for a comprehensive verdict on learning or school quality. ~ Peter Grimmett & Paul Shaker

 

New Brunswick Assessment Program (NBAP)

Mission StatementThe mission for the Provincial Assessment and Evaluation Branch is to inform all stakeholders of student achievement by constructing valid, educationally beneficial, evaluations at critical points in the system. The purpose of the New Brunswick Assessment Program (NBAP) is to: determine if students demonstrate the skills and concepts outlined in New Brunswick curriculum documents; report provincially on how well students have achieved provincial standards at given points in their schooling; assist schools, districts, and the province in monitoring student learning and; coordinate the administration of national and international assessments.

New Brunswick Assessment Program

Operational Standards Processes How items are chosen

Standards for Educational and Psychological Testing

The Standards for Educational and Psychological Testing, 2008 edition are a collaborative publication from the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. The intent of the Standards is to promote the sound and ethical use of tests and to provide a basis for evaluating the quality of testing practices.

The New Brunswick Assessment and Evaluation Branch adheres to the Standards when developing, administering, scoring and analysing large scale assessments within the scope of our mandate. 

Operational Standards

Standards for Educational and Psychological Testing

The Assessment Cycle

There are four (4) stages within the overall Assessment Cycle for each of the 13 assessments; the (1) Development Stage, (2) Administration Stage, (3) Data Stage and the (4) Communication Stage.

Within each of the stages, there are several phases which require the expertise from a variety of staff.

Each of these stages require strict adherence to the Standards for Educational and Psychological Testing and the Principles for Fair Student Assessment in Canada.. These standards and principles will be referenced where appropriate.

Processes

0

0 .2

0 .4

0 .6

0 .8

1 .0

-3 -2 -1 0 1 2 3

b

Ab i l i ty

Pro

bab

ilit

y

Item Characteristic Curve: ITEM0001

a = 0 .5 3 9 b = 0 .2 0 2

1 - 7

8 - 14

15 - 21

22 - 28

29 - 35

36 - 42

43 - 48

Matrix Plot of Item Characteristic Curves for Book A

1 - 7

8 - 14

15 - 21

22 - 28

29 - 35

36 - 42

43 - 48

Matrix Plot of Item Characteristic Curves for Book A

ACCOMMODATIONS AND EXEMPTIONS

Provincial assessments are compulsory for all students enrolled in New Brunswick schools.

In certain circumstances, it is necessary and appropriate for students to be exempted from participating in provincial assessments if the testing format does not allow demonstration of their learning, even with accommodations.

Exemptions fall into three broad categories: academic, social/emotional and unforeseen. Typically, academic exemptions are sought for students following an individualized Special Education Plan (SEP-I). Students who are exempt due to an SEP-I should be assessed on those goals that are set out in their Special Education Plan(SEP).

ACCOMMODATIONSAccommodations are those supports, methods and practices that are used in the classroom enabling a student to participate as independently as possible.

There are many ways in which accommodations can be used to support students when they are participating in a provincial assessment. Within the context of provincial assessments, students may require either universal or justifiable accommodation.

For the purpose of large scale assessment, accommodations are alterations that do not jeopardize the integrity or content of the test, yet give students an equal opportunity to demonstrate what they know and can do as part of the assessment process.

Types of Accommodations

UNIVERSAL

Universal accommodations are those strategies, technologies or adjustments that enable a student to reach prescribed outcomes and can be used at the teacher’s or student’s discretion. Universal accommodations do not necessitate an SEP or prior approval for provincial assessments.

JUSTIFIABLE

Justifiable accommodations are documented strategies, technologies or adjustments without which the student would not be able to demonstrate knowledge. These accommodations are documented within the SEP and require prior approval to be accessed during provincial assessments.

EXEMPTIONS

In certain circumstances, it is not appropriate for students to participate in provincial assessments. Any decision to exempt a student from any provincial assessment must be made in collaboration with the student’s parent(s) or guardian(s).

Within the context of provincial assessments, students may require

an exemption for academic, social/emotional or unforeseen reasons.

Types of Exemptions

ACADEMICThe student has been identified with exceptionalities and has a current individualized Special Education Plan (SEP-I) that identifies the need for the requested exemption in the assessed subject area.

Typically, these students are following an SEP-I and are not expected to attain the curricular outcomes and standards. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the SEP-I.

Typically, this constitutes the largest category of exemption requests.

SOCIAL/EMOTIONAL

If the student has been identified with a significant social, emotional or behavioural impairment and cannot be accommodated by such means as alternate location, extra time, etc., an exemption may be sought.

Typically, these students are following an Individual Behaviour Support Plan (IBSP) and are not expected to attain the curricular outcomes and standards at their grade level. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the IBSP.

.

UNFORESEEN

In the event of serious illness, bereavement or other exceptional and unforeseen circumstance, an on-the-spot exemption can be granted.

These situations require that the director of education be contacted immediately..

Provincial Assessment Schedule(International Schools)

• Grade 2 – Reading • Grade 8 – Mathematics• Grade 9 – ELPA (Graduation Requirement)• Grade 11/12 – ELPA/R • Grade 6 – Literacy/Mathematics/Scientific Literacy

English Language Proficiency Assessment – ELPA/R

Realignment of the Provincial Assessment Program

Challenge StatementsSeveral issues are of notable concern in regard to the current Provincial Assessment Program;

•The current regime of provincial assessments has been in place for 10 years and has served its purpose in terms of focusing on early literacy. Large scale assessment is not meant to be ‘terminal’.

•Scientific literacy is recognized nationally and internationally as a critical skill. The Provincial Assessment Program does not include scientific literacy in the current structure.

•Measures of French Language Programs are inconsistent in the current assessment structure. Feedback is provided at the end of Grade 4 for the French Immersion program, but not provided again until Grade 10.

•The current structure of assessment is not cost effective. Administering assessments as single units at the end of a variety of grade levels greatly increases the costs of shipping, handling, data entry and storage.

• There is very little feedback for high schools. Currently the English Language Proficiency Assessment / Reassessment at Grade 9 (based on end of Grade 8 outcomes) and the Provincial Literacy Assessment at Grade 10 French Immersion are the only systemic annual feedback that is provided to high schools. There are no measures in high school for mathematics or scientific literacy.

• The current structure is not well aligned to the national and international measures, specifically the Pan-Canadian Assessment Program (PCAP) as well as the Programme for International Student Assessment (PISA).

• Although it is imperative that trend lines be established, which require at least 3 years of data, assessment is not meant to be ‘terminal’ and need to be responsive to changing priorities.

Recommendation # 1

• To create a New Brunswick Balanced Assessment Framework document incorporating assessment as a holistic concept from formative to benchmarking to large scale and to develop a systemic implementation plan that includes accountability measures.

Discussion

• How do you see your role in supporting schools with a balanced assessment approach?

Recommendation #2To realign the Provincial Assessment Program to correspond with PCAP and PISA as well as respond to the ‘Challenge Statements’.

“New” New Brunswick Assessment Program (NBAP)

Grade 6 (Census) and (10 Field Test)

•Reading, Mathematics and Scientific Literacy (annual, census, administration in May) 

Grade 6 Reading will be by program of instruction; PIF booklets will include a short section in French Reading; Math and Scientific Literacy will include bilingual booklets for students in the FI Program.Grade 10 Math and Scientific Literacy will include all students and booklets will be bi-lingual where appropriate. English literacy will be assessed in the ELPA/R at Grade 9 and will not be included.

Grade 2 Reading (annual, census, administered in May)Grade 2 Reading will be in English to ensure the gains in early literacy are sustained and continued growth is monitored.

Grade 8 Mathematics

Grade 9 ELPA/R (Graduation Requirement)

Concerns1. There was a concern that removing writing from the Provincial Assessment Program would result in a de-emphasis in this area. •As part of the recommendation we have kept writing in the ELPA/R to ensure writing continues to be a high level focus. We will incorporate the writing into the overall score to ensure that the reliability is acceptable.

•As part of a balanced assessment framework, writing will be discussed as a classroom and school priority. Exemplars will be presented along with the standards at the end of each grade level so that teachers are comfortable with assessing at a student level within the context of portfolio assessment.

Development of the Provincial Assessment at Grade 6

Blue PrintAssessment Blue Prints are developed to guide the item development process. The Blue Print may include: (Mathematics Example)

Assessment Type:

Items Per Booklet: Mathematics

Mock Coding Session – G5 Mathematics

Competencies To Be Assessed:

QUESTIONS?