2018 Moderation User Guide

Table of Contents

Introduction.............................................................................................................................................2

What is Moderation?................................................................................................................................2

Internal Moderation.................................................................................................................................2

Critiquing Process.....................................................................................................................................3

Verification Process..................................................................................................................................3

Random Selection.....................................................................................................................................3

Internal Moderation Cover Sheet..............................................................................................................3

Forging Collegial Links And Support...........................................................................................................4

Student Exemplars and Other Supporting Documents................................................................................4

External Moderation.................................................................................................................................4

What to Submit for External Moderation...................................................................................................5

External Moderator Reports......................................................................................................................5

Sample Form Template: Response to External Moderation Report.............................................................6

Activities...................................................................................................................................................7

Activity Step 1 – Critiquing a task (first set of eyes)....................................................................................7

Activity Step 2 – Verifying your judgments (second set of eyes)..................................................................8

Activity Step 3 – External Moderation Outcomes.......................................................................................8

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

IntroductionModeration is a complex process that some kaiako need support to thoroughly understand how it works, because the process of external moderation often changes.

This resource (user guide) contains modified information from the NZQA website, for example information has been bullet-pointed to make it easier for kaiako to follow, understand and use.

A supporting power point presentation is available online at: https://goo.gl/NKfDLh

What is Moderation? It is about ensuring assessment materials used by the kaiako have been checked and modified It is about the external moderation of kaiako assessment decisions Secondary moderation happens every year NZQA selects a sample of internally assessed standards for external moderation based on results

reported by the school at the end of the previous year Randomly selected samples of student work are chosen for each selected internally assessed standard

across curriculum areas Selection needs to follow the school's documented random selection process Secondary moderation consists of internal and external moderation Moderation is a requirement of all kura/schools throughout NZ i.e. secondary schools, wharekura,

Kura Kaupapa Māori and private schools based on their NZQA Consent to Assess A kura/school must have NZQA Consent to Assess if they want to assess against standards on the

Directory of Assessment Standards.

Internal Moderation The quality of national assessment relies on quality assurance systems operating in every institution. Internal moderation supports the credibility of assessment by ensuring that assessment is valid and judgments are verifiable. Schools undertake quality assurance each year to ensure that: All assessment materials are critiqued and modified as necessary, before use. While marking, student work is checked to ensure it is authentic, sufficient, valid and verifiable. Judgments are consistent across classes (if more than one class is assessed against the same standard). A sample of each teacher's grade judgments are verified for each standard. Verification is conducted by a subject specialist. Benchmark samples of student work are retained (for Achieved, Merit, Excellence). Assessment materials for each standard are retained with a random selection of student work for the

purpose of external moderation. Kaiako respond to external moderator advice in their reports before the materials are used again. Kaiako use external professional networks to maintain their understanding of the standards.

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Critiquing ProcessThe critiquing process (first set of eyes) involves a teaching peer checking the assessment task before the task is administered with students, ensures the assessment task: focuses on the requirements specified in the standard provides the opportunity for students to present evidence at all grades has an appropriate and applicable context has clear student instructions for students to follow.

Verification Process The verification process (second set of eyes) involves a subject-specialist teaching-peer checking the marking or judgments that have been made against student work. This aims to ensure that kaiako judgments are consistent with the standard, before results are reported to NZQA. A key point is that only a subject-specialist (who has the standard-specific knowledge required to verify the grades) should undertake verification of student grades.

Student samples should be purposefully selected for verification, rather than being selected randomly. You could purposefully select samples of work that are you are unsure about and verging on grade boundaries and that you think need reviewing (e.g. should this student be awarded a merit or an excellence grade?) .

The point of the verification process is to ensure kaiako are satisfied that their decisions are consistent with the standard.

Random Selection For every standard assessed by the school a random selection of student work for external moderation should be made as soon as possible after assessment. Random selection: must be undertaken by someone other than the kaiako who marked the work must use a method that does not allow for prediction must include the work of all students who submitted work for the assessment must follow school policy.

Internal Moderation Cover Sheet Schools must document their internal moderation process and the Internal Moderation Cover Sheet was developed for this purpose. Schools can access the internal moderation cover sheet online (in WORD or PDF format) and adopt or adapt it to guide them through the internal moderation process.

The links to the 2018 Internal Moderation Cover Sheet are: WORD format: http://www.nzqa.govt.nz/assets/Providers-and-partners/Assessment-and-moderation/

internal-moderation-cover-sheet.docx

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

PDF format: http://www.nzqa.govt.nz/assets/Providers-and-partners/Assessment-and-moderation/internal-moderation-cover-sheet.pdf

Forging Collegial Links And Support Kaiako should establish external networks with teaching colleagues in other schools, subject associations and subject advisers to support them with the moderation process. It is recommended that senior management in all schools continue to encourage and, if necessary, facilitate these contacts.

Contact can be maintained (as appropriate) through online social networking, instant messaging, video conferencing or mailing lists, clusters, using Google docs, using Google hangouts and online communities such as VLN, POND, Google+, LinkedIn.

External professional interaction helps kaiako to maintain a current understanding of the standards.

Student Exemplars and Other Supporting Documents Annotated exemplars of TKI resources that can be used to assess against standards are available on the NZQA subject pages: http://www.nzqa.govt.nz/ncea/subjects/

Clarification and Assessment Specification documents for standards provide assistance to increase teacher confidence when making assessment judgments, these can also be found on the NZQA subject pages:http://www.nzqa.govt.nz/ncea/subjects/

Conditions of Assessment are available on the related subject page, accessible via the TKI page called Resources for Internally Assessed Achievement Standards:http://ncea.tki.org.nz/Resources-for-Internally-Assessed-Achievement-Standards

Teachers may seek further clarification on how to interpret a standard by asking specific questions of moderators using the NZQA Request for Clarification form:http://www.nzqa.govt.nz/assets/Providers-and-partners/Assessment-and-moderation/Request-for-clarification-of-an-internally-assessed-standard.docx

External Moderation National external moderation provides an assurance that assessment decisions, in relation to assessment standards, are consistent nationally.

NZQA carries out a school check moderation of internally assessed standards in secondary schools to ensure that:

Assessment judgments (marking of students' work) are at the national standard. The assessment materials used (tasks, activities or tests) are at the national standard.

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Each school has an annual submission date for external moderation. The submission date is confirmed early in the New Year when schools are advised which standards have been selected for moderation.

What to Submit for External Moderation A completed 2018 Secondary Moderation Cover Sheet A copy of the task and any supporting resources. A copy of the assessment schedule. The 8 pieces of student work that have been randomly selected and stored for moderation.

For more information about the selection methods and moderation, refer to:

Guidelines for the Selection of Student Work for Moderationhttp://www.nzqa.govt.nz/assets/Providers-and-partners/Assessment-and-moderation/sec-mod-selection-guidelines.pdf

Assessment (including Examination) Rules for Schools with Consent to Assess (6.5)http://www.nzqa.govt.nz/about-us/our-role/legislation/nzqa-rules/assessment-including-examination-rules-2018/6/5/

External Moderator Reports You can source external moderator reports from the General Access area of your school login. You

will need to complete an Education Provisioning Application (EPA). Talk to your Principal’s Nominee (PN).

Your PN could print it out for you but it is good for you to be able to navigate the General Access features in the school login.

Moderator reports are accessed through the Moderation Results link. Look for your subject and if the standard you’re looking for is hyperlinked, the report is there. Print it off and take heed of the comments moderators have made. Comments should be constructive

not destructive. If the assessment materials need to be modified (M) or do not meet the national standard (X) the

kaiako must address the identified issues using the moderator comments as a guide. If the moderator didn’t need to view the assessment material presented, you will see an (N). It doesn’t

mean Not Achieved. Kaiako must respond to issues identified by the external moderation process. This process must also be documented (See the sample response form below (Sample Response to

External Moderation Report) which can be adapted).

If you require further information about moderation, talk to your Principal’s Nominee (PN).

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

The PN can contact your school’s NZQA School Relationship Manager if further guidance is required.

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Sample Form Template: Response to External Moderation Report

Subject /Marau

Summary of moderation outcomesStandard and Version

Level Materials

N / M / X

Verification X of Y decisions appropriate e.g. 4 / 8

Issues identified by moderator

Wāhanga (Marau) Ako Action Plan (To be completed by kaiako)

Action Responsibility

(Kaiako Name)

Completion Date

(PN)

Sign off when complete

(PN)

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Signed:

Date:

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Activities The following activities take you through the complete quality assurance process. Practice the complete process so that you can get it right! Practice makes perfect!

This activity could be conducted sometime throughout the year and preferably in a teacher cluster group, a small group of individual kaiako, or in your syndicate or department / faculty hui.

You may have assessment materials from previous years that may need to be critiqued or reviewed. Use them to practice on

You may have some student samples stored away securely that haven’t been requested for external moderation. Share them with your cluster so that they can verify your grades

You should be able to access moderator reports from previous years, you can do this via your school login – if you don’t have a password to access the General Features of your school login - talk to your Principal’s Nominee

Refer to the online presentation at https://goo.gl/NKfDLh that confirms the complete quality assurance process outlined in this document.

Activity Step 1 – Critiquing a task (first set of eyes) Before even starting this process, you should have already arranged to connect with a subject-specific colleague within, or external to your kura, to verify each other’s student grades – (for verification purposes).

You can check the assessment task if you are NOT the developer of it. If you do develop your own, you will need to get another subject-specific teacher (preferably) to check it for you prior to use. Most kaiako use the resources available on the MOE/NZQA subject pages. Note: Te Reo Māori Whakarongo tasks available here MUST be modified before they are used with students, because they are publicly available.

1. Print a copy of the Internal Moderation Cover Sheet. 2. Print a copy of an achievement standard relevant to your hōtaka ako (marau). 3. Print the level of Te Marautanga o Aotearoa (TMoA) or the New Zealand Curriculum (NZC) or

Te Aho Arataki Marau (Curriculum Guidelines for Teaching and Learning Te Reo Māori in English- medium Schools Years 1-13) relevant to the level of the standard you have selected.

4. Print an assessment task for the level of the achievement standard you have selected. 5. Proceed to critique the assessment task and always refer to the standard and level of the

curriculum you have selected to ensure you don’t go off track. 6. Remember the achievement standard is derived from the TMoA, NZC or Te Aho Arataki Marau. 7. Use the Internal Moderation Cover Sheet to document this process.

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.

Activity Step 2 – Verifying your judgments (second set of eyes) After you have marked your students work and given them a grade: 1. Refer to the Internal Moderation Cover Sheet again. Mark all student work. 2. Prepare a purposefully selected sample of student work to forward to your subject-specific verifier. 3. List your students A,B,C etc, on the back of the cover sheet and write your grades for your

students. 4. Submit your work to your verifier using the best way for you both e.g.; Google docs, Skype, mail

etc. 5. Once the samples are returned from your verifier, check any comments or difference in grades. 6. If there are any differences in grades discuss these with your verifier and come to a conclusion as

to what grade will be reported to NZQA. 7. Send a randomly selected sample of student work, the assessment task and the assessment

schedule for NZQA external moderation when requested along with the completed Secondary Moderation Cover Sheet.

Activity Step 3 – External Moderation Outcomes When materials are returned from the external moderator and their reports are available: 1. Use the Sample Response Form (provided previously in this user guide) to respond to any issues

that have been identified by the external moderator. (This sample form can be modified by you as you prefer)

2. File all documents pertaining to that one standard in a folder or box, just in case NZQA interviews you during a Managing National Assessment (MNA) review.

3. You may need to show this cover sheet to your Principal’s Nominee to show that you have followed up on the identified issues.

4. This completes the quality assurance process for this standard. 5. You would need to go through this same process for every standard you use to assess your

students.

Produced as part of a Ministry of Education project.© New Zealand Ministry of Education 2018 – copying restricted to use by New Zealand education sector.