CTLM Moderation Workshop forDietetics

Rita Kizito

04-04-2017

Moderation Context

Agenda

• Main aims of moderation

• Practical guidelines and report formats

• Practice with an example?

• Way forward

http://www.che.ac.za/sites/default/files/publications/CHE_accreditation_criteria_Nov2004_0.pdf

Criterion 6

The different modes of delivery of the programme have

appropriate policies and procedures for internal

assessment; internal and external moderation;

monitoring of student progress; explicitness, validity and

reliability of assessment practices; recording of

assessment results; settling of disputes; the rigour and

security of the assessment system; RPL; and for the

development of staff competence in assessment.

Minimum Requirements(i) The programme has appropriate policies and

procedures in all modes of delivery for:

1. Internal assessment of student learning achievements by

academic staff responsible for teaching a course/module

of the programme in a system that includes internal

moderation.

2. External moderation of students’ learning achievements

by appropriately qualified personnel. Moderators are

appointed in terms of clear criteria and procedures and

conduct their responsibilities in terms of clear guidelines.

3. Monitoring student progress in the course of the

programme.

4. Ensuring the validity and reliability of assessment

practices.

5. Secure and reliable recording of assessment results.

6. Settling of student disputes regarding assessment

results.

7. Ensuring the security of the assessment system,

especially with regard to plagiarism and other

misdemeanours.

8. Development of staff competence in assessment.

ii) There are

appropriate policies

and procedures for RPL,

including the

identification,

documentation,

assessment, evaluation

and transcription of

prior learning against

specified learning

outcomes, so that it can

articulate with current

programmes and

qualifications.

Assessment instruments

are designed for RPL in

accordance with the

institution’s policies on

fair and transparent

assessment.

Terminology - Assessment ( Prof. Gresse 2015)

HEQC Definition: Assessment –

Systematic evaluation of a student’s

ability to demonstrate the acquisition

of the learning goals intended in a

curriculum.

The systematic process of collecting,

analyzing and interpreting student

performance (data) for a variety of

different purposes (Gresse).

Products ( essays, portfolios, exams, forms of evidence etc.…)

Practices ,Processes & Principles

Assessor role

…for each module

Moderation Terminology

Check and verify the reliability, correctness and validity of the examination and marking process of a module.

• Before the assessment /examination• After the assessment/examination

Internal Moderation: undertaken by trained university academic staff members.

2. TYPES OF MODERATION

3. MODERATION PROCESS

Moderator – A

person, apart from

the examiner, that

is appointed by the

institution to be

responsible for

ensuring the

standard of the

examination and its

accompanying

marking framework

and response

exemplars, and for

marking a

representative

sample of

examination

responses.

(HEQC definition)

1. MAIN AIMS OF MODERATION

External Moderation:undertaken by competent, non -university staff members officially appointed by institution.

*Before the

assessment/examination

• Verify that the

assessment

assignments (i.e. the

examination questions

and/or assignments)

adhere to the learning

outcomes.

• Verify that the

assessment

assignments conform to

the required

standards.Age”

After the assessment/examination

• Verify that all assessment products

(answer papers, essays, etc.) have

been marked.

• Verify that the evaluation of the

assessment products has been

conducted in a fair and reasonable

manner ( mark a sample of at least

10% of the assessment products

completely and conduct a summary

review of at least 20% of a further

sample).

• Verify that the system used to

calculate and record marks is reliable

(mark sample).Age”

Terminology – Moderation (continued)

Look at whether:

• the paper will be understood easily by the students,

• the marks are calculated correctly,

• the question numbers are correct,

• there are no language mistakes,

• the paper is in agreement with the outcomes of the module and the

work that was covered,

• the paper is pitched at the correct cognitive level :

60% lower cognitive first year,

50% second year,

40% third year and

20% low cognitive level fourth year,

• the questions are asked in different ways and different types of

questions are included, etc.

Terminology – Moderation (Prof. Gresse)

The moderator writes a report

Activity 1

You are provided with two report forms:• The first one is from the Business School , University of

Edinburgh ( for Internal Moderation)• The second one is from the Law Faculty at the University

of the Western Cape ( for External Moderation)

Analyse the two report forms briefly and answer the following questions:a) To what extent does each form allow for checking

and verifying the reliability, correctness and validity of the examination and marking process ?

b) Which one would you use in your context?c) What would you change ( add or remove?)

•Processes followed in collecting, recording and reporting on evidence collected.

Processes

•Types of Instruments and their potential to collect evidence on student performance

Instruments •Criteria used to make decisions about the evidence

Instruments -Standards

•Evidence that confirms or cannot be used to uphold the assessment results

•

Results

What should be moderated ?Moderators collect evidence of assessor competence by

• re-marking ( checking and verifying the way evidence of

student performance is collected)

• reviewing the assessor’s judgements on the competence

or lack of competence of the students concerned.

This requires some knowledge of :

Moderating the assessment process

The list or narrative you develop will reflect the particular context (discipline), highlight strengths, weaknesses, challenges, etc. There is, therefore, no right or wrong narrative, simply one that is context appropriate.

• Sufficiency of evidence collected• How was the evidence collected?• Exam scripts ( numbers?)• Assessment plans and assessment records • Assessment policies?

Moderating the assessment Instruments

( techniques or devices) used in the collection of evidence (types of questions, essays, MCQs, oral exams, portfolios etc.)

Four common criteria

• Appropriateness• Fairness• Validity• Reliability

Check Prof Gresse’s document (s) on Assessment

Moderating the assessment results

What standards / criteria were used to make

decisions about the evidence?

• Mark schedules

• Rubrics

• Compare memorandum with the assessment

instrument.

• How were marks assigned - were they correctly

calculated?

• Compare the memorandum or rubric used in

judging learner performance with the assessment

criteria provided .

• Moderator decides whether or not the assessor’s

interpretations/judgements were valid, reliable, fair

and consistent.

Activity 2

1. In a group or in pairs, design a rubric that could be used to decide whether or not an assessment ( examination) for a module you are moderating has the required standards. Use the rubric to distinguish between an assessment ( examination) which is satisfactory, which is not satisfactory but can be improved , and one which is below the required standard.

2. Use your rubric to moderate your own assessment ( or examination).

Way Forward ?

Please direct any questions to

CTLM

Rita Kizito

E-mail : Rita.Kizito@nmmu.ac.za

Tel : 041 504 3418

THANK YOU

Some useful resources

ALTC Moderation for Fair Assessment in Transnational Learning and

Teaching Project (2008-2010) .Moderation of Assessment in Higher

Education: A Literature Review. Available at

https://intranet.ecu.edu.au/__data/assets/pdf_file/0019/504253/Moderation-

of-assessment-in-higher-education-A-literature-review.pdf

Department of Education ( 2008). Moderate assessment. Available at :

http://www.oerafrica.org/FTPFolder/Moderate_Assessment.pdf

Bloxham, S., & Boyd, P. (2007). Developing Effective Assessment In Higher

Education: A Practical Guide: A Practical Guide. McGraw-Hill Education (UK).