Brunel University Briefing for External Examiners Spring 2013

Brunel UniversityBrunel University

Briefing for External Briefing for External ExaminersExaminersSpring 2013Spring 2013

Derek MilliganDerek Milligan

Director of Academic Programme DevelopmentDirector of Academic Programme Development

TopicsTopics

Panels of ExaminersPanels of Examiners Boards of ExaminersBoards of Examiners Mitigating Circumstances PanelsMitigating Circumstances Panels RegulationsRegulations

Panels and Boards are responsible Panels and Boards are responsible directly to Senatedirectly to Senate

Senate Regulation 4 governs Panels of Senate Regulation 4 governs Panels of Examiners, Boards of Examiners and Examiners, Boards of Examiners and Mitigating Circumstances PanelsMitigating Circumstances Panels

Panels and Boards of ExaminersPanels and Boards of ExaminersCombined PurposeCombined Purpose

To ensure:To ensure:Fairness to all students and each Fairness to all students and each

studentstudentRigour of assessmentRigour of assessmentRobustness of the results and Robustness of the results and

decisionsdecisionsMaintenance of StandardsMaintenance of Standards

Assessment WaypointsAssessment Waypoints

Specify assessment tasks and criteriaSpecify assessment tasks and criteria Approval of tasks and criteriaApproval of tasks and criteria Students takeStudents take Mitigating circumstancesMitigating circumstances Marking/GradingMarking/Grading Moderation (Internal and External)Moderation (Internal and External) Panels of ExaminersPanels of Examiners Boards of ExaminersBoards of Examiners

Panels of Examiners - PurposePanels of Examiners - Purpose

To verify and confirm the marks/grades for each To verify and confirm the marks/grades for each module/blockmodule/block

Not to look at individual students (unless error or Not to look at individual students (unless error or additional info presented)additional info presented)

May make specific recommendations to Boards May make specific recommendations to Boards concerning particular assessmentsconcerning particular assessments

Implement Senate Regulations 4.52-4.66Implement Senate Regulations 4.52-4.66

Mitigating Circumstances Panels are different Mitigating Circumstances Panels are different – – see latersee later

Senate Regulation 4.64Senate Regulation 4.64

A Panel of Examiners shall not confirm grades/marks for an A Panel of Examiners shall not confirm grades/marks for an assessment block until it is satisfied with the integrity and fairness of assessment block until it is satisfied with the integrity and fairness of the assessment(s) leading to the grades/marks. Where the Panel of the assessment(s) leading to the grades/marks. Where the Panel of Examiners has insufficient confidence in the integrity and fairness of Examiners has insufficient confidence in the integrity and fairness of the outcomes of an assessment, it shall take appropriate action in the outcomes of an assessment, it shall take appropriate action in order to achieve sufficient confidence. The Panel of Examiners may order to achieve sufficient confidence. The Panel of Examiners may require the reconsideration by assessors of the grades/marks for the require the reconsideration by assessors of the grades/marks for the complete cohort of students taking an assessment. Only in complete cohort of students taking an assessment. Only in exceptional circumstances may the Panel directly adjust the exceptional circumstances may the Panel directly adjust the grades/marks for a particular assessment and must then must record grades/marks for a particular assessment and must then must record the justification and rationale for the adjustment. Grades/marks for the justification and rationale for the adjustment. Grades/marks for an individual student may not be adjusted, unless they have been an individual student may not be adjusted, unless they have been wrongly recorded or additional information is presented.wrongly recorded or additional information is presented.

Role of External Examiners at PanelsRole of External Examiners at Panels Full membership of the PoE (including voting)Full membership of the PoE (including voting) Contribute to the collective academic decision-makingContribute to the collective academic decision-making Have a distinct influence on the PoE (through their Have a distinct influence on the PoE (through their

independence and their overview)independence and their overview) EEs are not empowered to change marks or grades – all EEs are not empowered to change marks or grades – all

decisions are made collectivelydecisions are made collectively May ask to inspect any documents involved in the May ask to inspect any documents involved in the

assessment processassessment process See SR4.122 – 4.132See SR4.122 – 4.132

Issues for PanelsIssues for Panels

Potential Assessment Design Errors Potential Assessment Design Errors Academic JudgmentAcademic Judgment Moderation effectivenessModeration effectiveness Extraordinary factorsExtraordinary factors Adjustment of ResultsAdjustment of Results Justification of ActionsJustification of Actions Recording of Decisions at all stages!Recording of Decisions at all stages!

Design ErrorsDesign Errors

Potential Locations:Potential Locations:Learning Outcomes CoverageLearning Outcomes CoverageElement Weightings Element Weightings Assessment SpecificationAssessment SpecificationSub-element combination methodSub-element combination methodAssessment Criteria SpecificationAssessment Criteria SpecificationMarking/Grading strategyMarking/Grading strategyModeration processModeration process

Academic JudgementAcademic JudgementThe vast majority of the collective The vast majority of the collective

academic judgement is exercised in academic judgement is exercised in PANELS, not in Boards, of ExaminersPANELS, not in Boards, of Examiners

Boards are more about judgement by Boards are more about judgement by academics, rather than academic academics, rather than academic judgement – but there may be judgement – but there may be extraordinary academic judgements extraordinary academic judgements neededneeded

Boards of Examiners - PurposeBoards of Examiners - Purpose

To receive confirmed marks/grades from PanelsTo receive confirmed marks/grades from Panels To consider individual student profile of achievementTo consider individual student profile of achievement To take into account mitigating circumstances and To take into account mitigating circumstances and

determine appropriate actionsdetermine appropriate actions To decide on progression and re-assessmentTo decide on progression and re-assessment

ororTo recommend to Senate appropriate awards for each To recommend to Senate appropriate awards for each studentstudent

Implement Senate Regulations 4.52-4.58 and 4.67-4.80Implement Senate Regulations 4.52-4.58 and 4.67-4.80NB Boards now have few powers of discretion within the NB Boards now have few powers of discretion within the regulations (compared with the old regulations) – regulations (compared with the old regulations) – but all but all decisions must still be justifieddecisions must still be justified

Role of External Examiners at BoardsRole of External Examiners at Boards Full membership of the BoE (including voting)Full membership of the BoE (including voting) Contribute to the collective academic decision-makingContribute to the collective academic decision-making Have a distinct influence on the BoE (through their Have a distinct influence on the BoE (through their

independence and their overview)independence and their overview) EEs are not empowered to change marks or EEs are not empowered to change marks or

grades/progression/classification unilaterally – all grades/progression/classification unilaterally – all decisions are made collectivelydecisions are made collectively

Assure themselves that due process followed and Assure themselves that due process followed and appropriate consideration and decisions have been appropriate consideration and decisions have been made by the BoEmade by the BoE

See Senate Regulations SR4.122 – 4.132See Senate Regulations SR4.122 – 4.132

Issues for BoardsIssues for Boards

Mitigating CircumstancesMitigating Circumstances Academic JudgmentAcademic Judgment Progression and Re-assessmentProgression and Re-assessment AwardsAwards Extraordinary Decisions - DiscretionExtraordinary Decisions - Discretion Justification of DecisionsJustification of Decisions Recording of Decisions at all stages!Recording of Decisions at all stages!

Condonation, Set-Aside & Borderlines (UG pre-2009 Condonation, Set-Aside & Borderlines (UG pre-2009 only)only)

Board DiscretionBoard DiscretionPotential Locations:Potential Locations: Action re MCsAction re MCs Treatment of AP(E)L – NB New processes for Treatment of AP(E)L – NB New processes for

2013/4 will obviate need for BoE discretion2013/4 will obviate need for BoE discretion Extraordinary situationsExtraordinary situations Re-assessment (mainly PG)Re-assessment (mainly PG)

Condonation/Set-Aside/Promotion across Condonation/Set-Aside/Promotion across boundaries/Re-assessment (UG Old SR2 only)boundaries/Re-assessment (UG Old SR2 only)

Mitigating Circumstances PanelsMitigating Circumstances Panels Report to Board of ExaminersReport to Board of Examiners Consider MC submissions from students Consider MC submissions from students Have no access to academic results when considering Have no access to academic results when considering

casescases Consider the likely impact on studies/assessmentConsider the likely impact on studies/assessment Accept or reject MCsAccept or reject MCs Make decisions directly re coursework submission Make decisions directly re coursework submission

deadlines with MCsdeadlines with MCs All other accepted MCs -> Board of ExaminersAll other accepted MCs -> Board of Examiners MC Panel will identify serious cases to the BoEMC Panel will identify serious cases to the BoE SR4 governs treatment of MCs + see Guidance on MCsSR4 governs treatment of MCs + see Guidance on MCs

Actions re MCs by BoardsActions re MCs by Boards

See Senate Regulations 4.43 – 4.51See Senate Regulations 4.43 – 4.51 Essentially, Boards can take any appropriate Essentially, Boards can take any appropriate

action it sees fit except:action it sees fit except: The marks/grades for individual assessment The marks/grades for individual assessment

elements (e.g. exam script, piece of coursework, elements (e.g. exam script, piece of coursework, etc) may not be changed – i.e., the actual level etc) may not be changed – i.e., the actual level of performance recorded for an individual of performance recorded for an individual assessment (confirmed by a PoE) cannot be assessment (confirmed by a PoE) cannot be altered.altered.

Grounds for Appeal (SR6)Grounds for Appeal (SR6) 6.5 A student may appeal against the decision of a Board of 6.5 A student may appeal against the decision of a Board of

Examiners on any of the following grounds:Examiners on any of the following grounds:

aa that there exist circumstances materially affecting the student's that there exist circumstances materially affecting the student's performance which were not known to the Board of Examiners when performance which were not known to the Board of Examiners when its decision was taken and which it was not reasonably practicable for its decision was taken and which it was not reasonably practicable for the student to make known to the Board beforehand;the student to make known to the Board beforehand;

bb that there were procedural irregularities in the conduct of the that there were procedural irregularities in the conduct of the examinations and/or assessment procedures, including assessment examinations and/or assessment procedures, including assessment of coursework, of such a nature as to create a reasonable possibility of coursework, of such a nature as to create a reasonable possibility that the result might have been different had they not occurred;that the result might have been different had they not occurred;

cc that there is evidence of prejudice, bias or inadequate that there is evidence of prejudice, bias or inadequate assessment on the part of one or more Examiners.assessment on the part of one or more Examiners.

6.6 No appeal shall be allowed on the grounds that, although the 6.6 No appeal shall be allowed on the grounds that, although the decision of the Examiners was properly made, the Board of decision of the Examiners was properly made, the Board of Examiners is alleged to have erred in its judgement of the academic Examiners is alleged to have erred in its judgement of the academic standard achieved by the student.standard achieved by the student.

DocumentationDocumentation

For all summative assessment there should be a For all summative assessment there should be a complete documented audit trail for all complete documented audit trail for all assessments –assessments –

all the way from specification of an individual piece of all the way from specification of an individual piece of assessment and its checkingassessment and its checking

through marking and moderationthrough marking and moderation to Panel and Board decisionsto Panel and Board decisions

Senate RegulationsSenate RegulationsUndergraduate Programmes & AwardsUndergraduate Programmes & Awards SR2 (2009 onwards)SR2 (2009 onwards) SR2 (pre-2009 entry, very few students now) SR2 (pre-2009 entry, very few students now)

Postgraduate Programmes & AwardsPostgraduate Programmes & Awards SR3 (2006 onwards)SR3 (2006 onwards) SR3 (pre-2006 entry, very few students now)SR3 (pre-2006 entry, very few students now)

Assessment in Taught ProgrammesAssessment in Taught Programmes SR4 (combined)SR4 (combined)

Old UG (pre-2009) RegulationsOld UG (pre-2009) Regulations

Progression and Awards based on % average + Progression and Awards based on % average + gaining of 360 creditsgaining of 360 credits

Compensatory mechanismsCompensatory mechanisms Condonation (at BoE discretrion)Condonation (at BoE discretrion) Set-aside (at BoE discretion)Set-aside (at BoE discretion) Best 100 creditsBest 100 credits

Trailing allowedTrailing allowed No limit but much BoE discretion on volume of No limit but much BoE discretion on volume of

re-assessmentre-assessment

Current Regulations (SR2/3)Current Regulations (SR2/3) Transparency of Standards of AwardsTransparency of Standards of Awards Equity of TreatmentEquity of Treatment University Grade Descriptors are Primary Standards University Grade Descriptors are Primary Standards

ReferenceReference Grades are primary performance measureGrades are primary performance measure Defined minimum grade profile requirements for awardsDefined minimum grade profile requirements for awards Credit is not “awarded” – credit is solely a Credit is not “awarded” – credit is solely a

volume/weighting metricvolume/weighting metric ““Core” assessments can be defined (must be passed)Core” assessments can be defined (must be passed) Reassessment volume right = limit (no UG BoE Reassessment volume right = limit (no UG BoE

discretion, but some at PGT currently)discretion, but some at PGT currently)

Current UG (2009) RegulationsCurrent UG (2009) Regulations De-modularised Regulations (Assessment Blocks & De-modularised Regulations (Assessment Blocks &

Study Blocks)Study Blocks) 17-point grade scale (A*,A+,A,A-,…F) with % mark 17-point grade scale (A*,A+,A,A-,…F) with % mark

equivalence definedequivalence defined Progression and (Threshold) Award based on Grade Progression and (Threshold) Award based on Grade

Profile over defined volume of CreditProfile over defined volume of Credit Classification based on combination of GPA + volume in Classification based on combination of GPA + volume in

classclass No compensation/condonation mechanismsNo compensation/condonation mechanisms No Trailing allowed (Except for some cases of MCs and No Trailing allowed (Except for some cases of MCs and

with individual Senate approval)with individual Senate approval) Integration of Sandwich placements into degreeIntegration of Sandwich placements into degree

UG Integration of Sandwich UG Integration of Sandwich Placements (SR2 2009-, 2.46-48)Placements (SR2 2009-, 2.46-48)

Special award name (defined in the Programme Specification) Special award name (defined in the Programme Specification) available for sandwich students who have passed (D-) the available for sandwich students who have passed (D-) the placement assessment.placement assessment.

For this special award the placement module grade is included in For this special award the placement module grade is included in the Level 2 grade profile for award.the Level 2 grade profile for award.

Percentage contribution to Level 2 of the placement module is Percentage contribution to Level 2 of the placement module is defined in the Programme Specification.defined in the Programme Specification.

All students who have failed the placement module (or who choose All students who have failed the placement module (or who choose not to have it included in the award grade profile) are eligible for the not to have it included in the award grade profile) are eligible for the

standard (non-sandwich) awardstandard (non-sandwich) award.. 2012/13 is first year of awards with integrated sandwich 2012/13 is first year of awards with integrated sandwich

placementsplacements

UG Grade ScaleUG Grade ScaleIndicativeMark Band

DegreeClassEquivalent

Grade Grade Point

90 and above 1 A* 17

80-89 1 A+ 16

73-79 1 A 15

70-72 1 A- 14

68-69 2.1 B+ 13

63-67 2.1 B 12

60-62 2.1 B- 11

58-59 2.2 C+ 10

53-57 2.2 C 9

50-52 2.2 C- 8

48-49 3 D+ 7

43-47 3 D 6

40-42 3 D- 5

38-39 Fail E+ 4

33-37 Fail E 3

30-32 Fail E- 2

29 and below Fail F 1

Current PGT (2006) RegulationsCurrent PGT (2006) Regulations

Progression and (Threshold) Award based on Grade Progression and (Threshold) Award based on Grade Profile over defined volume of CreditProfile over defined volume of Credit

Masters Pass, Merit, Distinction ClassificationsMasters Pass, Merit, Distinction Classifications Classification based on minimum grade profile for classClassification based on minimum grade profile for class No compensation/condonation mechanismsNo compensation/condonation mechanisms SR3 for 2013/4 onwards will provide alignment with SR3 for 2013/4 onwards will provide alignment with

current SR2current SR2 De-modularised Regulations (assessment & study blocks)De-modularised Regulations (assessment & study blocks) Finer grade scaleFiner grade scale Use of GPA for ClassificationUse of GPA for Classification

PGT ClassificationPGT Classification

Masters Distinction minimum profile:Masters Distinction minimum profile:

120 cr A (incl Dissertation) + 30 cr B + 30 cr C120 cr A (incl Dissertation) + 30 cr B + 30 cr C Masters Merit minimum profile :Masters Merit minimum profile :

120 cr B (incl Dissertation) + 60 cr C120 cr B (incl Dissertation) + 60 cr C Masters Pass minimum profile :Masters Pass minimum profile :

120 cr C (incl Dissertation) + 60 cr D120 cr C (incl Dissertation) + 60 cr D PGDip minimum profile : 60 cr C + 60 cr DPGDip minimum profile : 60 cr C + 60 cr D PGCert minimum profile : 30 cr C + 30 cr DPGCert minimum profile : 30 cr C + 30 cr D

Old UG (pre-2009) ClassificationOld UG (pre-2009) Classification L3:L2 weighting 2:1L3:L2 weighting 2:1 % Average over best 100 credits at L2 & L3 + 360 credits % Average over best 100 credits at L2 & L3 + 360 credits

awardedawarded 33rdrd: 40% average: 40% average 2.2: 50% average2.2: 50% average 2.1: 60% average2.1: 60% average 11stst: 70% average: 70% average

Variable BordelinesVariable Bordelines Condonation and set-aside used to award credit when Condonation and set-aside used to award credit when

module not passed – at BoE discretionmodule not passed – at BoE discretion Please note that very few, if any, of Bachelors awards at Please note that very few, if any, of Bachelors awards at

2012/13 BoEs will be under these regulations2012/13 BoEs will be under these regulations

Current UG (2009-) ClassificationCurrent UG (2009-) Classification L3:L2 weighting 2:1 as beforeL3:L2 weighting 2:1 as before Two-stage classification decisionTwo-stage classification decision Test 1 – limits of sub-threshold credit volume Test 1 – limits of sub-threshold credit volume

(determines eligibility for 3(determines eligibility for 3rdrd)) Test 2 – Minimum GPA + In-class Volume Test 2 – Minimum GPA + In-class Volume

(determines classification)(determines classification)

Please note that almost all Bachelors awards at Please note that almost all Bachelors awards at 2012/2013 BoEs will be under these regulations2012/2013 BoEs will be under these regulations

Current UG Classification – Test 1Current UG Classification – Test 1

Maximum credit volume of assessment blocks

containing core assessments below D-

Maximum (non-weighted) credit volume of non-core Grade Band E (E+,E,E-)

Maximum credit volume of non-core Grade F

Level 2 Level 3 Level 2 + 3 Level 2 Level 3

1st 0 0 0 0 0 0

2.1 0 20 0 20 0 0

2.2 0 40 20 40 0 0

3rd 0 40 40 60 0 0

Current UG Classification – Test 2Current UG Classification – Test 2 Class

1st 2.1 2.2 3rdVolume of grades (weighted) in Class or better

Minimum weighted GPA

At least 33% of grades (weighted) in Class or better

14.5 11.5 8.5 NA


13.5 10.5 7.5 NA


12.5 9.5 6.5 NA


11.5 8.5 5.5 NA


10.5 7.5 4.5 NA


NA NA NA 3.3

Classification Example 1Classification Example 1 Level 2 Profile (20 cr blocks): BBLevel 2 Profile (20 cr blocks): BB--CCCC++DE DE (E non-(E non-

core)core)

Level 2 GPA: Level 2 GPA: 12+11+9+10+6+3/6= 12+11+9+10+6+3/6= 8.58.5 Level 3 Profile (20 cr blocks): AALevel 3 Profile (20 cr blocks): AA--BBBB--CDCD Level 3 GPA: Level 3 GPA: 15+14+12+11+9+6/6= 15+14+12+11+9+6/6= 11.1611.16

Test 1: Test 1: No core credit below D- = 0 No core credit below D- = 0 Grade F credit = 0 Grade F credit = 0 Non-core credit in E band = 20 at Level 2 Non-core credit in E band = 20 at Level 2

=> => maximum class = 2.1maximum class = 2.1Test 2:Test 2: Overall weighted GPA = (8.5+(11.16x2)) /3= Overall weighted GPA = (8.5+(11.16x2)) /3=

10.2710.27Consider for 2.1 with weighted GPA = Consider for 2.1 with weighted GPA = 10.2710.27 Needs 50% in 2.1 Class to get a 2.1Needs 50% in 2.1 Class to get a 2.1 Proportion of Proportion of weighted grade B- weighted grade B- or better = or better =

1/3 at L2 and 2/3 at L3 = (1/3 +2/3 +2/3)/3 = 1/3 at L2 and 2/3 at L3 = (1/3 +2/3 +2/3)/3 = 55.55%55.55% => => Classification is 2.1Classification is 2.1

Maximum credit volume of

assessment blocks

containing core

assessments below D-

Maximum (non-weighted) credit volume of non-core Grade

Band E (E+,E,E-)


Level 2

Level 3

Level 2 + 3

Level 2

Level 3

1st 0 0 0 0 0 0

2.1 0 20 0 20 0 0

2.2 0 40 20 40 0 0

3rd 0 40 40 60 0 0

Class

1st 2.1 2.2 3rd

Volume of grades (weighted) in Class or better



14.5 11.5 8.5 NA


13.5 10.5 7.5 NA


12.5 9.5 6.5 NA


11.5 8.5 5.5 NA


10.5 7.5 4.5 NA


NA NA NA 3.3

Grade

Grade

Point

A* 17

A+ 16

A 15

A- 14

B+ 13

B 12

B- 11

C+ 10

C 9

C- 8

D+ 7

D 6

D- 5

E+ 4

E 3

E- 2

F 1

Classification Example 2Classification Example 2 Level 2 Profile (20 cr blocks): BBLevel 2 Profile (20 cr blocks): BB--CCCC++DE DE (E non-(E non-

core)core)

Level 2 GPA: Level 2 GPA: 12+11+9+10+6+3/6= 12+11+9+10+6+3/6= 8.58.5 Level 3 Profile (20 cr blocks): ABCDDDLevel 3 Profile (20 cr blocks): ABCDDD Level 3 GPA: Level 3 GPA: 15+12+9+6+6+6/6= 15+12+9+6+6+6/6= 9.09.0

Test 1: Test 1: No core credit below D- = 0 No core credit below D- = 0 Grade F credit = 0 Grade F credit = 0 Non-core credit in E band = 20 at Level 2 Non-core credit in E band = 20 at Level 2

=> => maximum class = 2.1maximum class = 2.1Test 2:Test 2: Overall weighted GPA = (8.5+(9.0x2)) /3= Overall weighted GPA = (8.5+(9.0x2)) /3= 8.838.83Consider for 2.1 with weighted GPA = Consider for 2.1 with weighted GPA = 8.838.83 Needs 58% in 2.1 Class to get a 2.1Needs 58% in 2.1 Class to get a 2.1 Proportion of Proportion of weighted grade B- weighted grade B- or better = or better =

1/3 at L2 and 1/3 at L3 = (1/3 +1/3 +1/3)/3 = 1/3 at L2 and 1/3 at L3 = (1/3 +1/3 +1/3)/3 = 33.33%33.33% => => 2.1 not achieved2.1 not achieved

Consider for 2.2 with weighted GPA = Consider for 2.2 with weighted GPA = 8.838.83 Needs 33% in 2.2 Class to get a 2.2Needs 33% in 2.2 Class to get a 2.2 Proportion of Proportion of weighted grade C- weighted grade C- or better = or better =

2/3 at L2 and 1/2 at L3 = (2/3 +(1/2 +1/2))/3 = 2/3 at L2 and 1/2 at L3 = (2/3 +(1/2 +1/2))/3 = 55.55%55.55%=> => Classification is 2.2Classification is 2.2

Maximum credit volume of

assessment blocks

containing core

assessments below D-

Maximum (non-weighted) credit volume of non-core Grade

Band E (E+,E,E-)


Level 2

Level 3

Level 2 + 3

Level 2

Level 3

1st 0 0 0 0 0 0

2.1 0 20 0 20 0 0

2.2 0 40 20 40 0 0

3rd 0 40 40 60 0 0

Class

1st 2.1 2.2 3rd

Volume of grades (weighted) in Class or better



14.5 11.5 8.5 NA


13.5 10.5 7.5 NA


12.5 9.5 6.5 NA


11.5 8.5 5.5 NA


10.5 7.5 4.5 NA


NA NA NA 3.3

Grade

Grade

Point

A* 17

A+ 16

A 15

A- 14

B+ 13

B 12

B- 11

C+ 10

C 9

C- 8

D+ 7

D 6

D- 5

E+ 4

E 3

E- 2

F 1

Documents

Brunel University Briefing for External Examiners Spring 2013