Estimating Student Achievement Challenges of making reliable and objective judgements Graduate...

Unit 3Estimating Student Achievement

Challenges of making reliable and objective judgements

Graduate Certificate in the Assessment of Student Learning

2

Reliable & objective judgement challenges

Reliable - dependable, consistent, trustworthy

Objective – impartial, detached, unbiased

Awareness of potential biases or judgement ‘errors’

3

Common ‘errors’ Pre-judging Confusing achievement with effort Different standards different students Cultural stereotyping Gender stereotyping ‘halo’ effect ‘proximity’ error ‘central tendency’ error ‘severity/leniency’ error

4

Prejudging

Letting expectation rule over actual evidence of learning when

A student exceeds our expectations Doesn’t demonstrate the capacity we

expect

To minimise potential to prejudgeAssess each piece of evidence of learning on

its own merits.

5

Confusing achievement with effortEffort does not always equate to ability bigger/longer is not necessarily better Appearance may not match response

quality

To minimise potential to confuse achievement with effort

Assess each piece of evidence of learning on its own merits.

6

Different standards different studentsBuilding tolerance standards applied

‘More able’ students +/- 6mm ‘Average’ students +/- 8mm ‘Less able’ students +/- 10mm

Potentially fraught difference.Might preclude some students meeting

highest standard.

7

Cultural stereotyping

Extensive research available

Klenowski & Gertz (2009) culture fair assessment

Kraiger & Ford (1985) 74 studies race effect, bias to own race

Baker (2005) Potential cultural group sabotage

Perry & Delpit (1998) Potential to dismiss legitimate responses

8

Gender stereotyping Assuming girls and boys will perform at

higher/lower levels than each other

Gender effect on rating

Performance effect due to gender mix

Review items demonstrating gender diff.

9

Other common ‘errors’ Halo effect

◦ One aspect of performance influences rating on another aspect

Proximity error◦ Tendency to give similar rating to criteria located

near each other

‘central tendency’ error◦ Rarely award very high or very low ratings

10

Severity or leniency

Comparability/reliability impacted

Tendency to be hard or easy marker

Congdon & McQueen (2000)

To minimise potential to be severe or lenient assessorAccess to: work samples across levels; moderation;

and, professional dialogue.

11

Common ‘errors’ Pre-judging Confusing achievement with effort Different standards different students Cultural stereotyping Gender stereotyping ‘halo’ effect ‘proximity’ error ‘central tendency’ error ‘severity/leniency’ error

12

Ensure fair and equitable assessment practices

Learning to be established by the evidence

Assessment mantra

13

Participant requirements summary Note potential biases that may exist in your context Consider ways to expose biases Consider ways to minimise biases/ ‘errors’

Share examples of best practice.Initiate and build discussion, explore issues related to

providing fair and equitable assessment and assessment results.

See participant requirements for further details.