Teacher Malpractice in Assessment: the International Context: Ofqual Ethics Symposium

© UCLES 2015

Teacher Malpractice in

Assessment: the International

Context

Ardeshir Geranpayeh, Ph.D.

Head of Psychometrics & Data Services

Oxford, 26 March 2015

Outline• Introduction

• Cheating (Malpractice):

- What, Where, How, Why, Who

- Consequences: Threat to Test Validity

• Standards and Regulatory Requirements

• How to avoid Cheating:– Prevention vs Detection

– Policies on punishment

• School Collusion: Teacher Malpractice

Cheating in the News

Driving test fraud:

More learner drivers

caught cheating,

BBC News, 26 Jan 2015

Cheating in the News

• 2,440 Chinese students caught cheating

in latest high-tech scam

Chinese state television reported on Sunday

that invigilators detected abnormal radio

signals from an illegal frequency during

national licensing tests for pharmacists in

Shaanxi province. The independent

Wednesday, 28 October 2014

Cheating in the News

According to The Harvard Crimson, a survey

of this year's incoming freshmen revealed

that 42 percent had cheated on high school

homework before arriving at the Ivy League

institution.

The Huffington Post,

18 February 2015

Cheating in the News

More than 50,000 English language tests

taken by overseas students to extend their

British visas have been declared invalid or

questionable as a result of an official

investigation into cheating on a huge scale.

The Guardian, 24 June 2014

Cheating in the News

Cheating in Bihar:

India arrests

Hundreds over Bihar

Blatant Cheating

In Schools

BBC News

21 March 2015

Cheating in the News

'Cheating teachers have lost morality':

Dirty tricks on

the rise to boost

exam grades

MailOnline

13 June 2014

abcNews

abcNews Poll

1 in 3 they

themselves

have cheated.

Rising to 43%

of older teens.

Most say

cheaters don't

get caught.

What is Cheating?

“Any action that violates the rules for

administering a test” Cizek, 1999:3

Where does it happen?

It can take a variety of forms

Pen Scanners

There are YouTube

videos on How to Scan a

Document on the Go

Text Messaging

Examinee can ask

questions and get

answers from friend

during test via text

messaging.

Toilet Breaks!

iPodSong names are renamed

with notes or test answers

for viewing on the screen.

Text files can be stored.

Audio notes can be

stored.

Video notes can be

stored.

Calculator

Notes are entered

into calculators that

have memory for

storing notes.

Camera Phone

Examinee takes

pictures of a test with

a camera phone and

sends picture to

another person who

can text message

correct answers back.

New Technology

Google Glass

Share your

adventures instantly

to Google+, Facebook,

YouTube …

Apple Watch

Why does it happen?

– Material rewards such as

• Access to life chances

• Competitiveness

• Lack of self confidence

• Publication of league tables

(Schools)

– Unintentional Cheating

• Students, material sharing

• Teachers and Schools

Who is involved?

• Maladministration

• Imposters

• Leaked paper / answers

• Prohibited material

• Copying

What is at stake?

– Threat to test validity

– Score obtained by fraudulent means is not

valid

– Has negative impact on the validity of scores

obtained by other candidates

– Denying opportunities to others

– Where cheating is seen to be widespread,

even honestly obtained test results may lose

credibility and certificates become devalued

What is at stake?

• Test Security is now seen as one of the most

important aspects of test Validity

• A Test may not be Valid if it is not secure

Standards for Prevention of

Cheating

Explicit statements in the Standards for

Educational and Psychological Testing

(1999)

Standards for Prevention of

Cheating1. Protect the security of tests (standard 11.7)

2. Ensure that individuals who administer the tests are proficient in administration procedures and understand the importance of adhering to directions provided by the test developer (standard 13.10)

3. Inform examinees that it is inappropriate for them to have someone else take the test, for them to disclose secure test materials, or engage in any other form of cheating (standard 8.7)

Standards for Prevention of

Cheating

4. Ensure that test preparation activities and materials provided to students will not adversely affect the validity of test score inferences (standard 13.11) and

5. Maintain the integrity of test results by eliminating practices designed to raise test scores without improving students, real knowledge, skills, or abilities in the area tested (standard 15.9)

Cheating detection:

Regulatory requirements

Ofqual’s conditions around

malpractice • Ofqual is a risk-based regulator of qualifications

(other than degrees) that sets the standards

that awarding organisations need to meet when

they design, deliver and award regulated

qualifications, via a number of conditions of

recognition.

• Ofqual doesn’t tell AO’s how to manage

malpractice. It tells AO’s what must result from

their behaviour.

Condition A8: Malpractice and

maladministrationCondition A8 of the General Conditions of Recognition

covers:

Preventing malpractice and maladministration

Investigating and managing the effect of malpractice and

maladministration

Procedures relating to malpractice and maladministration

Dealing with malpractice and maladministration

Examples of ‘Positive indicators’The awarding organisation: has ways of working that reduce risks and incidents

occurring

makes sure that individuals involved in the delivery of its qualifications understand and routinely follow these ways of working

takes all reasonable steps to provide information which is not inaccurate or misleading

has policies, practices and/or procedures that reduce the risk of malpractice and maladministration

When Ofqual monitored Cambridge English Language Assessment they were happy with how we are managing malpractice.

Examples of ‘Negative indicators’ Using an individual or Centre to assist with an

investigation when there is a suspicion or allegation that they were connected to, or responsible for, the case being investigated

Not demonstrating steps taken to prevent repeat incidents of previous cases

not following its own procedures during investigation

not properly evaluating or validating evidence collected during investigation

not keeping records/documents relating to the incident for an appropriate period

not being able to enforce action taken against those involved, and cannot demonstrate a good reason why it has been unable to do so

How to avoid Cheating?

How to avoid Cheating?

• The Best practice is “Prevention”

• No Statistical magic can replace good

invigilation

• Tension between Faster/on demand results

and proper data forensic processes

• Use variety of ID checks

– Candidate ID Verification

– Biomedical checks

– Hand writing

Policies on punishment

• Once a cheating is detected, an action has to

be put in place to

• Stop fraudulent use of test results

• Deter future cheaters

• Punishment is dependent on the level of

cheating, which in turn can depend on 5 levels

of cheating detection

• Individual candidates, Group of candidates,

Teacher Malpractice, Test Centre collusion

and Widespread cheating

Level of punishment

• Withdrawing results/certificate (individual)

• Re-taking the exam (suspect results)

• Life Ban (if stakes is high or imposters)

• Informing stake holders (regulator)

• Legal action (insider)

What about School collusion:

Teacher Malpractice

• Students implicated may not have been

involved in the cheating

• Whilst the candidate’s results may be

cancelled if school was to be blamed, no

further action will normally be taken

against candidates

Teacher Malpractice: the Atlanta case

• In 2009, Beverly Hall, Ex-Atlanta school

superintendent, was named national

superintendent of the year largely based

on her district’s improved standardised

test scores

• She was once credited with turning

around some of the city’s lowest-

performing schools

Teacher Malpractice: the Atlanta case

• Biggest US cheating scandal in US History

• Cheating detected on a 2009 standardized

state test involving 178 teachers and

principles, 56 schools investigated cheated,

43 people were indicted

• Prosecutors set out to reveal a culture of fear

and pressure existed under Hall

• Georgia Governor determination to trace its

source

• Cheating traced back to 2001

Teacher Malpractice: the Atlanta case

• The scandal testifies that cheating is no

longer seen as an old-fashioned battle

between teachers and students

• When the stakes are high, teachers would

also be willing to cheat

Final Remarks

• Use new psychometric techniques to

detect school collusion

• Erasure analysis detection

• Multifaceted approach to monitor teachers’

assessment

• Malpractice awareness among teachers

• Collaborative dialogue with teachers as

the guardians of students’ achievement

Thank you for listening.

Geranpayeh.a@CambridgeEnglish.org