‘Do we need exams?’

Wendy Reid Medical Director HEE

Past – Vice President RCOG

Assessment of doctors....

• Demanded by the public• Required by the regulator• Necessary for the definition of ‘profession’But.....• Opaque methods• No direct input from the public• Examinations are often ‘historical’ not

designed for their present purpose

What is Assessment ?

• A biopsy ofknowledge and skills

“clinical competence”

Critical questions in assessment

• WHY are you doing the assessment?• WHAT are you going to assess?• HOW are you going to assess it?

• HOW WELL is the assessment working?

WHY are you doing the assessment?

• Is its purpose:

– Formative?

– Summative?

Graduation/ PG CertificationGraduation/ PG Certification

In course/ in training feedback In course/ in training feedback

Critical questions

• WHY are you doing the assessment?• WHAT are you going to assess?• HOW are you going to assess it?

• HOW WELL is the assessment working?

WHAT are we testing?

Clinical competence• Knowledge

– factual– applied: clinical reasoning

• Skills– communication– clinical

• Attitudes– professional behaviour

A model of clinical competence

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67.

Knows

Shows how

Knows how

Does

Pro

fess

ion

al a

uth

enti

city

Cognition =knowledge

Behaviour = Skills + attitude

• WHY are you doing the assessment?• WHAT are you going to assess?• HOW are you going to assess it?

• HOW WELL is the assessment working?

Testing formats

Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67.

Knows

Shows how

Knows how

Does

Pro

fess

ion

al a

uth

enti

city

Written/ Computer based assessment

Performance/hands on assessment

Testing formats

Knows

Shows how

Knows how

Does

Knows Factual tests: SBA, SAQ, (EMQ)

Knows how (Clinical) Context based tests:SBA, SAQ, (EMQ)

Shows howPerformance assessment in Vitro: OSCE

DoesPerformance assessment in vivo:WBA eg mini-CEX, DOPs, TBA

Critical questions• WHY are you doing the assessment?• WHAT are you going to assess?• HOW are you going to assess it?

• HOW WELL is the assessment working?

How well is the assessment working?

• Is it valid? • Is it reliable? • Is it doing what it is supposed to be doing?

• To answer these questions, we have to consider the characteristics of assessment instruments

** Define the purpose of the assessment

Characteristics of assessment instruments

• Validity (V)• Reliability (R)• Educational impact (E)• Acceptability (A)• Cost (C)

Specialty Training & Education Programme

5 6 7

Specialist Training Curriculum

CCT

3 4

Full registration

Log Book

Subspecialty 2-3yr

1 2

Advanced Training Modules

Women's HealthModule

1*

Foundation

Annual Review of Competence (ARCP)

2

Basic

Intermediate

Part 1 MRCOG

Exam

Part 2MRCOG

Exam

Curriculum• ‘Run-through’ i.e. Appointed once, progress by

assessment• Iterative 7 years – average doctor takes 9.8 years• First 2 years – basic knowledge, must pass part 1

of exam• Middle 3 years – intermediate, must pass part 2

of exam• Final 2 years – advanced, continue with core work

and learning but add specialist modules

Principles of curriculum

• Competency based• Performance measured• Iterative time – ‘weigh’ points• Transition clearly defined at each stage• Flexibility in advanced training• Generic skills across core• Log book – e-portfolio• Knowledge and application of knowledge tests• Workplace based assessments

Aim of curriculum

• Produce well trained Obstetricians & Gynaecologists ready for consultant posts in the NHS

• Produce doctors with flexibility of career choice, well advised throughout training

• Produce doctors who will advance the care of women

• Re-defined in ‘Tomorrow’s Specialist’ publication 2012

Options during trainingDoctors are allowed to:• Work less than full time (50% or more)• Take time out of the programme to work overseas or

do research (maximum 3 years)• Can move into formal Academic training pathway• ‘Pause’ – personal reasons, Olympics, Maternity leave• Apply for sub-specialty training from end of year 5But...Every doctor does the MRCOG examination

MRCOG Examination

• Any graduate can enter from anywhere in the world, need evidence of medical degree

• Part 1 – test of basic knowledge applied to clinical O&G. Written papers (EMQs, MCQs)

• Part 2- application of knowledge, 2 written elements require pass before OSCE element

• Reviewed in 2013 – new proposal to split part 2 and have oral element as part 3

The MRCOG Overseas Centres

Part 1 Success Rates

Part 2 Success Rates

Why Take the MRCOG?

“It is one of the most highly recognised and well-respected degrees in my country” [India]

“It is a window through which I can have more knowledge and find the chance of training in O&G” [Sudan]

“It would give me the best chance at getting first-world training which I could use to advance the level and quality of health care service provided in my coutry” [Trinidad]

Why Take the MRCOG?

“I wish to have an international degree with expertise in evidence-based medicine, audits and protocols…to serve patients better” [India]

“Passing…means that I have achieved an appropriate level to implement RCOG standards to improve women’s health.” [Saudi Arabia]

“It is a prestigious and well-recognised qualification.” [Pakistan]

Consultant Country of Qualification

© Royal College of Obstetricians and

Gynaecologists

Principles of Assessment

Yes, we need exams

• Public confidence• Professional recognition• RCOG standard• International credibility – for the college nad

for individuals• But they must be fit for purpose, modern,

reflect best educational practice and embrace evidence based techniques