S tructured O bjective C linical E xamination SOC E Slide 2 S
tructured O bjective C linical E xamination Slide 3 What is OSCE ?
series of stations with tasks series of stations with tasks planned
planned marking form marking form examiner examiner patient : SP
patient : SP organization > examination organization >
examination Slide 4 Why OSCE ? before OSCE (1975) before OSCE
(1975) viva (oral), long case, short case viva (oral), long case,
short case valid ? know how NOT show how know how NOT show how
reliable ? different patients different patients different
examiners different examiners Slide 5 Why OSCE ? more valid more
valid show how more reliable more reliable same task | patient same
examiner or same structured marking sheet Slide 6 Basic Structure
12348765 Slide 7 12348765 12435687 Slide 8 Basic Structure :
Parallel 12348765 A B CD Slide 9 Basic Structure : Double 2 123
7654 Slide 10 How to start ? blueprint of the whole OSCE blueprint
of the whole OSCE design the station design the station design the
mark sheet design the mark sheet Slide 11 Blueprint reversed table
of classification reversed table of classification Slide 12 Station
& Marking learning by doing learning by doing 8 groups : name
list 8 groups : name list 8 stations 8 stations 2(p) - 4(x) - 4(d)
2(p) - 4(x) - 4(d) signal signal materials & ID materials &
ID task (flexible) task (flexible) lunch : 3rd floor lunch : 3rd
floor Slide 13 Station Design station time 4-15 min. station time
4-15 min. total time < 2 hrs total time < 2 hrs focus task
focus task examiner used : Y | N, who? examiner used : Y | N, who?
pilot pilot Slide 14 Type of Stations static | written static |
written practical : technique practical : technique clinical
clinical Slide 15 Marking Sheet Design checklist checklist rating
scale rating scale score score Slide 16 Checklist dichotomous : Yes
| No dichotomous : Yes | No Pros Pros high objectivity high
reliability easy to feedback Cons Cons only quantity check Slide 17
Checklist How to improve? How to improve? stem clear clear
observable observable not too long not too long overall not too
long not too long Slide 18 Rating Scale rating rating quality
concern quality concern lower objectivity lower objectivity lower
reliability Slide 19 Rating Scale How to improve? How to improve?
3-7 scale more clarification of each scale more raters rater
training common errors of rating scale common errors of rating
scale Slide 20 Rating Scale common errors common errors leniency
error central tendency error halo effect logical error proximity
error contrast error Slide 21 Examiner station developer station
developer non station developer non station developer teacher not
teacher other staff other staff SP SP participation =>
reliability participation => reliability Slide 22 Observation
direct direct indirect indirect one-way mirror monitor video Slide
23 Getting Feedback: How? verbal verbal marked checklist & be
the subject marked checklist & be the subject marked checklist
& watch video marked checklist & watch video printed answer
printed answer relevant papers relevant papers Slide 24 Getting
Feedback : When? during the exam during the exam intra-station in
another station stress? NB: too much information! after the exam
after the exam end of all stations Slide 25 Setting an OSCE
learning by doing | 8 groups learning by doing | 8 groups
structured task | medical student V structured task | medical
student V time time 7 min. test (without feedback) 5 min. test + 2
min. with feedback available tools : pls ask available tools : pls
ask draft of test and marking sheet : ~ 4 p.m. draft of test and
marking sheet : ~ 4 p.m. preparation 8 - 9 a.m. preparation 8 - 9
a.m. Slide 26 Minimal Passing Score criterion-referenced ( )
criterion-referenced ( ) holistic modified Angoff norm-referenced (
) norm-referenced ( ) borderline method relative method Slide 27
Holistic Method medical schools faculty-wide pass mark medical
schools faculty-wide pass mark e.g. 60% e.g. 60% Slide 28 Modified
Angoff Method group of experts group of experts get the OSCE get
the OSCE Think of a group of borderline candidates Think of a group
of borderline candidates decide the passing score decide the
passing score expert discussion is acceptable in original Angoff
expert discussion is acceptable in original Angoff Slide 29 Minimal
Passing Score criterion-referenced ( ) criterion-referenced ( )
holistic Modified Angoff norm-referenced ( ) norm-referenced ( )
borderline method relative method Slide 30 Borderline Method
marking form : checklist + global rating marking form : checklist +
global rating all categorized borderline students all categorized
borderline students mean scores of borderline group mean scores of
borderline group Slide 31 Borderline Method Mean borderline score =
(72+70+80) / 3 = 74 Slide 32 Relative Method 1st method : Wijnen
Mothod 1st method : Wijnen Mothod Passing mark = mean -1.96SE 2nd
method 2nd method 60% of the 95th percentile rank score Slide 33
Minimal Passing Stations criterion-referenced criterion-referenced
Slide 34 Staff & OSCE : Like emotional comfort emotional
comfort validly assess validly assess consistent consistent Slide
35 Staff & OSCE : Dislike too compartmentalized too
compartmentalized no opportunity to observe the complete patient
evaluation of the student no opportunity to observe the complete
patient evaluation of the student repetitive nature => boring
repetitive nature => boring Slide 36 Students & OSCE fairer
than other methods fairer than other methods less stressful less
stressful unsure whether the important aspects tested unsure
whether the important aspects tested Slide 37 Limitations of OSCE
lengthy preparation lengthy preparation need more observational
skill of the staff need more observational skill of the staff
costly costly low inter-station correlation low inter-station
correlation test security? test security? Slide 38 Whats next?
evaluation => learning evaluation => learning summative =>
formative summative => formative Slide 39 Innovation senior
student as SP and examiner in OSCE senior student as SP and
examiner in OSCE study sheet listing Dx that might appear on the
OSCEs study sheet listing Dx that might appear on the OSCEs add
structured oral exam into OSCE add structured oral exam into OSCE
GOSCE GOSCE Slide 40 GOSCE : Group OSCE Pros Pros economy mutual
teaching mutual support opportunity to examine social skill Cons
Cons lack of individual assessment different participants do
different tasks Slide 41 Potential Use of GOSCE formative
assessment formative assessment end-of-course assessment
end-of-course assessment exploring interpersonal relationship
exploring interpersonal relationship teaching method for short
course teaching method for short course Slide 42 Re-using OSCE
stations across rotation in the same academic year across rotation
in the same academic year statistically OK from year to year from
year to year statistically not OK Slide 43 Conclusion : OSCE What ?
What ? stations + tasks + checklist Why ? Why ? more valid, more
reliable How ? How ? How to organize? How to analyze?
