Formative Assessment Case Studies (FACS)

Formative Assessment Case Studies (FACS)

Shekhar Kumta, Jack Cheng, James WareChinese University of Hong Kong

(Refer to CD for FACS engine and sample workshop case study)

Hong Kong International Consortium

What Are FACS?

We have designed a user friendly engine for generating and delivering web-based, interactive, clinical scenarios

These case studies allow accommodation for multimedia features & require formative feedback to facilitate student learning.

RCTs have established the effectiveness of FACS

and students prefer using this learning and assessment model.

FACS engine provides a value added option for administering items from IDEAL’s non restricted (formative) bank.

Clinicians feel FACS provide a convenient means to generate & present a meaningful clinical context for one’s formative items.

FACS enables not only incorporation of, but enhanced use for formative items that are

Vignette-based A type

Vignette-based R- type

Key features based

FACS Strength: enhanced authenticity to the assessment

Evidence: one can assess student learning by testing only some elements that are essential to the resolution of that problem (i.e. the clinical problem’s key features).

Using key features is also an effective way of facilitating student learning (not just the assessment of their learning).

key features based assessments correlate better with problems solving ability and clinical reasoning than patient management problems

Option of using Key Features

How to Select Case Studies for FACS

• Identify areas where students frequently encounter difficulties understanding clinical concepts.

the clinical scenario needs to set a context so that application of knowledge becomes the focus for the virtual problem .

ensure steps within the problem enable strategic formative feedback in order to develop & facilitate understanding.

Workshop Objectives for FACS

Short Term Create a FACS

Write or incorporate existing formative items

Incorporate formative feedback

Intermediate

Install the FACS templates provided on your own or your institution’s computers / servers

Long Term Develop a bank of educationally relevant and validated FACS

for sharing among consortium members

+

Apply to several cases

1. Graduating Level

2. What do interns do?

3. Objectives = assessment

4. Practical issues

5. Consider theoretical issues

Generic Problem

Objectives

+

The Problem and Objectives

Generic problemGeneric problem Scenario Cases

Clinical detailAction List 1Feedback 1

Explanation 1 Action List 2 Feedback 2

etc

The Problem and Objectives

+

Chest pain

1. Clinical features

2. Prioritise management

3. Interpret investigations

4. Include discharge etc..

5. Anatomy of thorax

Generic Problem

Objectives

+

Suggest some notes

The Problem and Objectives

Generic problem ScenarioScenario

Writing the Scenario

INCLUDE:

1. Age and gender

2. Site of care

3. Summary of presentation

a) Keep generic at this stage

b) Onset, duration, severity

c) ~2 – 6 lines

4. Other (e.g., family history)

The Scenario:

• shall always include common info

• realism important

• summary form

Writing the Scenario

The Scenario:

Important, the same stem shall apply to several cases, e.g., pneumothorax and MI etc.

38 year old Chinese male with

a twenty year history of heavy

smoking is admitted to the

A&E Department with sudden

onset of chest pain one hour

before and is apparently

having difficulty in breathing.

Writing the Scenario

Choices of Cases

Generic problem Scenario CasesCases

+

Case 1Create several related cases:

• common scenario

• common cases

• common objectives

+

Case 2

Case 3

+

Case 4 . . .

Choices of Cases

+

Pneumothorax, tensionThe Cases

+

Pneumothorax, simple

Myocardial infarction

+

Acute oesophagitis

Choices of Cases

Scenario Cases Clinical detailClinical detail

Writing Clinical Detail

Case 1: Pneumothorax

Consider site

Usually vital signs

Important info:

a) Specific clinical findings

b) + special tests

Clinical detail

Clinical images

+

Writing Clinical Detail

Clinical detail

With Added value

HR: 120bpmBP: 80/60 mm HgRR: 30 / minT: 37.5o C

Hyper-resonant Rt., absent BS and reduced respiratory movement Rt.,

Writing Clinical Detail

Writing an Action List

Scenario Cases Clinical detail

Action List 1Action List 1

+

Lead-in

1. Brief

2. Likely choices

3. Pick one best response

4. Ensure four or more

Include

Up to “n” possible actions

Writing an Action List

+

What is your next action?Lead-in

Incorrect

Correct choice

Incorrect

+

1.Order chest radiograph

2. Insert a chest drain

+

3. Order a chest CT

Writing an Action List

Scenario Cases Clinical detail

Action List 1Feedback 1 for Feedback 1 for incorrect choicesincorrect choices

Writing Feedback

+

For all incorrect choices

1. Why inappropriate

or

2. Trigger further thought

3. Linkages to resources

4. Another image?

Feedback

Included

Writing Feedback

+

1. Life threatening situation . . .For each feedback that is incorrect give a summarised form of info.

Will offer re-entry assessment after incorrect choices

+

2. Quite incorrect . . .

3. No place for definitive . . .

+

4. Tests unnecessary . . .

Writing Feedback

Scenario Cases Clinical detail

Action List 1Feedback 1 for incorrect choices

Explanation 1 for Explanation 1 for correct choicecorrect choice

Explanation for Correct Choice

REPEATED SEQUENCE

+

Summarised

1. Video of procedure

2. Text for action

or

3. Result (s) of investigation

Explanation

Added value

Explanation for Correct Choice

Scenario Cases Clinical detail

Action List 1Feedback 1 for incorrect choices

Explanation 1 for correct choice

Action List 2Action List 2

Writing an Action List

+

Lead-in

1. Brief

2. Likely choices

3. Pick one best response

4. Ensure four or more

Action List

Up to “n” possible actions

Writing an Action List

+

What is your next action?Lead-in

Partially incorrect

Correct choice

Incorrect

+

1.Order chest radiograph

2. Admit for observation

+

3. Discharge after . . .

Writing an Action List

Scenario Cases Clinical detail

Action List 1Feedback 1 for incorrect choices

Writing Feedback

Explanation 1 for correct choice

Action List 2 Feedback 2 for Feedback 2 for incorrect choicesincorrect choices

+

For all incorrect choices

1. Why inappropriate

or

2. Trigger further thought

3. Linkages to resources

4. Another image?

Feedback

Included

Writing Feedback

+

1. Not an immediate priority . . Don’t forget the further assessment to re-enter at the last action point

+

2. Pt. needs monitoring . . .

3. Too early for discharge . . .

+

4. Review later for need . . .

Writing Feedback

Scenario Cases Clinical detail

Action List 1Feedback 1 for incorrect choices

Explanation 1 for correct choice

Action List 2 Explanation 2 for Explanation 2 for correct choice (s)correct choice (s)

Feedback 2 for incorrect choices

Explanation for Correct Choice

+

Summarised

1. Video of procedure

2. Text for action

or

3. Result (s) of investigation

Conclusion

Added value

Explanation for Correct Choice

+

Use relevant clinical info

1. MCQ format

2. + vignette

3. R type

4. Key features etc . . .

Stem

Re-entry Assessment

Options

+

Scenario Cases Clinical detail

Explanation 1 for correct choice

Action List 2

Explanation for Correct Choice

correct choice

# failed tries decides if exit

Feedback 2 for incorrect choices

incorrect choices

Action List 1

Assessment

CREATING A FACS

DEMONSTRATION