Upload
kaipo
View
54
Download
1
Embed Size (px)
DESCRIPTION
Writing Multiple Choice Questions. Amy S. Oxentenko, MD, FACP, FACG, AGAF Mayo Clinic, Rochester. Disclosures. Relevant Relationships: Consulting work for MKSAP 14-17 On the IM-ITE exam committee. Outline/Objectives. Define the test taker level before writing - PowerPoint PPT Presentation
Citation preview
Writing Multiple Choice Questions
Amy S. Oxentenko, MD, FACP, FACG, AGAFMayo Clinic, Rochester
• Relevant Relationships:
Consulting work for MKSAP 14-17
On the IM-ITE exam committee
Disclosures
Outline/Objectives
• Define the test taker level before writing
• List components of a multiple-choice question Objective, stem, lead line, options Annotated description of answers
• Evaluate your questions
Who Are You Writing For?
• Med school GI pathophysiology course? More details; physiology on USMLEs
• Resident end-of-rotation exam? Presentation, testing, management
• GI Fellow board review course? More advanced understanding of above
• IM or GI CME course? Very general vs very specific
What are the Stakes of the Questions?
• Board certification?
• Passing a class/rotation?
• Simple knowledge assessment?
• Solely for interaction?
The Stakes
ReproducibleAccurate
Learning
What Should Be Tested?
• For a clerkship/CME course Match goals/objectives
• For a standardized exam Match the targeted learner level
How Often Will Learner Encounter Information
• Frequent More relevant to test often Atypical manifestations
• Infrequent Test the “can’t miss” things Life threatening Typical manifestations
Types of Questions
• Single best answer With clinical scenario
• Factoid questions• All of the following EXCEPT• Matching• True/False• A and B, A and C, all of the above
What are the Components of a Multiple Choice Question (MCQ)?
STEM
OPTIONS
LEADLINE
Question Components
EXPLANATION
OBJECTIVEANSWER
Question Components
PLEASE TAKE THE PRETEST.
Steps to Writing the MCQ
1. Determine the objective
2. Create the lead line and options
3. Draft the stem
4. Write the explanation, if required
The Objective
Example: Objective
• Recall alcoholic hepatitis
What is wrong with this?
Example: Objective
• Treat a patient with alcoholic hepatitis with corticosteroids in the absence of contraindications
More focused
The Objective
• Succinct sentence• Specifies a clinical skill to be learned• Uses action verbs to delineate a goal
Recognize, diagnose, treat, manage• Avoid vague action verbs
Remember, recall, know• Allows you to match curriculum
VERB+
TOPIC+
MODIFIER
Objective Format
Action Verbs for Objectives
Examples of Objectives:The Good, the Bad, the Ugly
• Good: Diagnose niacin deficiency in a patient
with carcinoid syndrome• Bad:
Recall features of niacin deficiency• Ugly:
Understand niacin deficiency
Other Objective Examples
• Select appropriate testing needed before beginning a biologic agent on an IBD patient.
• Evaluate for splenic vein thrombosis in a patient with isolated gastric varices.
• Manage a patient with celiac disease who has persistent symptoms on a GFD.
• Identify peptic ulcers with stigmata for rebleeding that require endoscopic therapy.
Lead Line
Example: Lead Line A 56-year old alcoholic stumbles into the ER, ataxic,
with nystagmus on examination. Although he appears confused and intoxicated, his blood alcohol level is normal. You start IV glucose and symptoms worsen.
The next step would be to administerA. NiacinB. CT scan of the headC. ThiamineD. InsulinE. Dismissal to home
What iswrong
with this?
Example: Lead Line A 56-year old alcoholic stumbles into the ER, ataxic,
with nystagmus on examination. Although he appears confused and intoxicated, his blood alcohol level is normal. You start IV glucose and symptoms worsen.
What is the next best step in management?A. NiacinB. CT scan of the headC. ThiamineD. InsulinE. Dismissal to home
Keep lead line general; avoid syntax errors
Lead Line
• Most commonly used lead lines: Which of the following is the most likely diagnosis? What is the next best step in management?
Half treatment options, half diagnostic options
• Avoid negative lead lines: All of the following EXCEPT Which of the following is LEAST likely Which would you NOT do
Lead Line• Keep the lead line generic
“Next step” or “management” Allows diagnostic & therapeutic options
• Avoid writing syntax or grammatical errors that help exclude answers
• “Cover the options” rule Should be able to guess the answers
The Options
Example #1: Options
What is the next best step in management?A. Add IV metronidazoleB. Continue po vancomycinC. Start cholestyramineD. Begin rifaximinE. Stop the po vancomycin
What is wrong
with this?
Example #1: Options
What is the next best step in management?A. Add IV metronidazoleB. Continue po vancomycinC. Start cholestyramineD. Begin rifaximinE. Stop the po vancomycin
Mutually exclusive
Example #2: Options
What is the next best step in management?
A. Metronidazole 500 mg po TID x 14 daysB. Vancomycin 250 mg IV QID x 10 daysC. Metronidazole 500 mg IV QID x 14 daysD. Vancomycin 125 mg po QID x 14 days
What is wrong with this?
Example #2: Options
What is the next best step in management?
A. Metronidazole 500 mg po TID x 14 daysB. Vancomycin 250 mg IV QID x 10 daysC. Metronidazole 500 mg IV QID x 14 daysD. Vancomycin 125 mg po QID x 14 days
Testing too many concepts!Drugs, dose, route, duration
Example #3: OptionsThe cause of diarrhea in most patients with Zollinger-Ellison syndrome is:
A. Antacid useB. High glucagon levelsC. Fat malabsorption from pancreatic lipase
inactivation by acid productionD. Bacterial overgrowth What is wrong
with this?
Example #3: OptionsThe cause of diarrhea in most patients with Zollinger-Ellison syndrome is:
A. Antacid useB. High glucagon levelsC. Fat malabsorption from pancreatic lipase
inactivation by acid productionD. Bacterial overgrowth Correct answer
longer, detailed
The Options
• 4 or 5 options (less may be better)• One is not right and the rest 100% wrong
C E A B DWrong Right
The Options
• 4 or 5 options (less may be better)• One is not right and the rest 100% wrong• One is correct, the rest plausible
C E A B DLeast correct Most correct
Options
• Avoid mutually exclusive options Increases/decreases/same Always/never; stop/continue
• Each option should test one concept Drug OR dose OR route OR duration
• Keep options brief; similar in length• Options should be homogeneous
Length, complexity
Stem
The Stem
• Chronologic order of presentation Age, gender (avoid race unless needed) Site of visit (ER, clinic, hospital) Chief complaint (add features) PMH/Meds/FH/SH (relevant or distracter) Vitals/Exam/Labs/Tests (pertinent)
The Stem
• Fine to have details to make the other options attractive; avoid extraneous info
• It takes time to read; make sure it is fitting in with time to administer test
• Organize labs if many
The Explanation
The Explanation
• Explain why right answer correct first• Explain why wrong answers incorrect
– Do this chronologically
• Example for explanation order:– If correct answer is B– B, A, C, D, E
How Do You Analyze Your Questions?
Question Performance
• Goal for correct answer 50-70% 40-90% may be allowed at times Difficulty score of > 90% (too easy) Difficulty score of < 40% (too hard)
• See if it discriminates: Top test takers Bottom test takers
Performs well; difficulty ok and discriminating
(A) (B) (C) (D)
Answer *
Top 50% 7 80 9 4
Bottom 50% 19 44 28 9
Total 13 62 18 7
Option 0.62
Performs poor; difficulty ok, non-discriminating
Option (A) (B) (C) (D)
Answer *
Top 50% 62 0 7 31
Bottom 50% 80 0 7 13
Total 71 0 7 22
Difficulty 0.71
Performs poor; difficulty low (hard), non-discriminating
Option (A) (B) (C) (D)
Answer *
Top 50% 3 45 2 50
Bottom 50% 6 51 5 38
Total 5 48 3 44
Difficulty 0.44
http://www.nbme.org/publications/item-writing-manual-download.html
The Pretest: How Did You Do?
Avoid Duplicating Terms
1. The primary role of the denter is to move the a) ormeb) lattenc) denter pind) stagle
Avoid Repeat Digits, Look at Pattern
2. How many steps are in an average carrum?a) 3.7b) 370c) 240d) 37
Avoid Absolute Terms:Never, Always, Every, All, None
3. The grotto is frequently humid becausea) every ictal is fabliterousb) langers are always ganteringc) reslins are often dewinestd) culers are never liblis
Avoid Inhomogeneous Options
4. Rigniter is aa) form of glatnickb) prestigious subset of the family of honorary gloutninsc) signal by a sellantd) crown of slessings
Avoid Repeating the Correct Option More Than Others
5. Which of the following pairs contributed the Sterlik track?
a) Smith & Johnsonb) Jones & Smithc) Johnson & Thomsond) Smith & Davis
Avoid Syntax or Grammatical Errors
6. The tarnon is proceeded by aa) oflerb) arglingc) isterind) glantern
Questions?
Summary
• Know your learner level
• Writing order: objective, lead line, options, stem
• Avoid errors in MCQ writing
• Evaluate the effectiveness of your MCQs
Thank you!