Writing Meaningful Test QuestionsKurt Gilliland, Ph.D.Dept. of Cell Biology and PhysiologyRyan Madanick, M.D.Division of Gastroenterology and HepatologyDept. of Medicine

Writing Meaningful Test QuestionsFormat of stems, lead-ins, and optionsWhat are common flaws to avoid?

Content of stems, lead-ins, and optionsHow can your items reflect learning objectives?How can you develop items that test application instead of recall?

1. The primary purpose of the stam is to remove which of the following?

A. CarmB. DentonC. MeniceD. Stam bar

1. The primary purpose of the stam is to remove which of the following?

A. CarmB. DentonC. MeniceD. Stam barWord repeat:A word or phrase is included in the stem and in the correct answer.Item Flaw

2. Which of the following pairs has won the greatest number of Abby awards?

A. Jones and SmithB. Smith and TaylorC. Smith and WhiteD. White and Allen

2. Which of the following pairs has won the greatest number of Abby awards?

A. Jones and SmithB. Smith and TaylorC. Smith and WhiteD. White and AllenConvergence strategy:The correct answer included the most elements in common with other options.Item Flaw

3. How many pounds of pressure are exerted by a callam?

A. 260B. 150C. 2.6D. 2600

3. How many pounds of pressure are exerted by a callam?

A. 260B. 150C. 2.6D. 2600Language in the options is not parallel:Options are in an illogical order, in this case.Item FlawsLogical cues:A subset of the options are collectively exhaustible. Often, the correct answer is in the middle of the range.

4. The stanon is aided by a

A. anstelB. immonC. octalD. port

4. The stanon is aided by a

A. anstelB. immonC. octalD. portItem FlawGrammatical cues:One or more distractors dont follow grammatically from the stem.

5. Why does the stanon frequently overheat?

A. All grestels are belious.B. No immons are directly fectitious.C. Octals are usually casable.D. Ports are always actial.

5. Why does the stanon frequently overheat?

A. All grestels are belious.B. No immons are directly fectitious.C. Octals are usually casable.D. Ports are always actial.Item FlawsAbsolute terms:Terms such as always or never are used in options.Frequency terms in the topics are vague:Terms such as rarely or usually not consistently defined or interpreted, even by experts.

6. Stamation normally occurs when the

A. anstels rupture.B. immon falls and the denton is in place.C. octal rotates easily.D. ports pass over the carm.

6. Stamation normally occurs when the

A. anstels rupture.B. immon falls and the denton is in place.C. octal rotates easily.D. ports pass over the carm.Item FlawLong correct answer:Correct answer is longer, more specific, or more complete than other options.

BONUS. Find 4 flaws.

Regarding syphilis, all are true except

A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain.B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in T. pallidum infections but become negative in tertiary syphilis.C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic centers and a surrounding rim of plasma cells.D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta with aortitis and aneurysm.E. None of the above

BONUS. Find 4 flaws.

Regarding syphilis, all are true except

A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain.B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in T. pallidum infections but become negative in tertiary syphilis.C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic centers and a surrounding rim of plasma cells.D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta with aortitis and aneurysm.E. None of the aboveItem Flaws1. Short stem but long, heterogeneous options.2. A negative term in the stem.3. Unfocused stem.4. Use of none of the above as option.

Frequency TermsWhat do terms like usually or frequently mean to you?

Frequency Terms

The most important prognostic factor in AIDS is

A. CD4:CD8 ratioB. presence of Kaposis sarcomaC. absolute leukocyte countD. absolute CD4 countE. level of GP120 antibody

The most important prognostic factor in AIDS is

A. CD4:CD8 ratioB. presence of Kaposis sarcomaC. absolute leukocyte countD. absolute CD4 countE. level of GP120 antibodyUnfocused stemUse of absolute terms such as most

What is the cause of muscle soreness that is present the day or two after exercise?

A. Accumulation of lactic acidB. Microscopic muscle damage leading to an inflammatory response including the release of bradykinin, PGE2, and histamine which sensitive nerve endingsC. Muscle stiffnessD. Use of anabolic steroids

What is the cause of muscle soreness that is present the day or two after exercise?

A. Accumulation of lactic acidB. Microscopic muscle damage leading to an inflammatory response including the release of bradykinin, PGE2, and histamine which sensitive nerve endingsC. Muscle stiffnessD. Use of anabolic steroidsLong distractorHeterogeneous distractors

The tophi of gouty arthritis:

A. are a form of erythema nodosumB. have a fibrinoid center surrounded by peripheral histiocytesC. are composed of prominent osteophytes at the DIP joints of fingersD. none of the above

The tophi of gouty arthritis:

A. are a form of erythema nodosumB. have a fibrinoid center surrounded by peripheral histiocytesC. are composed of prominent osteophytes at the DIP joints of fingersD. none of the aboveUnfocused stemShort stem, long distractorsNone of the above

Sigmund Freuds Interpretation of Dreams was published in what year?

A. 1911B. 1912C. 1913D. 1914E. 1915

Sigmund Freuds Interpretation of Dreams was published in what year?

A. 1911B. 1912C. 1913D. 1914E. 1915Correct answer (1913) in middle of rangeQuestion is recall of minutia and not application

Aspirin should NOT be prescribed when the patient has which of the following conditions?

A. Peptic ulcers B. HeadacheC. FeverD. PainE. Inflammation

Aspirin should NOT be prescribed when the patient has which of the following conditions?

A. Peptic ulcers B. HeadacheC. FeverD. PainE. InflammationUse of not is acceptable in this case since the teaching point was in fact a contraindication.

Which of the following is true regarding lysosomal storage disease?

A. / B. /C.D.

Which of the following is true regarding lysosomal storage disease?

A. / B. /C.D. Unfocused stem

*****************************