Test Blueprinting: How to Build a Good Multiple Choice Test Item

Presented by Amy Thibaut

Director of Social StudiesSpring Branch ISD

Topic Suggested Timeframe

% of Six

Weeks # of Test

Items

Early India, China and Africa completed

1 day

Byzantine and Islamic Civilizations

10 days

Middle Ages

15 days

World History 2nd six weeks 26 Instructional days/50 question test

•% of days spent on a topic makes up what % of six weeks?

•% of test questions should match % of six weeks spent on topic. How do we figure this?

• # of days taught ÷ # of instructional days = % of six weeks spent on topic

•Total # of items x % of weeks = # of topic items to be included on test

3.8%

38%

57.6%

2

19

29

Parts of a test item:The Stimulus

•Any item that is used to answer the question•Not required of every question

The stem•Poses the question

The Distracter• Answer Choices

Linking Items to Objective

Items must measure what they purport to measure.

• Use the objective as the guide to write the item

• Use the objective to evaluate the item

• Use content-specific terminology appropriate for the targeted grade level

8.20 Citizenship. The student understands the rights and responsibilities of citizens of the United States. The student is expected to :

(B) summarize rights guaranteed in the Bill of Rights;

Among the rights guaranteed by the First Amendment is the right to

A. the free exercise of religion

B.keep and bear arms

C.due process of law

D.a speedy and public trial

Is this question aligned to the TEKS?

Checking for Alignment

• Get into groups by content group

• Using the handout with 15 TAKS items and TEKS Correlations, read each question and determine if it is aligned to the TEKS listed at the end of the question.

• How could the question be changed to make it more aligned to the TEKS?

Qualities of a Good Multiple-Choice Item

• Measures the skill for which it is written• Present the problem or question as clearly

as possible• Has only one best and correct answer• Is as straightforward and concise as

possible• Has four choices that are parallel in

structure and logical• Contains option that are plausible, but

incorrect• Is written in such a way that it is grade-

appropriate

So you’re the little woman who wrote the book that made this great war!

Abraham Lincoln to Harriet Beecher Stowe, 1862

To which book is President Lincoln referring in the excerpt above?

A. Uncle Tom’s Cabin

B. The Wealth of Nations

C. Democracy in America

D. McGuffey’s Eclectic Reader

Qualities of a Bad Multiple-Choice Item

• Tests trivial information• Contains unnecessary/confusing

information in the stem or options

• Is tricky or cute• Gives clues (cues) to the correct

answer (Clang associations)• Poses a question for which many

defensible answers are possible• Contains a bias toward or

against a group of individuals

Which of the following kings of England was so bad that no king of England will name his son in honor of him?

A. King William the Conqueror

B. King Henry VIII

C.King Charles II

D.King John

Critique the question

Is there a Clang Association?

Is it trivia?

Effective Stems are…

•focused on one idea•clear and concise•phrased as a direct

question•phrased positively

Writing Convincing Response Options• Use only one best,

clearly correct answer• Develop plausible

distracters• Use homogeneous

answer choices• Avoid clues• Be efficient

Use only one best, clearly correct answer• Avoid answer choices

that overlap.• Avoid answer choices

that are subsets of another option.

• Avoid answer choices that could be defended as equally correct.

Develop plausible distracters

Plausible distracters include…• common misconceptions.• statements that sound

logical but that are not based on the material given in the stimulus and question.

• common or familiar phrases.• statements that are true but

that do not answer the question.

Use homogeneous answer choicesAnswer choices should be

parallel in terms of content, structure, and length.

• Distracters should be in the same form as the key.

• No answer option should stand out.

Be Efficient

• During the mid-1800s the expanding U.S. economy required better methods of communication. Communications were improved significantly by Samuel Morse’s invention of the A. TypewriterB. TelephoneC. TelegraphD. phonograph

Answer choices should be brief.Answer choices should be brief.No longer than a sentenceRepeated phrases should be placed into the Repeated phrases should be placed into the stemstem

Answer choices should be organized.

• Alphabetical• Ascending or

descending order• Shortest to longest• Same order as

presented in stimulus

Speaker 1: The government must give accused persons the right to a quick and public trial.

Speaker 2: The government should not force citizens to house soldiers in their homes.

Speaker 3: People should be allowed to peacefully protest actions of the government.

Speaker 4: People should not be deprived of their property without fair payment.

Which speaker is voicing a right guaranteed by the First Amendment to the U.S. Constitution?

A. Speaker 1

B. Speaker 2

C. Speaker 3

D. Speaker 4

Writing Convincing Response Options:Additional Considerations

• Avoid use of “none of the above,” “all of the above,” or “I don’t know”

• Avoid use of negative words in the answer options

• Balance the keys

Sensitivity

Be aware of issues that are emotionally upsetting or distracting to an examinee

• Death or severe illness• Extremes in socioeconomic

conditions• Unfair representation of

groups• Controversial topics

Fairness

Provide an equal opportunity for success

• Item measures the intended objective

• Content, context, and vocabulary are age and grade-level appropriate

• Groups treated with dignity and respect

Use homogeneous answer choices

Distracter Guidelines

If the key contains: Then distracters should NOT contain:

A long phrase or sentence

A general rule

A complete statement

A means to an end

Simple, common language

Unqualified words or statements

Negative descriptions

Short phrases or sentences

Specific instances

Incomplete statements

End product

Professional jargon

Qualifying words

Positive descriptions

Use a variety of stimuli• Textbox of primary source

material• Graphs and Charts• Pictures or illustrations• Spoke Diagrams• Multiple visuals: comparison

maps, charts• Political Cartoons• Time lines• Flow Charts• Graphic designs• Headlines• Speaker Questions

You Analyze the Question:

• Using the handout provided, identify the problems, if any, you see with the test design of each question.

Which of the following was not a characteristic of Greek art?

A. It symbolized the pride of the people in their city-state.

B. It expressed Greek love for harmony, balance, order, and moderation.

C. It combined beauty and usefulness.

D. It stressed the importance of the gods and the unimportance of humans.

Rousseau: Man is born free, yet everywhere he is in chains. The government is created by a contract among the people and receives its powers from them. The government exits to serve the people, and when it no longer does so, the people may change it.Locke: All men have certain natural rights; the right to life, liberty, and property. The purpose of government is to protect these rights. If it fails to do so, the people may set up a new government.Voltaire: It is to him who masters our minds by the force of truth, not to those who enslave men by violence…that we owe our reverence. Popular government is in itself, therefore, less (unjust), less (hated) then despotic power.Montesquieu: There is no liberty if the judicial power is not separated from the legislative and executive branches.

According to the theories of Locke and Rousseau, how did government originate?A. A social contract – an agreement among people.B. By God’s commandC. By royal decreeD. By elections held among the upper classes

Rousseau: Man is born free, yet everywhere he is in chains. The government is created by a contract among the people and receives its powers from them. The government exits to serve the people, and when it no longer does so, the people may change it.Locke: All men have certain natural rights; the right to life, liberty, and property. The purpose of government is to protect these rights. If it fails to do so, the people may set up a new government.Voltaire: It is to him who masters our minds by the force of truth, not to those who enslave men by violence…that we owe our reverence. Popular government is in itself, therefore, less (unjust), less (hated) then despotic power.Montesquieu: There is no liberty if the judicial power is not separated from the legislative and executive branches.

What was Montesquieu’s opinion of the French system of government before the revolution?A. It was perfect just the way it was.B. There was no liberty because there was no separation of

powersC. The legislative branch was fine but the executive branch was

weak.D. The executive branch was fine but the legislative branch was

weak.

Which of the following is a definition of technology?

A. the application of knowledge for practical ends

B. the investigation of natural phenomena

C. items created by artisansD. none of the above

Which of the following is an example of technology?

A. irrigation canalsB.stone axC.computersD.all of the above

Karl Marx believed

A. socialism could be achieved by peaceful progress

B. socialism was inevitable because capitalism would self-destruct

C. socialism should reward the economic skills of the upper class

D. socialism was an impossible solution to difficult times

The leader of the Bolsheviks who successfully led the communist take-over of Russia was

A.Vittorio OrlandoB.Nicolai LeninC.Leo TolstoyD.James Joyce

What is considered the first act of military aggression in World War II?A. The invasion of

Ethiopia by ItalyB. The invasion of China

by JapanC. The invasion of Russia

by GermanyD. The attack of Pearl

Harbor

The Truman Doctrine said that the U.S. would help any country fighting against communism. This idea was popular because it would “bottle up” the communists. Therefore, it introduced the concept of

A.appeasementB. isolationismC.containmentD.imperialism