1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”

1

Writing Test Blueprints and Test Items

“Software for Creating and Delivering

Assessments With Powerful Reports”

2

Objectives

204/20/23

Develop test items

Survey the test blueprint

04/20/23Copyright 2000 QuestionMark. All rights reserved.

At the end of this presentation, you will understand how to:

Document the test blueprint

3

Best Practices for Developing Tests


1.Analyze the job 2.Document the test blueprint 3.Survey the blueprint 4.Develop items 5.Alpha test and revise items6.Beta test the exam 7.Select items and set the cut-score 8.Go live with the test

4

Define objectives in behavioral terms Action verb Condition(s) Standard(s)

Use concrete verbiage Meet key criteria

Sample Work Process for Writing Test Objectives


5

Major test objectives or topics (job tasks)

Subtopics (subtasks) within each major performance area

Rank test objectives and topics (tasks) in order of importance

How Do You Document the Test Blueprint?


6

Select contributors Re-prioritize tasks (objectives) Query time spent

How Do You Survey the Test Blueprint?


7

Question Writing Process

Start from your surveyed blueprint Select a level of difficulty guideline Write question stems Write correct answers Write distractors Edit


8

What Does a Level of Difficulty Guideline Give You?

Consistency Set performance standards Yardstick to gauge level of difficulty


9

Level of Difficulty Guideline

Evaluation Synthesis Analysis Application Comprehensio

n Knowledge


10

Knowledge and Comprehension

Evaluation SynthesisAnalysisApplication Comprehension - explain, predict,

paraphrase Knowledge - define, label, list


11

Application and Analysis

Evaluation Synthesis Analysis – compare, contrast, relate Application – apply, solve, computeComprehensionKnowledge


12

Synthesis and Evaluation

Evaluation – appraise, evaluate, judge Synthesis – formulate, construct, create AnalysisApplicationComprehensionKnowledge


13

Apply Appropriate Test Standards

Formal standards Administrative policies Item writing guidelines


14

Question (Response) Types

Perception offers eight question types: Multiple choice True/false Multiple response Numeric item Word response Selection item Matrix item Hot spot item


15

Preferred Item Types

Easily understood Minimize guessing Avoid testing trivial information Avoid candidate scrolling


16

How to Write Stems

Present a single, definitive problem Use a question format Objective congruence Appropriate level of difficulty Active voice


17

What to Avoid in Stems

Questions that build Negatively phrased questions “Always” and “Never” “Corporate-centric” questions Humor Trick questions “Trainer-centric” questions


18

Guidelines to Write Options

Comparable in content, length, & logic Non-repeating Parallel language Logically or numerically ordered Avoid “All of the above” and “None of the

above” Plausible distractors


19

Avoid Grammatical Clues


Option A is the only option that makes grammatical sense.

The best way to increase the reliability of a test is to:

A. Increase the test length. [Correct]B. Removing poor quality items. C. Test should be readable for all test

takers.

20

Avoid Association Clues


Option B includes the word “seven.”

Who are the “Magnificent Seven?”

A. A pro basketball team [Correct]B. A group of seven fictional, heroic,

Western figures [Correct]C. A protest group made famous during

the 1960sD. A vegetarian rock group

21

What is the cost of an item that normally sells for 9.99 USD that is discounted 25%?

Order Options Numerically


WrongA. 5.00B. 7.50C. 2.50D. 6.66

RightA. 2.50 B. 5.00C. 6.66D.7.50

22

What are the three important concerns in achievement testing? V=validity R=reliability E=efficiency O=objectivity

Order Options Logically


WrongA. V, R, E B. O, V, RC. R, O, E D. E, O, V

RightA. V, R, O B. V, R, E C. V, E, O D.R, E, O

23

Objective statement by item

Item naming conventions Item difficulty or weight

Evolving the Test Blueprint


24

Thank you for your attention

Thank you for your time and attention. We hope you have enjoyed learning about how to write test blueprints and test items.

We at Question Mark strive to improve our products and services. To let us know how you felt about this presentation, visit our North American home page and complete the Learning Center Survey: http://www.questionmark.com/surveys.htm

Question Mark provides numerous presentations to help our customers, and prospective customers, successfully deploy computerized assessments. We provide public “web-seminars” that use a combination of conference calls and web technology to discuss computerized assessment issues. You can register for a public seminar by going to: www.questionmark.com/seminars/index.asp

For other material that you might find useful, please visit our Learning Center at: www.questionmark.com/learningcenter/


Documents

1 Writing Test Blueprints and Test Items “Software for Creating and Delivering Assessments With Powerful Reports”