View
229
Download
2
Category
Tags:
Preview:
Citation preview
1
Writing Test Blueprints and Test Items
“Software for Creating and Delivering
Assessments With Powerful Reports”
2
Objectives
204/20/23
Develop test items
Survey the test blueprint
04/20/23Copyright 2000 QuestionMark. All rights reserved.
At the end of this presentation, you will understand how to:
Document the test blueprint
3
Best Practices for Developing Tests
04/20/23Copyright 2000 QuestionMark. All rights reserved.
1.Analyze the job 2.Document the test blueprint 3.Survey the blueprint 4.Develop items 5.Alpha test and revise items6.Beta test the exam 7.Select items and set the cut-score 8.Go live with the test
4
Define objectives in behavioral terms Action verb Condition(s) Standard(s)
Use concrete verbiage Meet key criteria
Sample Work Process for Writing Test Objectives
04/20/23Copyright 2000 QuestionMark. All rights reserved.
5
Major test objectives or topics (job tasks)
Subtopics (subtasks) within each major performance area
Rank test objectives and topics (tasks) in order of importance
How Do You Document the Test Blueprint?
04/20/23Copyright 2000 QuestionMark. All rights reserved.
6
Select contributors Re-prioritize tasks (objectives) Query time spent
How Do You Survey the Test Blueprint?
04/20/23Copyright 2000 QuestionMark. All rights reserved.
7
Question Writing Process
Start from your surveyed blueprint Select a level of difficulty guideline Write question stems Write correct answers Write distractors Edit
04/20/23Copyright 2000 QuestionMark. All rights reserved.
8
What Does a Level of Difficulty Guideline Give You?
Consistency Set performance standards Yardstick to gauge level of difficulty
04/20/23Copyright 2000 QuestionMark. All rights reserved.
9
Level of Difficulty Guideline
Evaluation Synthesis Analysis Application Comprehensio
n Knowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
10
Knowledge and Comprehension
Evaluation SynthesisAnalysisApplication Comprehension - explain, predict,
paraphrase Knowledge - define, label, list
04/20/23Copyright 2000 QuestionMark. All rights reserved.
11
Application and Analysis
Evaluation Synthesis Analysis – compare, contrast, relate Application – apply, solve, computeComprehensionKnowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
12
Synthesis and Evaluation
Evaluation – appraise, evaluate, judge Synthesis – formulate, construct, create AnalysisApplicationComprehensionKnowledge
04/20/23Copyright 2000 QuestionMark. All rights reserved.
13
Apply Appropriate Test Standards
Formal standards Administrative policies Item writing guidelines
04/20/23Copyright 2000 QuestionMark. All rights reserved.
14
Question (Response) Types
Perception offers eight question types: Multiple choice True/false Multiple response Numeric item Word response Selection item Matrix item Hot spot item
04/20/23Copyright 2000 QuestionMark. All rights reserved.
15
Preferred Item Types
Easily understood Minimize guessing Avoid testing trivial information Avoid candidate scrolling
04/20/23Copyright 2000 QuestionMark. All rights reserved.
16
How to Write Stems
Present a single, definitive problem Use a question format Objective congruence Appropriate level of difficulty Active voice
04/20/23Copyright 2000 QuestionMark. All rights reserved.
17
What to Avoid in Stems
Questions that build Negatively phrased questions “Always” and “Never” “Corporate-centric” questions Humor Trick questions “Trainer-centric” questions
04/20/23Copyright 2000 QuestionMark. All rights reserved.
18
Guidelines to Write Options
Comparable in content, length, & logic Non-repeating Parallel language Logically or numerically ordered Avoid “All of the above” and “None of the
above” Plausible distractors
04/20/23Copyright 2000 QuestionMark. All rights reserved.
19
Avoid Grammatical Clues
04/20/23Copyright 2000 QuestionMark. All rights reserved.
Option A is the only option that makes grammatical sense.
The best way to increase the reliability of a test is to:
A. Increase the test length. [Correct]B. Removing poor quality items. C. Test should be readable for all test
takers.
20
Avoid Association Clues
04/20/23Copyright 2000 QuestionMark. All rights reserved.
Option B includes the word “seven.”
Who are the “Magnificent Seven?”
A. A pro basketball team [Correct]B. A group of seven fictional, heroic,
Western figures [Correct]C. A protest group made famous during
the 1960sD. A vegetarian rock group
21
What is the cost of an item that normally sells for 9.99 USD that is discounted 25%?
Order Options Numerically
04/20/23Copyright 2000 QuestionMark. All rights reserved.
WrongA. 5.00B. 7.50C. 2.50D. 6.66
RightA. 2.50 B. 5.00C. 6.66D.7.50
22
What are the three important concerns in achievement testing? V=validity R=reliability E=efficiency O=objectivity
Order Options Logically
04/20/23Copyright 2000 QuestionMark. All rights reserved.
WrongA. V, R, E B. O, V, RC. R, O, E D. E, O, V
RightA. V, R, O B. V, R, E C. V, E, O D.R, E, O
23
Objective statement by item
Item naming conventions Item difficulty or weight
Evolving the Test Blueprint
04/20/23Copyright 2000 QuestionMark. All rights reserved.
24
Thank you for your attention
Thank you for your time and attention. We hope you have enjoyed learning about how to write test blueprints and test items.
We at Question Mark strive to improve our products and services. To let us know how you felt about this presentation, visit our North American home page and complete the Learning Center Survey: http://www.questionmark.com/surveys.htm
Question Mark provides numerous presentations to help our customers, and prospective customers, successfully deploy computerized assessments. We provide public “web-seminars” that use a combination of conference calls and web technology to discuss computerized assessment issues. You can register for a public seminar by going to: www.questionmark.com/seminars/index.asp
For other material that you might find useful, please visit our Learning Center at: www.questionmark.com/learningcenter/
04/20/23Copyright 2000 QuestionMark. All rights reserved.
Recommended