Basic Test Construction

April Magoteaux & Adam MoskowitzAssessment Faculty Fellows

Columbus State Community CollegeNovember 2015

Assessment and Tests• One method for

assessing whether students meet the learning outcomes is to develop, administer, and analyze results of written tests.

• Planning, writing, and evaluating tests is sophisticated and serious.

Planning and Construction• This presentation

provides basic information on planning and constructing a test.

• Interpreting and analyzing test statistics, item analysis, and grading are important when using tests as assessment, but are not covered in this presentation.

Planning: Purpose• What is the purpose of

the test?• Readiness• Placement• Learning tool• Diagnosis• Teaching effectiveness• Formative assessment• Summative assessment

Planning: Comprehensive• Formative

• Comprehensive – prepares for long term retention of learning; important in certain areas

• Not Comprehensive – tends to be easier for students to navigate

• Summative

• Comprehensive – customary for assessment of summative learning; capstone courses; prepare for licensure or credentialing exams

• Not Comprehensive – not customary

Planning: Transparency• Inform students about

the purpose of all tests in the course.

• Inform students about policy on taking tests, review of test, answers, and content.

Planning: Test Blueprint• Outlines what is to be assessed on the tests• Use faculty expertise• Could be a grid with:• Content areas• Cognitive levels• Objectives• Emphasis or number of questions per topic• Critical knowledge areas

Sample Blueprint

Cognitive Level Knowledge

Comprehension

Application

Analysis

Content Stress

Communication

Nursing Process

Critical Thinking

Professional Standards

Planning: Desired Student Level• Match cognitive level to

desired student level.• Tempting to write lowest

cognitive level test questions on memory.• Lower cognitive levels

may be appropriate for beginners.

Content: Lead• Cognitive Level ?

• Child• Adult• Parent• Scientist• Chemist• Nurse• Environmentalist

Sample Blueprint-Lower Cognitive Level

Cognitive Level Knowledge

Comprehension

Application

Analysis

Content Stress 2

Communication 4

Nursing Process 4

Critical Thinking 3

Professional Standards

2

Sample Lower Level Question

• What is Osteoporosis?• A. Arthritis with psoriasis.• B. Reduced muscle

protein.• C. Loss of bone mass.• D. Rheumatoid arthritis.

Sample: Higher Level Question

An 82 year old, underweight female has been admitted to the orthopedic unit with a hip fracture. Her last bone density test result was -2.5 T-score. What does the nurse consider in caring for this patient? Select all that apply.

Options• A. Surgical repair may be

challenging due to osteoporosis.• B. High protein and

calories are likely needed. • C. Thin body structure is

an advantage in her bone health.• D. Potential for addiction

to narcotics is a priority.

• E. Restriction of oral intake may predispose her to delirium.• F. Bed-rest post-

operatively is essential to prevent emboli.• G. Serum total protein

and albumin indicate nutritional status.

Content: Easy Difficult • Math, physics, engineering• 1st semester, 2nd semester, 3rd semester• Technology: touch screen, word processing, hardware

design, software design, video game design• Automotive: operate turn signal, inflate tires, change

tire, repair axle, install wiring

Style of Question: Easy Difficult• True-False• Multiple Choice One

Answer• Multiple Choice All or

None• Select All That Apply• Fill in the Blank• Essay

Guessing True or False: 50% chance of answering correctly

Multiple Choice:25% chance of answering correctly if 1 answer and 3 deflectors

Basic Skills in Test Writing

Regardless of content and difficulty, there are test construction principles for writing tests.

Grammar, Spelling, & Punctuation• Students deserve a well-

written test.• Errors promote anxiety

and distrust in students.• Students should be able

to focus on content, not errors.• Goal: assess learning, not

emotion.

Outcome: Adaptation to Stress• Integrating stress,

emotion, and crisis MAY be part of learning outcomes• Examples: police,

emergency personnel, mental health professional• Testing may occur in

laboratory or clinical setting

Consistent Writing Throughout

• Style-manner of writing• Diction-word choice, level

of vocabulary• Tone-formal, informal,

serious• Voice-professor,

profession, poetic• Sentence Fluency

• Example:• The guy was drunk.• The man was inebriated.• The gentleman was

intoxicated.

Avoid• Slang• Jokes• Jargon• Contractions• Negative style (except)• Colloquial expression• Connecting questions• Bias (acceptable prejudice)• Abbreviations unless

explained, or need to know

Covert Bias• Bias can be subtly

accepted prejudice: • ageism • “kids today”• repeated use of certain

pronouns such as he

• Grandma Got Run Over by the Reindeer• Blonde Jokes• Anti-Aging• “Senior Moment”

Subtle Bias: Examples Since children today spend most of their time playing video games, the obesity in America has increased. What does the nurse include in her teaching plan to decrease game playing?

Playing video games may benefit learning, health, & socialization (Bowen, 2014).

Bowen, L. (2014). Video game play may provide learning, health, social benefits, review finds. Monitor on Psychology, 45(2), 10. Retrieved from http://www.apa.org/monitor/2014/02/video-game.aspx

The higher incidence of major depressive disorder in the older adult is normal since age brings agitation, and physical and mental deterioration.

Typography and Punctuation

Vocabulary• Consistent: nurse or

healthcare worker or RN• Customary for college,

program, discipline

• Vulnerability scan• Attack surface• Indicator of compromise• Attack analysis• Threat forensics• IDS/IPS• Perimeter firewall

Multiple Choice Question Style• Begin questions with a

capital.• End question with a

question mark.• Begin options with a

capital.• End options with a

period.

1.What is the abbreviation for ammonia?

a. Cub. NH3c. O2d. Ra

(period omitted to correlate with custom in chemistry)

Multiple Choice: Stem & Options

Stem• Use meaningful stem.• Include words in the stem

that would be repeated in options.• Eliminate excessive

wording in stem (reading test versus concept test).

Options• Ensure one is correct.• Homogeneous content,

form, grammar, length• Randomly assign correct

option• Avoid “all”, “always”,

“never” unless completely true.

Grammar Inconsistency with Deflectors

1. Sunscreen with a protection factor of 4 or greater effectively:

a. blocks vitamin D synthesis through the skin.b. are needed for prevention of anemia and hypocalcemia.c. usually prevents sunburn and blistering.d. is needed for prevention of Lyme disease.

Length Inconsistency with Deflectors

1.What are possible problems with prolonged sun exposure?

a. Melanoma.b. Osteoporosis, iron-

deficiency anemia.c. Long bone fractures,

weight loss, glaucoma.d. Hepatic

encephalopathy, urinary retention, herpes zoster.

Items with Numbers• List answer options in

sequential order.• Refrain from beginning a

sentence with a number.• Use Arabic symbols for

numbers rather than letters.• Use only approved and

customary abbreviations students need to know.

What is the recommended calcium intake for a 20 year old?

a. 500 mg/day.b. 700 mg/dayc. 850 mg/day.d. 1,000 mg/day.

Complete the Sentence Style• Begin the item with a

capital.• End the item with a

colon.• Begin each option with a

lower case letter (unless proper noun).• End each option with a

period.

1. Check the pounds per square inch (psi) when the tires are:a. cold.b. hot.c. warm.d. new.

(psi might be a need to know abbreviation for the discipline)

Time for Students• Time limit recommended

for assessment of learning.• Use faculty judgement:

student level, complexity of questions, future assessments.• Typical: 1-2 minutes per

question.

Time for Course, Facilities, Faculty, • Online, blended,

traditional• Course hours: testing

time versus learning time• Online testing: secure,

non-secured• Facility availability:

computer, timing, census• Test development:

faculty, publisher, national

Time for Students• Course room learning

time versus course room testing time• Using tests for learning• Learning outside of

course room• Using tests of learning

can motivate learning

Tests for Learning• May be used in a variety

of ways:• Small group• During class together• Assignments• As help for test taking skills

and content in course• Promote understanding of

complicated content• Rationales for correct and

incorrect options may increase understanding

• May help avert:• “Just tell me what is on the

test.”• Test anxiety• Relying on memory

without critical thinking• Prejudice or bias relying on

previous assumptions

Sample Test Question for Learning• Two patients in the

hospital are becoming amorous in the hallway in view of other patients. How does the nurse intervene?

Options• Wheel patient in

wheelchair out of the area.• Tell them, “Hey you two,

take it to your room.”• Set limits by prohibiting

touching each other.• The nurse should not

intervene.

• Ask them if they are interested in birth control.• Discuss prevention of

disease related to intimacy.

Answer Depends…• Correct answer depends

on many factors such as competency, mental status, amorous behavior• May promote discussion

of acceptance of whole person despite handicap or illness

• May help students become more comfortable with difficult topics such as intimacy, death, spirituality, loss

Time for Faculty• Test Development• Stock• Design and write

• Grading• Essay require more time• Short answer• Computerized or hand• Collection of tests and

results

• Test analysis• Computer or hand• Evaluation• Adjustments

Questions? Need Help?• April Magoteaux• 287-5158• amagotea@cscc.edu

Resources• Bastable, S. B. (2013). Nurse as educator: Principles of

teaching & learning for nursing practice (4th ed.). Burlington, MA: Jones & Bartlett Learning.• Billings, D. M., & Halstead, J. A. (2013). Teaching in

nursing: A guide for faculty (4th ed.). St. Louis, MO: Elsevier Health Sciences.• Gronlund, N. E., & Waugh, C. K. (2012). Assessment of

student achievement (10th ed.). Upper Saddle River, NJ: Pearson.• National Council of State Boards of Nursing. (2015).

NCLEX & other exams. NCSBN. Retrieved from https://www.ncsbn.org/nclex.htm