DESIGNING ASSESSMENT QUESTIONSPresented by:Ms. Fatima Anwaar

WHAT IS AN ASSESSMENT? In education, the term assessment refers to

the wide variety of methods or tools that educators use to evaluate, measure, and document the academic readiness, learning progress, skill acquisition, or educational needs of students.

While assessments are often equated with traditional tests—especially the standardized tests developed by testing companies and administered to large populations of students—educators use a diverse array of assessment tools and methods to measure everything from a four-year-old’s readiness for kindergarten to a twelfth-grade student’s comprehension of advanced physics.

Just as academic lessons have different functions, assessments are typically designed to measure specific elements of learning—e.g., the level of knowledge a student already has about the concept or skill the teacher is planning to teach or the ability to comprehend and analyze different types of texts and readings.

Assessments also are used to identify individual student weaknesses and strengths so that educators can provide specialized academic support, educational programming, or social services. In addition, assessments are developed by a wide array of groups and individuals, including teachers, district administrators, universities, private companies, state departments of education, and groups that include a combination of these individuals and institutions.

While assessment can take a wide variety of forms in education, the following descriptions provide a representative overview of a few major forms of educational assessment.

HIGH-STAKES ASSESSMENTS High-stakes assessments are typically

standardized tests used for the purposes of accountability—i.e., any attempt by federal, state, or local government agencies to ensure that students are enrolled in effective schools and being taught by effective teachers.

In general, “high stakes” means that important decisions about students, teachers, schools, or districts are based on the scores students achieve on a high-stakes test, and either punishments (sanctions, penalties, reduced funding, negative publicity, not being promoted to the next grade, not being allowed to graduate) or accolades (awards, public celebration, positive publicity, bonuses, grade promotion, diplomas) result from those scores. 

PRE-ASSESSMENTS Pre-assessments are administered before

students begin a lesson, unit, course, or academic program. Students are not necessarily expected to know most, or even any, of the material evaluated by pre-assessments—they are generally used to

(1) establish a baseline against which educators measure learning progress over the duration of a program, course, or instructional period, or

(2) determine general academic readiness for a course, program, grade level, or new academic program that student may be transferring into.

FORMATIVE ASSESSMENTS Formative assessments are in-process

evaluations of student learning that are typically administered multiple times during a unit, course, or academic program. The general purpose of formative assessment is to give educators in-process feedback about what students are learning or not learning so that instructional approaches, teaching materials, and academic support can be modified accordingly. Formative assessments are usually not scored or graded, and they may take a variety of forms, from more formal quizzes and assignments to informal questioning techniques and in-class discussions with students.

SUMMATIVE ASSESSMENTS Summative assessments are used to

evaluate student learning at the conclusion of a specific instructional period—typically at the end of a unit, course, semester, program, or school year. Summative assessments are typically scored and graded tests, assignments, or projects that are used to determine whether students have learned what they were expected to learn during the defined instructional period.

COMPARISON OF FORMATIVE ASSESSMENTS AND SUMMATIVE ASSESSMENTS Formative assessments are commonly said to

be for learning because educators use the results to modify and improve teaching techniques during an instructional period, while summative assessments are said to be of learning because they evaluate academic achievement at the conclusion of an instructional period.

Or as assessment expert Paul Black put it, “When the cook tastes the soup, that’s formative assessment. When the customer tastes the soup, that’s summative assessment.”

INTERIM ASSESSMENTS Interim assessments are used to evaluate

where students are in their learning progress and determine whether they are on track to performing well on future assessments, such as standardized tests, end-of-course exams, and other forms of “summative” assessment.

Interim assessments are usually administered periodically during a course or school year (for example, every six or eight weeks) and separately from the process of instructing students (i.e., unlike formative assessments, which are integrated into the instructional process)

PLACEMENT ASSESSMENTS Placement assessments are used to

“place” students into a course, course level, or academic program. For example, an assessment may be used to determine whether a student is ready for Algebra I or a higher-level algebra course, such as an honors-level course. For this reason, placement assessments are administered before a course or program begins, and the basic intent is to match students with appropriate learning experiences that address their distinct learning needs.

SCREENING ASSESSMENTS Screening assessments are used to determine

whether students may need specialized assistance or services, or whether they are ready to begin a course, grade level, or academic program. Screening assessments may take a wide variety of forms in educational settings, and they may be developmental, physical, cognitive, or academic. A preschool screening test, for example, may be used to determine whether a young child is physically, emotionally, socially, and intellectually ready to begin preschool, while other screening tests may be used to evaluate health, potential learning disabilities, and other student attributes.

Assessments are also designed in a variety of ways for different purposes:

STANDARDIZED ASSESSMENTS Standardized assessments are designed,

administered, and scored in a standard, or consistent, manner. They often use a multiple-choice format, though some include open-ended, short-answer questions.

Standardized tests can be administered to large student populations of the same age or grade level in a state, region, or country, and results can be compared across individuals and groups of students.

STANDARDS-REFERENCED OR STANDARDS-BASED ASSESSMENTS Standards-referenced or standards-

based assessments are designed to measure how well students have mastered the specific knowledge and skills described in local, state, or national learning standards.

Standardized tests and high-stakes tests may or may not be based on specific learning standards, and individual schools and teachers may develop their own standards-referenced or standards-based assessments. 

COMMON ASSESSMENTS Common assessments are used in a school or district to

ensure that all teachers are evaluating student performance in a more consistent, reliable, and effective manner.

Common assessments are used to encourage greater consistency in teaching and assessment among teachers who are responsible for teaching the same content, e.g. within a grade level, department, or content area. They allow educators to compare performance results across multiple classrooms, courses, schools, and/or learning experiences (which is not possible when educators teach different material and individually develop their own distinct assessments). 

Common assessments share the same format and are administered in consistent ways—e.g., teachers give students the same instructions and the same amount of time to complete the assessment, or they use the same scoring guides to interpret results. Common assessments may be “formative” or “summative.”

PERFORMANCE ASSESSMENTS Performance assessments typically require

students to complete a complex task, such as a writing assignment, science experiment, speech, presentation, performance, or long-term project, for example. Educators will often use collaboratively developed common assessments, scoring guides, rubrics, and other methods to evaluate whether the work produced by students shows that they have learned what they were expected to learn. Performance assessments may also be called “authentic assessments,” since they are considered by some educators to be more accurate and meaningful evaluations of learning achievement than traditional tests.

Portfolio-based assessments are collections of academic work—for example, assignments, lab results, writing samples, speeches, student-created films, or art projects—that are compiled by students and assessed by teachers in consistent ways. Portfolio-based assessments are often used to evaluate a “body of knowledge”—i.e., the acquisition of diverse knowledge and skills over a period of time. Portfolio materials can be collected in physical or digital formats, and they are often evaluated to determine whether students have met required learning standards. 

The purpose of an assessment generally drives the way it is designed, and there are many ways in which assessments can be used. A standardized assessment can be a high-stakes assessment, for example, but so can other forms of assessment that are not standardized tests. A portfolio of student work can be a used as both a “formative” and “summative” form of assessment. Teacher-created assessments, which may also be created by teams of teachers, are commonly used in a single course or grade level in a school, and these assessments are almost never “high-stakes.” Screening assessments may be produced by universities that have conducted research on a specific area of child development, such as the skills and attributes that a student should have when entering kindergarten to increase the likelihood that he or she will be successful, or the pattern of behaviors, strengths, and challenges that suggest a child has a particular learning disability. In short, assessments are usually created for highly specialized purposes

QUESTION!! A question is a linguistic expression used to

make a request for information, or the request made using such an expression. The information requested should be provided in the form of an answer.

“There are these four ways of answering questions.

Which four? There are questions that should be answered

categorically [straightforwardly yes, no, this, that].

There are questions that should be answered with an analytical (qualified) answer [defining or redefining the terms].

There are questions that should be answered with a counter-question.

There are questions that should be put aside. — Buddha, Sutta Pitaka

USES The principal use of questions is to elicit

information from the person being addressed, by indicating, more or less precisely, the information which the speaker (or writer) desires.

However, questions can also be used for a number of other purposes.

Questions may be asked for the purpose of testing someone's knowledge, as in a quiz or examination.

Raising a question may guide the questioner along an avenue of research.

TYPES OF QUESTIONS A research question is an interrogative

statement that manifests the objective or line of scholarly or scientific inquiry designed to addresses a specific gap in knowledge.

Research questions are expressed in a language that is appropriate for the academic community that has the greatest interest in answers that would address said gap.

These interrogative statements serve as launching points for the academic pursuit of new knowledge by directing and delimiting an investigation of a topic, a set of studies, or an entire program of research.

BY PURPOSE

Various categorizations of questions have been proposed with regard to research projects, one system distinguishes:

descriptive questions, used primarily with the aim of describing the existence of some thing or process

relational questions, designed to look at the relationships between two or more variables

causal questions, designed to determine whether certain variables affect one or more outcome variables

For the purpose of surveys, one type of question asked is the closed-ended (also closed or dichotomous) question, usually requiring a yes/no answer or the choice of an option(s) from a list .

There are also nominal questions, designed to inquire about a level of quantitative measure, usually making connections between a number and a concept (as in "1 = Moderate; 2 = Severe; 3 = ...").

 Open-ended or open questions give the respondent greater freedom to provide information or opinions on a topic.

Some types of questions that may be used in an educational context are listed in Bloom's Taxonomy of educational objectives.

These include questions designed to test and promote:

Knowledge: Who, what, when, where, why, how . . . ? Describe . . . ?

Comprehension: Retell . . .  Application: How is . . . an example of . . . ?; How

is . . . related to . . . ?; Why is . . . significant? Analysis: What are the parts or features of . . . ?

Classify . . . according to . . . ; Synthesis: What would you infer from . . . ? What ideas

can you add to . . . ? How would you design a new . . . ? What would happen if you combined . . . ? What solutions would you suggest for . . . ?

Evaluation: Do you agree that . . . ? What do you think about . . . ? What is the most important . . . ? Place the following in order of priority . . . ? How would you decide about . . . ? What criteria would you use to assess . . . ?

The one type of questions are called polar questions that contain yes/no as an answer.

The other main type of question (other than yes–no questions) is those called wh-questions (or non-polar questions). These use interrogative words (wh-words) such as when, which, who, how, etc. to specify the information that is desired.

The name derives from the fact that most of the English interrogative words (with the exception of how) begin with the letters wh.

These are the types of question sometimes referred to in journalism and other investigative contexts as the Five Ws.

Tag questions are a grammatical structure in which a declarative statement or an imperative is turned into a question by adding an interrogative fragment (the "tag"), such as right in "You remembered the eggs, right?", or isn't it in "It's cold today, isn't it?"

Tag questions may or may not be answerable with a yes or no.

DESIGNING EXAMINATION ASSESSMENT QUESTIONS! Examinations are a very common

assessment and evaluation tool in universities and there are many types of examination questions.

 This tips sheet contains a brief description of seven types of examination questions, as well as tips for using each of them:

1) multiple choice, 2) true/false, 3) matching, 4) short answer, 5) essay, 6) oral, and 7) computational. 

1. MULTIPLE CHOICE

Multiple choice questions are composed of one question (stem) with multiple possible answers (choices), including the correct answer and several incorrect answers (distractors). 

WHAT ARE DISTRACTERS? Distractors are: A) Elements of the exam layout that distract

attention from the questionsB) Incorrect but plausible choices used in multiple choice questionsC) Unnecessary clauses included in the stem of multiple choice questions

TIPS FOR WRITING GOOD MULTIPLE CHOICE ITEMS:

Avoid In the stem: Long / complex sentences Trivial statements Negatives and double-negatives Ambiguity or indefinite terms, absolute statements, and broad generalization Extraneous material Item characteristics that provide a clue to the answer misconceptions In the choices: Statements too close to the correct answer Completely implausible responses ‘All of the above,’ ‘none of the above’ Overlapping responses (e.g., if ‘A’ is true)

Do Use In the stems: Your own words – not statements straight out of the

textbook Single, clearly formulated problems In the choices: Plausible and homogeneous distracters Statements based on common student misconceptions True statements that do not answer the questions Short options – and all same length Correct options evenly distributed over A, B, C, etc. Alternatives that are in logical or numerical then ‘C’ is

also true) order At least 3 alternatives

2…………..TRUE/FALSE

True/false questions are only composed of a statement. Students respond to the questions by indicating whether the statement is true or false. For example: True/false questions have only two possible answers (Answer: True).

Like multiple choice questions, true/false questions: Are most often used to assess familiarity with

course content and to check for popular misconceptions

Allow students to respond quickly so exams can use a large number of them to test knowledge of a broad range of content

Are easy and quick to grade but time consuming to create

True/false questions provide students with a 50% chance of guessing the right answer. For this reason, multiple choice questions are often used instead of true/false questions.

TIPS FOR WRITING GOOD TRUE/FALSE ITEMS:

Avoid

Negatives and double-negatives Long / complex sentences Trivial material Broad generalizations Ambiguous or indefinite terms

Do Use

Your own words The same number of true and false

statements (50 / 50) or slightly more false statements than true (60/40) – students are more likely to answer true

One central idea in each item

3………………..MATCHING

Students respond to matching questions by pairing each of a set of stems (e.g., definitions) with one of the choices provided on the exam. These questions are often used to assess recognition and recall and so are most often used in courses where acquisition of detailed knowledge is an important goal. They are generally quick and easy to create and mark, but students require more time to respond to these questions than a similar number of multiple choice or true/false items.

TIPS FOR WRITING GOOD MATCHING ITEMS:

Avoid Long stems and options Heterogeneous content (e.g., dates mixed

with people) Implausible responses Do Use Short responses 10-15 items on only one

page Clear directions Logically ordered choices (chronological,

alphabetical, etc.)

4…………….SHORT ANSWER

Short answer questions are typically composed of a brief prompt that demands a written answer that varies in length from one or two words to a few sentences. They are most often used to test basic knowledge of key facts and terms.

Avoid Trivia Long / complex sentencesDo Use Your own words Specific problems Direct questions

5…………………ESSAYS

Essay questions provide a complex prompt that requires written responses, which can vary in length from a couple of paragraphs to many pages. Like short answer questions, they provide students with an opportunity to explain their understanding and demonstrate creativity, but make it hard for students to arrive at an acceptable answer by bluffing. They can be constructed reasonably quickly and easily but marking these questions can be time-consuming and grader agreement can be difficult.

TIPS FOR WRITING GOOD ESSAY ITEMS:

Avoid Complex, ambiguous wording Questions that are too broad to allow time for

an in-depth responseDo Use Your own words Words like ‘compare’ or ‘contrast’ at the

beginning of the question Clear and unambiguous wording A breakdown of marks to make expectations

clear Time limits for thinking and writing

6………………….ORAL EXAMS

Oral examinations allow students to respond directly to the instructor’s questions and/or to present prepared statements. These exams are especially popular in language courses that demand ‘speaking’ but they can be used to assess understanding in almost any course by following the guidelines for the composition of short answer questions. Some of the principle advantages to oral exams are that they provide nearly immediate feedback and so allow the student to learn as they are tested. There are two main drawbacks to oral exams: the amount of time required and the problem of record-keeping. Oral exams typically take at least ten to fifteen minutes per student, even for a midterm exam. 

7……………..COMPUTATIONAL

Computational questions require that students perform calculations in order to solve for an answer. Computational questions can be used to assess student’s memory of solution techniques and their ability to apply those techniques to solve both questions they have attempted before and questions that stretch their abilities by requiring that they combine and use solution techniques in novel ways.

EFFECTIVE COMPUTATIONAL QUESTIONS SHOULD:

Be solvable using knowledge of the key concepts and techniques from the course. Before the exam solve them yourself or get a teaching assistant to attempt the questions.

Indicate the mark breakdown to reinforce the expectations developed in in-class examples for the amount of detail, etc. required for the solution.

To prepare students to do computational questions on exams, make sure to describe and model in class the correct format for the calculations and answer including:

How students should report their assumptions and justify their choices

The units and degree of precision expected in the answer

Thank-You for your patience!!!!!!!