Norm-referenced Scores

Norm-referenced scores are produced by a two stage process:o Determination of total scores on the test using the test’s scoring criteria;o Conversion of these total scores to a new form that includes information about the

performance of other people who took the test. Ex. Percentile rank – comparison of a student’s actual scores to the number of correct answers produced by other students in the class.

Most standardized test (intelligence, achievement, skills, personality tests) produce norm-referenced scores.

It is the appropriate choice when the goal is to sort examines into how much they know or what they are like relative to each other. Ex. For rating, selection, placement or diagnosis.

Norm-referenced Test

Definition of terms:

Norm group – the group to which the score is compared. Standardized sample – it is the norm group. Item difficulty – the proportion of people who answer an item correctly. It is needed to sort

examinees into groups based on their abilities. Hence, it must contain some very difficult items that can only be answered by most knowledgeable or skillful test-taker. It is usually used in maximal performance tests.

Norm-referenced Test

Item discrimination – refers to the way different types of people respond to an item. It is used to sort people into groups according to personal characteristics, not level of ability. Hence, the test must include items that differentiate among different types of people. Ex. We need items that a depressed person answers differently from a person who is not depressed.

Criterion-referenced Score

Criterion – rather comparing a person’s performance with others, performance is evaluated relative to the domain of perfect performance – correct answers to all questions. Its goal is to represent the extent to which test takers have mastered the test domain.

Not all maximal-performance tests are suitable for criterion-referenced scoring. It is most appropriate when there is a clearly defined specific content to be tested.

It is difficult to develop criterion-referenced systems for large scale standardized tests that are designed for examinees with varying experience & backgrounds.

Test Objectives

Test objectives must be considered by test developers when writing their tests.

Objectives – refers specifically to instructional objectives measured by criterion-referenced goals.

When writing objectives, the specificity of the objectives will affect the way test items are written.

Hopkins (Hopkins 1996, as cited by Erford B., et al. 2007) contended that tests that measure educational objectives must define these objectives must define these objectives in terms of “Bloom’s taxonomy”

What is Bloom’s Taxonomy?

It is one method of categorizing objectives according to cognitive complexity devised by Bloom, Englehart, Furst, & Kratwohl (1956). It is a taxonomy of educational objectives for the cognitive domain from the simplest level (knowledge) to the most complex level (evaluation). Levels are presumed to be hierarchical. Meaning higher level objectives are assumed to include & be dependent on, lower level cognitive skills. It includes (Bloom’s taxonomy – The Cognitive Domain):

1. Knowledge – objectives at this level require the students to remember. Test items ask students to recall, recognize facts, terminology, problem-solving strategies or rules.

2. Comprehension – requires some level of understanding. Test items require student to translate, to restate what has been read, to see connections or relationships among parts to see connections or relationships among parts of communication draw inferences.

3. Application – requires the student to use previously acquired information in a setting other than which it was learned. It presents the problem the problem in an often applied context.

4. Analysis – requires the student to identify logical errors or to differential among facts, opinions, assumptions, hypotheses, or conclusions. It requires the student to draw relationships among ideas or to compare and contrast.

5. Synthesis – requires the student to produce something unique or original. It requires the student to solve unfamiliar problem in a unique way.

6. Evaluation – requires the student to form judgments about the value or worth of methods, ideas, people or products.

Bloom’s Taxonomy – The Affective Domain

The affective taxonomy describes objectives that reflect underlying emotions, feelings, or values rather than thought complexity. It has developed by Kratwohl, Bloom, & Masia (1964). It describes a process by which another person’s groups or society’s ideas, beliefs, customs, etc. are gradually accepted & internalized by a different person, group, or society. It begins with a minimal, partial or incomplete acceptance of an alternative point of view & culminates with the complete integration of this point of view into an individual’s personal belief system. The levels are generally considered to be hierarchical:

1. Receiving (attending) – student must have at least an awareness of some stimulus. Once this has occurred, a willingness at least to listen must be present (tolerance). A student will next be able to attend selectively to various aspects of the context within which the stimulus exists.

2. Responding – student responses indicate more than passive listening, but require active participation. A student acquiesces to a teacher’s request to engage in an activity. More complete responding is indicated by a willingness to engage in an activity, even when allowed a different choice. The highest level within this category is indicated by satisfaction after engaging in a response.

3. Valuing – student judges an activity as to its worthiness & tend to do so consistently that the pattern is recognizable to others. The basic sublevel is the acceptance of a belief, idea or attitude, leading to internalization. Then, the student pursues the idea & demonstrates a preference for it. Finally, after being convinced, the student expresses commitment to the idea & demonstrates the conviction by pursuing the goal.

4. Organization – ideas have now become organized into a value system. Student conceptualize value by analysis and generalizations that reflect valued idea (cognitive, but included here since it was done after valuing). Next, the ideas are incorporated into the organization of a value system. The valued ideas are arranged to foster consistency & compatibility with each other.

5. Characterization by a value or value complex – student behaves in a way that is consistent with his/her value system automatically. The first sublevel is characterized by a generalized set, meaning the individual is predisposed to perceive & react according to an internalized value system. The next level is characterization, when the individual’s thoughts and behaviors are consistent.

The Psychomotor Domain

In addition to the cognitive & affective taxonomies, a taxonomy of the psychomotor behaviors has been developed by Harrow (1972).

It virtually includes all behaviors like speaking, writing, eating, etc. It is the most difficult to classify into taxonomic levels since almost all of the actions involve

cognitive, and often, affective components. Nevertheless, this taxonomy may prove useful to teachers in lower elementary grades &

teachers in PE, education, dance, theater, & other courses that involve considerable movements.

The following taxonomy levels ranges from the lowest level of observable reflexive behavior to the highest level, representing the most complex forms of nonverbal communication.

1. Reflex Movements – they are involuntarily movements that either are evidentat birth or develop with maturation. Sublevels include segmental reflexe, intersegmental reflexes, and suprasegmental reflexes.

2. Basic-fundamental Movements – they are inherent in more complex or skilled motor movements or skilled motor movements. Sublevels include loco motor movements, non-loco motor movements, and manipulative movements.

3. Perceptual Abilities – they refer to all abilities of an individual that send input to the brain for interpretation, which in turn affects motor movements. Sublevels include kinesthetic, visual, auditory, tactile discrimination, and coordinated abilities.

4. Physical Abilities – they are characteristics of an individual’s physical self, which when developed properly enhance smooth and efficient movement. Sublevels include endurance, strength, flexibility, & agility.

5. Skilled Movements – they are the result of learning, often complex learning. They result in efficiency in carrying out a complex movement or task. Sublevels include simple, compound, and complex adaptive skills.

6. Non discursive Communication – it is a form of communication through movement. Ex. Facial expression, postures, and expressive dance routines. Sublevels include expressive movement and interpretative movement.

Scaling

Scaling is a process by which a measuring device is designed and calibrated and the way numbers (other indices) – scale value are assigned to different amounts of the trait, attribute, or characteristic being measured. It is attaching numbers to the construct that the test is theorized to measure. Scaling can be fairly straightforward or extremely complicated.

Scales may be defined in many ways, whether they be:

Normal – a scale of measurements in which people or objects are placed in categories according to their properties. When numbers are used, they serve only as the category labels, they do not indicate extent of a property.

Ordinal – a scale of measurement in which people or objects are ranked on a property or dimension.

Interval – a scale of measurement in which people or objects are ranked using an equal unit system. A difference of 1 unit or 2 is equivalent at any location along the scale. It is the measurement used most often in testing.

Ratio – a scale of measurement containing both equal-interval units & a true zero point. A score of 10, therefore, is not only 5 units more that a score of 5 – it is twice as much as a score of 5.