Measurement

500 Research MethodsMike Kroelinger

Levels of Measurement

NominalLowest level -- used to classify variables into two or more categories.Cases placed in the same category must be equivalent. The categories must be exhaustive --all persons or items must fit into one of the categories.Must also be mutually exclusive -- one person or item can't fit more than one category.Examples race and gender.

Levels of Measurement

OrdinalNumbers only used to indicate the rank order of cases of a variable. Cannot measure or evaluate the different in value between each case.No mathematical or statistical operations (you can't add label 1 to label 2, etc.).Examples:

Hardness of metalPersonnel evaluations of performance

Levels of Measurement

IntervalHas all of the above characteristics with the added requirement of equal distances or intervals between labels

Represent equal distances in the variables of your study.

Examples:Foot-candle levels in lightingDifferences in air temperature

Levels of Measurement

RatioHas all of above features plus an absolute zero point.Enables you to multiple and divide scale numbers to create ratios between labels. Examples:

Income rangesNumber of years of schoolAge in years

Best Rule of Thumb

Measure at the highest level possible!

Reliability

A matter of whether a particular technique, applied repeatedly to the same object, would yield the same result each time.

Good example would be weight scales.

Another would be an architect's scale vs. a ruler.

ReliabilityReliability does not insure accuracy any more than precision insures it.

Changing your bathroom scales to let you weigh less is an example -- called bias. Light meters another example.

Reliability infers stability and consistency.Is your operational definition measuring something consistently and dependably? Repeats of measure yield consistent results.

Reliability

A highly unreliable measure cannot be valid !

A very reliable measure may still not be valid because you could be consistently measuring something other than what you intended to measure.

Types of Reliability

Test-Re-testRepeat of the measures -- have respondents complete questionnaire more than once.Observers record the same behavior more than once.

Types of ReliabilityParallel

Two similar or alternative forms of a measure are designed to be as similar as possible.Administered successively to the same group of subjects -- high correlation between scores of the measurements would mean high reliability.

Types of ReliabilitySplit-half

Always a good idea to make more than one measurement of a variable.As example -- a questionnaire with ten items that measure a variable (prejudice). With split half, you would randomly assign the items to two sets of five subjects. If the two sets of items measure the variable differently you would have reliability problems in measuring the variable.

Types of Reliability

Internal ConsistencyItem by item examination of relationships

attempts to determine to what extent the items measure the same concept.

Also referred to as homogeneity.Determined statistically.

Types of Reliability

Research-worker (Babbie)Measurement reliability can be caused by research workers.

Example -- coding the way interview questions are asked.

Ways to solve include extensive training, practice, and cross-checking

For example, a supervisor calls a subset of the S's and repeats a portion of the interview.

Verifies information by replication.

Validity

Validity refers to the extent to which an empirical measure adequately reflects the real meaning of the concept being studied.

Validity cannot be assessed directly.

Validity

What we want to know is the "goodness of fit" between an operational definition and the concept that it supposedly measures.

In short, your operational definition should truly reflect what the concept means! If you are measuring what you intend to measure, you have validity.

Validity

Excellent example is amniocentesis - to determine gender of unborn children.

Valid because it is almost perfect in accuracy of prediction. Not-valid example is old wife's tales about childbirth and gender.

Second example - scales for weighing yourself vs. a subjective estimate of how much you weigh.

Types of Validity

Face validityAn empirical measurement may or may not match common agreements and our individual ideas of a particular concept. For example, we may measure a person's morale by the number of grievances that they file with the union. We could not measure morale through the number of books the S's read during off-duty hours! The second measure would not have face validity.

Types of Validity

Criterion-related validity

Sometimes called predictive validity; is based on an external criterion. An example is the college board exam --validity is shown through its ability to predict college success of students.

Types of Validity

Content ValidityRefers to the degree to which a measure covers the range of meanings included with the concept. A good example is a math test -- will not show range of meanings if it covers addition alone -- must cover subtraction, multiplication, division.

Types of Validity

Construct ValidityBased on the way a measure relates to other variables within a conceptual framework or model.Emphasizes the meaning of responses to your instrument. Is the instrument measuring the intended concept or can it be interpreted as measuring something else?

Types of Validity

Construct ValidityGood example would be marital satisfaction – “gift buying” predicts satisfaction. If satisfied and dissatisfied spouses were equally likely to buy each other “gifts”, you would not have validity of your measure.

Summary

A valid measure is reliable but a reliable measure may not be valid!Measure at the highest possible level!Use prior studies whenever possible to select measures for your research. Make sure that you select studies that have reliability and validity. The more they are used, the better we define their reliability and validity!

Questions