MEASUREMENT. Measurement “If you can’t measure it, you can’t manage it.” Bob Donath,...

MEASUREMENT

Measurement

“If you can’t measure it, you can’t manage it.”

Bob Donath, Consultant

Measurement

Selecting measurable phenomena

Developing a set of mapping rules

Applying the mapping ruleto each phenomenon

What is to be measured?

CONCEPTS/CONSTRUCTS/VARIABLES

Operational DefinitionsOperational Definitions- Dimensions- Dimensions- Elements- Elements

ScalesScales

CONCEPT

A GENERALIZED IDEA ABOUT A A GENERALIZED IDEA ABOUT A CLASS OF OBJECTS, CLASS OF OBJECTS,

ATTRIBUTES, OCCURRENCES, ATTRIBUTES, OCCURRENCES, OR PROCESSES.OR PROCESSES.

(e.g., (e.g., Technology, Dynamism, Adoption, Technology, Dynamism, Adoption, Learning)Learning)

CONSTRUCT

An image or idea specifically invented for a given An image or idea specifically invented for a given research and/or theory-building purposeresearch and/or theory-building purpose

Higher level concepts for specialised scientific Higher level concepts for specialised scientific explanatory purposes that are not directly explanatory purposes that are not directly observable and for thinking about and observable and for thinking about and communicating abstractionscommunicating abstractions

Concepts and constructs are used at theoretical levels Concepts and constructs are used at theoretical levels

VARIABLE

Used at empirical levelUsed at empirical level Accept numeral or values for the purpose of Accept numeral or values for the purpose of

testing and measurementtesting and measurement

OPERATIONAL DEFINITION

A definition for a construct stated in terms of A definition for a construct stated in terms of specific criteria for testing or measurementspecific criteria for testing or measurement

Specifies what the researcher must do to Specifies what the researcher must do to measure the concept/construct under measure the concept/construct under

investigationinvestigation

SCALE- Definition

SERIES OF ITEMSSERIES OF ITEMS ARRANGED ACCORDING TO VALUEARRANGED ACCORDING TO VALUE FOR THE PURPOSE OF FOR THE PURPOSE OF

QUANTIFICATIONQUANTIFICATION A CONTINUOUS SPECTRUMA CONTINUOUS SPECTRUM

SCALE PROPERTIES

UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES PRESERVES ORDERPRESERVES ORDER DISTANCE/EQUAL INTERVALSDISTANCE/EQUAL INTERVALS NATURAL ORIGIN (ZERO)NATURAL ORIGIN (ZERO)

Types of Scales

OrdinalOrdinal

IntervalInterval

RatioRatio

NominalNominal

NOMINAL SCALE

Nominal Scales

Mutually exclusive Mutually exclusive and collectively and collectively exhaustive exhaustive categoriescategories

Exhibits the Exhibits the classification classification characteristic onlycharacteristic only

NOMINAL SCALE PROPERTIESUNIQUELY CLASSIFIESUNIQUELY CLASSIFIES Male/FemaleMale/Female Academic/AdminAcademic/Admin Asian/EuropeanAsian/European Strategy typesStrategy types Adopters/Non AdoptersAdopters/Non Adopters

Levels of Measurement

OrdinalOrdinal

IntervalInterval

RatioRatio

NominalNominal ClassificationClassification

ORDINAL SCALE

Ordinal Scales

Characteristics of Characteristics of nominal scale plus nominal scale plus an indication of an indication of orderorder

Implies statement Implies statement of greater than and of greater than and less thanless than

ORDINAL SCALE PROPERTIES

UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES

PRESERVES ORDERPRESERVES ORDER Win, Place & ShowWin, Place & Show Podium/Grid positionsPodium/Grid positions RankingRanking

Levels of Measurement

OrdinalOrdinal

IntervalInterval

RatioRatio

NominalNominal ClassificationClassification

OrderOrderClassificationClassification

INTERVAL SCALE

Interval Scales

Characteristics of Characteristics of nominal and ordinal nominal and ordinal scales plus the scales plus the concept of equality concept of equality of interval. of interval.

Equal distance Equal distance exists between exists between numbersnumbers

INTERVAL SCALE PROPERTIESUNIQUELY CLASSIFIESUNIQUELY CLASSIFIES

PRESERVES ORDERPRESERVES ORDER

EQUAL INTERVALSEQUAL INTERVALS Consumer Price Index (Base 100)Consumer Price Index (Base 100) Fahrenheit TemperatureFahrenheit Temperature Assessment of attitude, beliefs, intentionAssessment of attitude, beliefs, intention

Levels of Measurement

OrdinalOrdinal

intervalinterval

RatioRatio

NominalNominal ClassificationClassification

OrderOrderClassificationClassification

OrderOrderClassificationClassification DistanceDistance

RATIO SCALE

Ratio Scales

Characteristics of Characteristics of previous scales plus an previous scales plus an absolute zero pointabsolute zero point

ExamplesExamples WeightWeight HeightHeight Number of childrenNumber of children

RATIO SCALE PROPERTIES

UNIQUELY CLASSIFIESUNIQUELY CLASSIFIES

PRESERVES ORDERPRESERVES ORDER

EQUAL INTERVALSEQUAL INTERVALS

NATURAL ZERONATURAL ZERO Weight and distanceWeight and distance Age, years of serviceAge, years of service

Levels of Measurement

OrdinalOrdinal

IntervalInterval

RatioRatio

NominalNominal ClassificationClassification

OrderOrderClassificationClassification

OrderOrderClassificationClassification DistanceDistance

Natural OriginNatural Origin

OrderOrderClassificationClassification DistanceDistance

Goodness of Measures

Accurately measuring what Accurately measuring what actually the concept one sets out to actually the concept one sets out to measuremeasure

Should be easy and efficient to useShould be easy and efficient to use

Evaluating Measurement Tools

CriteriaCriteria

Validity

Practicality ReliabilityReliability

Validity

The ability of a scale to measure The ability of a scale to measure what was intended to be what was intended to be

measured.measured.

Validity Determinants

Content

ConstructCriterion

Increasing Content Validity

ContentLiterature Search

Literature Search

Expert Interviews

Expert Interviews

Group Interviews

Group Interviews

Question DatabaseQuestion Database

……Goodness of Measures

ValidityValidity Content Validity Content Validity

Includes Face ValidityIncludes Face ValidityHow well the dimensions and elements How well the dimensions and elements

of a concept have been delineatedof a concept have been delineatedProfessional agreement that a scale Professional agreement that a scale

logically appears to accurately measure logically appears to accurately measure what it is intended to measure what it is intended to measure

Validity Determinants

Content

Construct

Increasing Construct Validity

New measure of trustNew measure of trust

Known measure of trustKnown measure of trust

EmpathyEmpathy

CredibilityCredibility

……Goodness of Measures

Construct Validity – when empirical evidence Construct Validity – when empirical evidence generated by a measure is consistent with the generated by a measure is consistent with the theoretical logic of the concept theoretical logic of the concept

Convergent Validity – the measure should Convergent Validity – the measure should “converge” with other similar measures“converge” with other similar measures

Discriminant Validity – when the measure Discriminant Validity – when the measure has low correlation with measures of has low correlation with measures of dissimilar conceptdissimilar concept

Judging Criterion Validity

RelevanceRelevance

Freedom from biasFreedom from bias

ReliabilityReliability

AvailabilityAvailability

Criterion

Criterion-Related Validity

Concurrent – When the scale discriminates Concurrent – When the scale discriminates individuals who are known to be different; A individuals who are known to be different; A type of criterion validity whereby a new type of criterion validity whereby a new measure correlates with a criterion measure measure correlates with a criterion measure taken at the same time.taken at the same time.

Predictive Validity – Ability to differentiate Predictive Validity – Ability to differentiate among individuals with reference to a future among individuals with reference to a future criterioncriterion

VALIDITY

F A C E O R C O N TE N T

C O N C U R R E N T P R E D IC TIV E

C R ITE R IO N V A L ID ITY C O N S TR U C T V A L ID ITY

V a lid ity

Reliability

The degree to which measures are The degree to which measures are

free from random free from random errorerror and and therefore yield consistent results.therefore yield consistent results.

Reliability Estimates

Stability

InternalConsistency

Equivalence

Reliability Estimates

Stability

InternalConsistency

Equivalence

Stability of MeasuresStability of MeasuresTest-Retest Reliability – the Test-Retest Reliability – the

administering of the same scale or administering of the same scale or measure to the same respondents at two measure to the same respondents at two separate points in timeseparate points in time

Parallel-Form Reliability - when Parallel-Form Reliability - when responses on two comparable sets of responses on two comparable sets of measures tapping the same construct are measures tapping the same construct are highly correlatedhighly correlated

… Reliability

Reliability Estimates

Stability

InternalConsistency Equivalence

Equivalence

Variations at one point in time among Variations at one point in time among observers and samples of itemsobservers and samples of items

Reliability Estimates

Stability

InternalConsistency

Equivalence

… Reliability

Internal Consistency MeasuresInternal Consistency MeasuresInteritem Consistency Reliability - Interitem Consistency Reliability -

where items that are independent where items that are independent measures of the same concept correlate measures of the same concept correlate with one another.with one another.

Split-Half Reliability – reflects the Split-Half Reliability – reflects the correlations between two halves of an correlations between two halves of an instrument instrument

RELIABILITY

TE S T R E TE S T

S TA B IL ITY

E Q U IV A L E N T F O R M S S P L ITTIN G H A L V E S

IN TE R N A L C O N S IS TE N C Y

R E L IA B IL ITY

Understanding Validity and Reliability

Practicality

Economy InterpretabilityConvenience

SENSITIVITY

A measurement instrument’s ability to A measurement instrument’s ability to accurately measure variability in accurately measure variability in stimuli or responses.stimuli or responses.

Rating ScalesUsed to elicit responses with regard to the object, event, or person studies:

Dichotomous scaleDichotomous scale

Category scaleCategory scale

Likert scaleLikert scale

Numerical scaleNumerical scale

Semantic differential scaleSemantic differential scale

…..Rating Scales

Itemised rating scaleItemised rating scale

Fixed or constant sum rating scaleFixed or constant sum rating scale

Stapel scaleStapel scale

Graphic rating scaleGraphic rating scale

Consensus scaleConsensus scale

Ranking ScalesTo tap preferences between two or more objects or items

Paired ComparisonPaired Comparison

Forced ChoiceForced Choice

ComparativeComparative