1

Beginning the Research DesignTheory, Questions, Hypotheses

Designing Tests for the above:Conceptualization,

Operationalization, and Measurement

2

Conceptualization Process of specifying what we mean

when we use particular terms. Produces an agreed upon meaning

for a concept for the purposes of research.

Describes the indicators we'll use to measure the concept and the different aspects of the concept.

3

From Concept to Measurement Progression from what a term

means to measurement in a scientific study: Conceptualization Nominal Definition Operational Definition Measurements in the Real World

4

Four Levels of Measurement1. Nominal - offer names for labels

for characteristics (gender, birthplace).

2. Ordinal - variables with attributes we can logically rank and order.

5

Four Levels of Measurement3. Interval - distances separating

variables (temperature scale).4. Ratio - attributes composing a

variable are based on a true zero point (age).

6

MeasurementsThings Scientists Measure Direct observables - things that can

be observed simply and directly. Indirect observables - things that

require more subtle observations. Constructs - based on observations

that cannot be observed.

7

Measurement Quality

Reliability Validity

8

ReliablityGENERAL DEFINITION:Accuracy or precision of a measuring instrument. SPECIFIC DEFINITIONS:

1. Similar results - stability, dependability predictability

2. Accuracy – consistency

3. Absence of random or chance error -- extent to which errors of measurement are present in a measuring instrument

9

Tests for Checking Reliability

Test-retest method - take the same measurement more than once.

Equivalence: use "essentially the same" measurement items on the same instrument or on different instruments and compare the answers (same time period). Split-half, Random half, alternate forms. Use established measures.

10

Internal ValidityDEFINITION: the ability of the measuring instrument to

measure one's theoretical concepts.

METHODS OF ASSESSING VALIDITY:

PRAGMATIC (or Criterion) VALIDITY: predict to an outside criterion and compare the outcome to the outside criterion

a. Concurrent: comparison to an existing or current outside criterionb. Predictive: comparison to a future outside criterion

FACE VALIDITY: obvious and self-evident content

11

Validity (cont.)CONTENT VALIDITY: representativeness of

what is being measured to the intended concepts (capturing all the dimensions of the social concept)

CONSTRUCT VALIDITY: adequacy of the

measuring instrument for measuring the theoretical concepts and relationships; also adequacy of the logical structure of the conceptualization and operationalization.

12

Construct Validity

13

External Validity

14

Political Polls and Survey Sampling In the 2000 Presidential election,

pollsters came within a couple of percentage points of estimating the votes of 100 million people.

To gather this information, they interviewed fewer than 2,000 people.

15

Election Eve Polls - U.S. Presidential Candidates, 2000

Date Agency Gore Bush Nader

Buchanan

11/6 IDB/CSM 47 49 4 011/6 CBS 48 47 4 111/6 CNN/USA

Today] 46 48 4 1

11/6 Reuters/MSNBC

48 46 5 1

11/6 Voter.com

45 51 4 0

11/7 Results 48 48 3 1

16

Observation and Sampling Polls and other forms of social research,

rest on observations. The task of researchers is to select the

key aspects to observe, or sample. Generalizing from a sample to a larger

population is called probability sampling and involves random selection.

17

Types of Nonprobability Sampling Reliance on available subjects:

• Only justified if less risky sampling methods are not possible.

• Researchers must exercise caution in generalizing from their data when this method is used.

18

Types of Nonprobability Sampling Purposive or judgmental sampling

• Selecting a sample based on knowledge of a population, its elements, and the purpose of the study.

• Used when field researchers are interested in studying cases that don’t fit into regular patterns of attitudes and behaviors

19

Types of Nonprobability Sampling Snowball sampling

• Appropriate when members of a population are difficult to locate.

• Researcher collects data on members of the target population she can locate, then asks them to help locate other members of that population.

20

Types of Nonprobability Sampling Quota sampling

• Begin with a matrix of the population.• Data is collected from people with the

characteristics of a given cell. • Each group is assigned a weight appropriate

to their portion of the population.• Data should provide a representation of the

total population.

21

Probability Sampling Used when researchers want

precise, statistical descriptions of large populations.

A sample of individuals from a population must contain the same variations that exist in the population.

22

Probability Sampling Most effective method for selection

of study elements. Avoids researchers biases in

element selection. Permits estimates of sampling

error.

23

Populations and Sampling Frames Findings based on a sample represent

the aggregation of elements that compose the sampling frame.

Sampling frames do not always include all the elements their names imply.

All elements must have equal representation in the frame.

24

Types of Sampling Designs Simple random sampling (SRS) Systematic sampling Stratified sampling

25

Simple Random Sampling Feasible only with the simplest

sampling frame. Basic method assumed in most

statistical computations

26

Systematic Sampling Slightly more accurate than simple

random sampling. Arrangement of elements in the

list can result in a biased sample.

27

Stratified Sampling Rather than selecting sample for

population at large, researcher draws from homogenous subsets of the population.

Results in a greater degree of representativeness by decreasing the probable sampling error.

28

Multistage Cluster Sampling Used when it's not possible or

practical to create a list of all the elements that compose the target population.

Involves repetition of two basic steps: listing and sampling.

Highly efficient but less accurate.