Research Design

ED 592AFall 2011

Research Concepts

1. Quantitative vs. Qualitative & Mixed Methods2. Sampling3. Instrumentation4. Validity and Reliability5. Internal Validity6. External Validity7. Design – Experimental & Correlational

Qualitative vs. Quantitative

Qualitative: Investigate the quality of relationships, activities, situations, or materials.

Quantitative: The systematic empirical investigation of social phenomena via statistical, mathematical or computational techniques.

Descriptive Statistics

Mean - The mean can be defined as the arithmetic average of all values. The mean measures the central tendency of a variable.

Significant Difference - In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.

Even When you are Right you may be Wrong…

The risk of not being 100% confident that what you observe in an experiment is due to the treatment or what is being tested.

p < .05 (1 in 20 that any differences where not due to the treatment)

SamplingSampling: • Selecting those who will participate

• PopulationAll members of a particular group to whom the researcher

would like to generalize about

• Representative sampleSample that is similar to the population on all characteristics

Random Sampling

Random

• Simple RandomAll members have an equal chance of being selected

• Stratified RSCertain characteristics are represented in the sample in the same proportion as they

occur in the pop.

• Cluster RSUsing groups as the sampling unit rather than individuals

• Two-Stage RSSelect randomly and then choose individuals randomly from the groups

Purposive Sampling

Purposive sampling targets a particular group of people.

“Quasi” - design

Instrumentation

• InstrumentationThe entire process of collecting data in a research investigation

• DataKinds of information researchers obtain on the subjects of their research

• ClassifyWho will provide the data?What is your Method of data collection?Who will collect the data?What kind of responses are required?Self-report data (provided by the subjects of the study)Informant data (provided by other people about the subjects of the study)

Validity & Reliability

• ValidityAppropriateness, meaningfulness, correctness,

and usefulness of the inferences researchers make

• ReliabilityConsistency of scores or answers from one

administration of an instrument to another and from one set of items to another

Internal Validity

Any relationship observed between two or more variables should be unambiguous as to what it means rather than being due to “something else”.

• Selection BiasIndividuals or groups differing from one another in unintended

ways that are related to the variables to be studied • MortalityLoss of someone as the study progresses

Internal Validity (continued)

• Instrumentation: The way instruments are used • Decay – nature of the instrument, (including

scoring procedure) is changed in some way or another

• Data collection Bias – unconsciously distort the

data to make certain outcomes more likely

Internal Validity (continued)• LocationThe location creates alternative explanations for the results

• TestingThe impact of the pretest on the post-test results

• HistoryUnanticipated and unplanned for events may occur

during the course of a study that can affect the responses of subjects

• MaturationChange during an intervention may be due to the factors associated

with the passing of time rather than to the intervention itself

• Attitudes of SubjectsPerforming better because of the novelty of the treatment rather

than the specific nature of the treatment. (i.e. Hawthorne effect)

• ImplementationThe experimental group is treated in ways that are

unintended and not necessarily part of the method, yet give them an advantage of one sort or another

How to Reduce Threats

• Standardize conditions• Obtain as much information as possible on the

subjects• Organize information on the details of the

study• Choose an appropriate design!• Planning!

External Validity

Can you make generalizations based on your findings…?

External validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times.

Experimental Design

One-shot case studyX 0

One-group pretest-posttest0 X 0

Experimental

Static-Group ComparisonX 0--------------------

0

Static-Group pretest-posttest0 X 0---------------------0 0

Correlational Research

It is a measure of the association, or co-variation of two or more dependent variables.

http://frank.mtsu.edu/~sschmidt/methods/correlational.html