Upload
sonia-rippe
View
216
Download
0
Embed Size (px)
Citation preview
Research design
IMD09120: Collaborative Media
Brian Davison 2010/11
• Recap• Variables• Types of design• Hypotheses• Sampling• Your designs
Research design
Recap: tests you have done (1)
• Can the number of social networking accounts predict the number of contacts a person has?
• Do female students have significantly more social networking contacts than male students?
• Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
• Do students from different schools differ significantly in the number of social networking contacts they have?
Recap: tests you have done (2)
• Do students differ significantly by age in the number of social networking contacts they have?
• Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
• Is the preference for Facebook statistically significant?
• Is preferred social networking site related to gender?
Recap: tests you have used
• Correlation• Independent t-test• Paired t-test• ANOVA
• 2
Variables
• Things you can measure– Height, weight, age– Number of car journeys– Gender, nationality
– Attitude, belief, satisfaction
Types of variable• Continuous
– Infinitely divisible
• Discrete– Values come in defined
ordered steps
• Categorical– Values are categories with no
ordering
• Examples– Height– Number of car journeys– Gender– Nationality– Observed symptoms of disease– Attitude– Age– Belief– Weight– Satisfaction
Dependence• Independent variable
– Manipulated by the investigator
• Dependent variable– Measured for effect
• Examples– Does face-to-face differ
significantly from WebCT chat as a channel of communication in terms of satisfaction?
– Is preferred social networking site related to gender?
Confounding variable
• A factor that affects the dependent variable and which is not included in the design
• What confounding variables might there be in the F2F-chat experiment?
Research designs
• Correlational – Looking at how one variable changes in relation to another– eg smoking and incidence of cancer
• Experimental and quasi-experimental– Looking for effects of one variable on another– IV and DV
Correlation designs
• Collect data about both variables• Measure the extent to which their distributions match• Causation cannot be inferred from a correlation
• Which is correlational:
1. Can the number of social networking accounts predict the number of contacts a person has?
2. Do female students have significantly more social networking contacts than male students?
3. Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
Experimental designs
• Researcher manipulates IV• Subjects are randomly assigned to test conditions
• Which is experimental:
1. Can the number of social networking accounts predict the number of contacts a person has?
2. Do female students have significantly more social networking contacts than male students?
3. Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
Quasi-experimental designs
• Some variables cannot be manipulated– eg gender, occupation– Therefore random allocation to conditions is impossible
• Which is quasi-experimental:
1. Do students differ significantly by age in the number of social networking contacts they have?
2. Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
3. Is the preference for Facebook statistically significant?
Between or within subjects
• Between-subjects design– Different subjects randomly assigned to different conditions
• Within-subjects design– Same subjects used in all conditions
Within-subjects
• Also called related or repeated measures
• Pro:– Using same subjects helps to eliminate confounding variables– Fewer participants needed
• Con:– Subjects may perform differently on second test – ordering effects– Subjects more likely to guess the purpose of the study – demand effects– Not possible for quasi-experimental studies
• Ordering effects can be eliminated by counterbalancing
Between-subjects
• Also called independent or unrelated• May be significant variation within each group
• Pro:– No ordering effects– Demand effects are less likely
• Con:– More participants required– Less control over confounding variables
What kind of design?
• Do female students have significantly more social networking contacts than male students?
• Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication?
• Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?
Population
Pop. mean
Sampling error
Pop. mean
SamplePopulation
Spl. mean
What rule connects this series of numbers?
10, 20, 30
• Confirmation bias
Testing hypotheses
• Null hypothesis– There is no relationship / difference between two variables– ie. All observed differences are the result of sampling error– Probability values in tests = probability of the observed result if the null
hypothesis is true
• Alternative hypothesis– There is a relationship / difference between two variables– Specific to the study you are doing
Logic of hypothesis testing
1. Formulate a hypothesis
2. Measure the variables and examine their relationship (using samples)
3. Calculate the probability of the result if the null hypothesis is true
4. If the calculated probability is small enough, reject the null hypothesis
Two types of test
Is x significantly larger / smaller than expected?
Is x significantly different from expected?
How many tails?
• Do female students have significantly more social networking contacts than male students?
• Does face-to-face differ significantly from webCT chat as a channel of communication in terms of satisfaction?
• Do students from different schools differ significantly in the number of social networking contacts they have?
Short break
Effect size
Small effect Large effect
• A large effect size is easier to detect than a small one• The power of a test is a measure of its ability to detect an effect• More participants are required to detect a small effect size
Cohen’s power primer
• Tabulates required sample size according to– Confidence interval– Statistical test– Effect size
• Values for 95% confidence
• Cohen (1992)
Test Small Medium Large
Pearson’s r 783 85 28
T-test 393 64 26
ANOVA sample sizes for 95% confidence
Groups Small Medium Large
2 393 64 26
3 322 52 21
4 274 45 18
5 240 39 16
6 215 35 14
7 195 32 13
2 sample sizes for 95% confidence
df Small Medium Large
1 785 87 26
2 964 107 39
3 1,090 121 44
4 1,194 133 48
5 1,293 143 51
6 1,362 151 54
• More precise figures can be calculated using G*Power
How many variables do you have?
Are you looking for differences between conditions, or
relationships among variables?
Are you looking for differences between conditions, or
relationships among variables?
Do you have a between-participants or a within-
participants design?
Do you want a regression equation,
or simply the strength of a relationship?
Are you interested in a regression equation, or exploring clusters
of correlations?
Do you have more than one DV?
Do you have more than one IV?
Do you want to look for differences between conditions while controlling for the effects
of another variable?Independent
t-testRelated
t-testPearson’s
Product Moment Correlation Coefficient
Linear regression
Multiple regression
Factor analysis
One-way ANOVA ANOVA for multiple IVs
MANOVA
Analysis of covariance
Two More than two
Differences Relationships DifferencesRelationships
Between Within Strength Regression Regression Correlation
Yes
Yes
Yes
No
No
NoDancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)
How many variables do you have?
Are you looking for differences between conditions, or
relationships among variables?
Are you looking for differences between conditions, or
relationships among variables?
Do you have a between-participants or a within-
participants design?
Do you want a regression equation,
or simply the strength of a relationship?
Are you interested in a regression equation, or exploring clusters
of correlations?
Do you have more than one DV?
Do you have more than one IV?
Do you want to look for differences between conditions while controlling for the effects
of another variable?Independent
t-testRelated
t-testPearson’s
Product Moment Correlation Coefficient
Linear regression
Multiple regression
Factor analysis
One-way ANOVA ANOVA for multiple IVs
MANOVA
Analysis of covariance
Two More than two
Differences Relationships DifferencesRelationships
Between Within Strength Regression Regression Correlation
Yes
Yes
Yes
No
No
NoDancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157)
Your turn
• Design a study to test whether caffeine affects level of anxiety.
Your turn again
• Design a study to test whether mathematical ability is related to musical ability
One more time
• Design a study to test whether there is a relationship between doing sport and intention to vote
Coursework: variables and hypotheses
• What are you trying to improve?– eg group identity, social presence, group effectiveness– This is your DV
• What are you changing?– eg additional features, different layout, modified presentation– Your IV is WebCT version: may be old/new, or +/- combined factors
• What do you expect?– Eg adding feature X leads to reduced social loafing– This is your hypothesis– Null hypothesis is “there is no difference”
Coursework: research design
• Levels of IV
• How to measure DV
• Experimental groups– Within subjects or between subjects
• Eliminating confounding variables and unwanted effects– Quality of system – create an “as-is” Powerpoint– Ordering effects– Demand effects– Researcher bias
Statistical test
• Method 1– What question are you asking?– Which practical example matches?
• Method 2– Follow the flowchart
• Use one method to check the other