Conducting Educational Research: Ch. 9- Analyzing and Interpreting Experimental Research

EDCI 696Dr. D. BrownPresented by: Kim Bassa

Targeted TopicsAnalysis of dependent variables and different types of dataSelecting the appropriate statistic for experimental design and data Interpreting experimental analysisDisplaying experimental design dataDiscussing experimental results

The “Best” FitIt is important to choose the best design to…

Fit your research questionAddress validity issuesSelect ways to measure your dependent variable

Experimental Research

Experiments that have treatments but do not use random assignment to make comparisons are called

quasi-experiments.

Experimental research and Quasi-experimental are alike in that they attempt to determine if an independent variable had a

direct impact on a dependent variable.

Throughout the text they are both referred to as experimental research

Experimental DataData from experimental research are quantitative or qualitative.You will collect numerical data on your variables in order to conduct your analysis. You will use inferential statistics to make comparisons between conditions in an experimental intervention or across/between experimental groups.

Types of Data:

• Ordinal• Nominal• Interval• Ratio (Scale)

Statistic Selection

Statistical Tests rely on certain assumptions in order to provide accurate information.When assumptions for a statistical test are violated (or not tested), the results of the analyses will be invalid. In each experiment prior to the analysis, the researcher will set the level of significance, which is the probability.

Statistic Selection

Level of Significance is

represented by the small letter…

• p

Level of Significance is also

represented by the Greek letter

for…

• alpha

5% probability statistical differences

in an analysis would be due to chance or

measurement error is represented by…

• p<.05

Level of Significance (probability)

P<.05 means there is 95% statistical probability that differences are due to the intervention.

Statistic SelectionSelecting or matching the right statistic depends on the

type of data you and whether you want to compare mean scores for the dependent variable across groups or

conditions.

Statistic SelectionPutting It All Together

(Table 9.2)

Can students find main idea after learning a new

reading strategy?

Design: Post –onlyData: correct/incorrect or percentage correct (nominal or interval)

Statistic: descriptives

Do Students in Algebra I classes who engage in the XYZ curriculum perform significantly different on the state tests than students who

do not?

Design: Comparison group

Data: correct/incorrect or percentage correct (nominal or interval)

Statistic: chi-square or ANOVA

Statistic Selection

To compare pre/post nominal data

Use chi-square

To compare percentage c

orrect interval data

Use ANOVA

The t test only test means between 2 groups

The ANOVA tests scores for multiplegroups at a time.

Statistic SelectionTwo-way ANOVA: two independent variables with one rating to analyze

ANCOVA: another variable accounts

for difference in or covary with the

dependent variableMANCOVA:

several ratings to analyze as separate

dependent variables or any

suspected covariance

Main Effects: yielded results after analyzing multiple

variables, independent or

dependent

Interaction Effects: the interaction of the

effects of two or more independent

variables on a dependent variable

Post hoc: follow ups to the original

statistical test (i.e. Bonforonni or

Tukey)

Interpretation of ResultsData Entry

Software: Statistical Package for the Social Sciences (SPSS) or Statistical Analysis Software (SAS)Use a new computer file Enter data collection accuratelyKnow how to use software menus and commands

Understanding OutputKnow the statistic for the test that you run (i.e. F statistic for ANOVA )Know the level of significance (i.e. p<.05)Know the degrees of freedom (df)- approximately equal the number of participants for your data and used in the statistical calculation of the level of significance

Interpretation of Results

Reporting Experimental Results

The results section is where statistical out-comes are reported and only

includes factual information from the data-base outcomes.

The effect size (degree of difference between groups or conditions) is also reported in

the results section.

Interpretation of ResultsData displays provide the advantage of visually analyzing data. frequency tables and histograms are two useful displays and are commonly use to report data.

A frequency table is used to display nominal or categorical data.

A histogram display the relationship between two variables whose measures yield continuous scores.

Discussing Experimental ResultsThe researcher has the opportunity to interpret the results in the discussion section.This section should also include limitations, implications for practice, and future research needs.

Limitations

•Shortcomings

Implications for Practice

•How results can be used in the classroom or other use

Future Research Needs

•New/Improved ideas for research or an extension to the current research

Statistical Conclusion Validity

Statistical Conclusion Validity: is based on reliable implementation of independent variable as well as appropriate and correctly used measures and statistics, in order draw conclusions regarding the effect of the independent variable on the dependent variable.

Fidelity of Treatment Implementation:is based on the treatment being implemented reliably enough to know it was the cause of effects

Statistical Conclusion Validity

A small sample may contribute to a low

statistical power.

Low Statistical power means that it is less likely that the

statistical test could find statistical difference

Type I and Type II Error

SummarySelect the appropriate statistic

Enter the data

Interpret the results

Enhance your analysis through data displaysRemember that inferential analyses of experimental research lead to statistical levels of significance, not necessarily practical levels of significance (McMillian, 2004).