EXPERIMENTAL

RESEARCH

SELÇUK KARAYAPRAKNİHAN ALPEREN

• Quantitative Research helps researchers to prove their educational or psychological phenomena/hypothesis as true or false. Quantitative research also provides evidence via objective observations which are related with phenomena/hypothesis (Mertens, 1998).

• Quantitative Research provides statistical (numerical) information with percentages, graphs, charts, etc. and describes findings with numbers counter to Qualitative Research which describes findings with words.

• Experimental Research takes part in Quantitative Research Approaches. (Mertens, 1998, Airasian et al, 2006)

Qualitative Quantitative

Experimental Research

EXPERIMENTAL

• We are teachers, tutors, instructors or educators, but all of us are related with education. We may detect a problem while we are in classrooms or at home. Learner or learners may have a problem about something. After that we can talk this problem with other teachers and we may find out that other learners have same problems, then we can search it simply on internet. Again we can find out similar statements on internet. After a while we can design a treatment or teaching strategy to solve the problem. We should check our new strategy/treatment if it is useful and valid. EXPERIMENTAL RESEARCH.

Process

Process

• While doing Experimental Research, the steps are actually as like as other research types.

• The researcher selects and defines the problem, does literary analysis, selects participants, decides measuring instruments, prepares research plan, collects data, analyzes data, and presents the results.

• The researcher should have at least one hypothesis. The hypothesis determines what kind of difference is expected to occur between the independent and dependent variables to be able to support or refute the hypothesis.

• Airasian et al (2006) mentioned that there are three types of comparing independent and dependent variables as following;– Comparing two different methods (effects of

Computer-based and teacher-based teaching)– Comparing a new method and traditional method

(New hand-writing method vs traditional hand-writing method)

– Comparison of different amounts of single method (the effect of 20-min speaking session versus 40-mins speaking session)

• In Experimental Research Studies, there are two variables which are called;– Dependent (Control) variable,– Independent (Treatment/Experimental) variable.

Independent (Treatment/Experimental) Variable

• One of the group that researcher manipulates and applies his/her new strategies. The researcher tries to prove that his/her strategy is better or different than others.

• In educational research with experimental resaerch, researcher manipulates to method, instruction, arrangement of learning classroom, materials and duration of treatment. (Airasian et al 2006)

Dependent (Control) Variable

• Traditional method or different method is applied on control variable. This group is here to be able to observe what kind of differences occurs in the independent group.

• Characteristic features (such as gender, age, level of learners...etc) and other features (classroom resources, time of lessons,etc) are equal and only the teaching strategy is different between the two groups to be able to measure and analyze the effects of teaching strategy.

Role of the

Researcher

• The researcher selects and identifies the groups,• The researcher decides which group take which strategy,• The researcher controls every condition • The researcher tests, observes, compares the dependent

and independent variables, then presents the differences between these groups, finally states if his/her hypothesis is supported or refuted by the results.

• The most important role of the researcher is to be sure about dependent and independent groups have equal conditions and the only different thing is the treatment or teaching strategy (or what is related with hypothesis)

Validity and Reliablity

• There are Internal Validity and External Validity in Experimental Research.

• Internal Validity: Airasian et al (2006) defines it as “the degree which observed differences on the dependent variable are a direct result of manipulation of the independent variable, not some other variable” (p.598).

• External (ecological) Validity: Airasian et al 2006 defines it as “the degree to which study results are generalizable, or applicable to groups and environments outside the experimental setting”(p.597).

• To achieve maximum internal validity, the researcher should control everything (on participants, teachers, members of dependent group ..etc.) and create laboratory environment in the school. But if the researcher achieve maximum internal validity then the research study has poor reality and generalizability. Although, if the researcher tries to achieve maximum external validity, then the researcher cannot control independent and dependent groups. Thus researcher should balance external and internal validity.

• The researcher can first do his/her research with maximum internal validity and then, the researcher can redo it with external validity to test the internal validity to be sure that findings are generalizable (Airasian et al, 2006)

Threats to Internal Validity

1. History: It refers to any event that occurred during the experimental research study and it affected the results of the study. Make the other group to experience with same event can help the researcher to solve the problem.

e.g. If the researcher gives chocolate to students who read a book, he/she should give same thing in both dependent and independent groups.

2) Maturation: refers to psychological and biological changes during the study which may affect participants’ performance. (such as becoming stronger, older, wiser, bored or tired) . If both independent and dependent groups experienced with same maturation factors, then it is not a problem.

3) Testing: This problem occurs when the researcher uses pre-test and post-test in research study. When participants took pre-test, they became familiar with test, to be able to solve this, both independent and dependent groups take both pre and post-tests. The participants also focus on only test questions so the researcher may not decide new teaching strategy is better or they are familiar, so there should be time between pre and post-tests to solve this threat.

4) Instrumentation: happens when the researchers used pre-tests and post-tests with different difficulty levels, different observation tools and mechanical device failures in experimental research studies. The researcher can solve the problem with using pre-tests and post-tests in same difficulty level, using only one observation tools and be careful about the mechanical devices.

5) Statistical Regression: occurs when the researcher chooses from only extremely low or high level students as participants. Extremely Low level participants cannot perform lower and extremely high level ones cannot perform better.

6) Differential Selection of Participants: The researcher should arrange the independent and dependent groups with equal characteristics. If the researcher does not do this, results are not valid. (for ex. In one classroom, all of the students may be very successful in English and the other classroom may be beginners, in one classroom older students, in the other classroom younger). To solve it, the researcher can divide all participants first gender, then age and then their grades.

7) Mortality: Participants may drop out the research study for any reason and this may affect the results. To solve this problem and keep groups’ characteristics similar, the researcher can remove a participant with same characteristics who dropped out from other group.

Threats to External Validity and other threats on validity

1) Multiple-Treatment Interference: occurs when the same participants receive more than one treatment. Participants may be under effects of earlier treatment. This limits the generalizability.

2) Selection-Treatment Interaction: occurs when the participants of a variable react differently to the treatment than other potential participants in the population reacted or accessible population is different than the researcher’s target population. (e.g. The researcher is not accepted from 3 schools which fits to researcher’s population sample requirements and 4th school may accept the researcher which does not fit.)

3) Pre-test-Treatment Interaction : Pre-test may make participants notice the treatment and they may become more sensitive and interested in pre-test subjects while receiving the treatment. This may affect participants results and it may be different if they have not had pre-test. This affects generalizability. (while watching a film for educational purposes, pretested group may observe the film and the group without pre-test may enjoy the film as a comedy film).

4) Treatment Diffusion: Participants of dependent and independent groups’ may speak with each other and they can learn from each other. Automatically, participants may use traditional or new strategy items which is not valid. Teachers can also learn from each other and uses in their classrooms. To reduce this, the researcher can warn teachers not to speak about new strategy and control the students to reduce speaking rates.

5) Experimenter Effects: The researcher’s their own threat on their own research study. If the researcher knows some participants, and their performance, the researcher may evaluate differently. To solve it, the researchers can use blind-scoring where you do not know which test paper belongs to whom.

• Reactive Arrangements (Participants’ effects): occurs when participants act differently than expected.

• John Henry Effect: John Henry worked for a railroad company. He was a steel drivin’ man and when he heard that a steam powered machine is going to take him and his team’s place he worked hard to provide he was better than machine, as like as this students who believes that traditional way is better, they could behave atypically or they might not accept the new treatment.

• Placebo effect : Medical researchers use this. Subjects take water and sugar instead of real medicine and again they can feel better, in experimental research, subject may look like perform better with new strategy but it might not be the reality.

• Novelty effect: New treatment/strategy may increase learners’ interest, motivation and participation to the lesson. But, the researcher should be sure about if the strategy is really beneficial or it is just a new and different way which isinteresting for learners. To be able to solve this problem the researcher should apply the new strategy till it becomes quite old for students.

• Thank you

References

• Airasian, P., Gay, L. R., & Mills, G. E. (2006). Educational Research; Competencies for Analysis and Applications (8th Edition ed.). Upper Saddle River, New Jersey, USA: Prentice Hall.

• Mertens, D. M. (1998). Research Methods in Education and Psychology: Integrating Diversity with Quantitative & Qualitative Approaches. (C. D. Laughton, Ed.) Thousand Oaks, California, United States of America: SAGE Publications.

• The Variables Song [Video File]. Retrieved 20.12.2012 from http://www.youtube.com/watch?v=Hxbz656Euyw