Experimental Comparison of Experimental Comparison of

Inquiry and Direct Instruction in Inquiry and Direct Instruction in ScienceScience

Funded by the National Science Foundation’s Interagency Education Research Initiative (IERI/NSF 04-553) Award #0437655

Dr. William CobernDr. William CobernDepartment of Biological Department of Biological SciencesSciences

Dr. David Schuster Dr. David Schuster andand Betty Betty AdamsAdamsDepartment of PhysicsDepartment of Physics

The opinions expressed in this report are the sole responsibility of the researchers

Background to the StudyBackground to the Study

Inquiry-based or direct instruction in science?

Long-standing educational and political debates

Pendulum swings

Direct Prevalent in past

Inquiry National Science Education Standards

Constructivism?

A theory of learning – either way

We know about• Experientially-based √• Active-engagement √

But these can occur in both Inquiry and Direct

Both can be ‘hands-on’ and ‘minds-on’ if so designed.

Research Evidence?Research Evidence?

Our QuestionOur Question Is NOT

whether active-engagement experiential-based learning of science is more effective than passive non-experiential learning

But IS whether an inquiry approach or a direct approach to experientially-based instruction is more effective for science concept development

…when both approaches are expertly designed and well executed

This is the question our research addressed.

Inquiry-based Science InstructionInquiry-based Science Instruction

There are now many inquiry-based curricula and lessons

Their evaluations show these to be ‘successful.’

But compared to what?

Assumption is that the merit/superiority of inquiry science instruction is well-established

However, meta-analysis of many such studies (Educational Development Council, 2007) found

Not sufficiently unconfounded to draw the inferences

Research rigor has declined from 1984 - 2002

Threats to Validity of StudiesThreats to Validity of Studies

• Few controlled comparative studies

• Pitted against poor or nebulous ‘traditional’ instruction

• Few use randomized assignments or quasi-experimental controls for differences

• Evaluations not independent of developers or researchers

• Insufficient specification to allow replication

• Fidelity: implementation not compared to intended instruction

Direct InstructionDirect Instruction

• Well-designed direct instruction

• Some relevant work:

• Kirschner et.al. – Why mimimally-guided instruction does not work

• Klahr & Nigam – ‘Direct’ vs. ‘discovery’

• Ausubel – Issue is meaningful learning vs. rote learning

Instructional ContextInstructional Context

5 experienced middle school science teachers

8th grade students responding to flyers sent home by school district offices

2 week voluntary summer program with students randomly assigned to treatment group

In-class instruction only; intrinsic motivation

Tenets

Specificity

Fidelity

Objectivity

Transparency

Research Research FrameworkFramework

Specificity

Inquiry or Direct?Inquiry or Direct?

Specific about…Specific about…

A. The meanings of Inquiry and Direct

B. The nature and design of instructional

units

C. The assessment

A.A. What do we What do we meanmean byby ‘Inquiry’ and ‘Direct’ ‘Inquiry’ and ‘Direct’ Instruction?Instruction?

Single-word descriptors are vague, ambiguous and open to (mis)interpretation Such as “inquiry, direct, discovery,

didactic, conventional/traditional, lecture, active-engagement, hands-on …”

We need to specify models for each mode

1. Model of Guided-Inquiry 1. Model of Guided-Inquiry Science InstructionScience Instruction

PHASE DESCRIPTIONExploration Observing phenomena, posing questions,

exploring, investigating … (Science-in-the-making)

Concept Formation

Guided formation of relevant concepts. Formulation of principles, laws or models

Concept Application

Applying the concepts and principles to new situations. To solve problems, explain and predict.

The Learning Cycle

Karplus Cycle

Karplus Cycle DiagramKarplus Cycle Diagram

Epistemology: Exploration leads to concept formation Inductive aspects

(Note: 5-E Learning Cycle has Karplus at its heart)

2. Model of Direct 2. Model of Direct InstructionInstruction

Epistemology:Presentation then illustration/confirmationDeductive

PHASE DESCRIPTION

Concept Presentation

Presentation/reception of facts, concepts, laws etc. Ready-made-science. Explanation & clarification

Illustration & Confirmation

Illustrate with examples, demonstrations.Verification experiments.

Concept Application

Applying the concepts and principles to new situations. To solve problems, explain and predict.

The Direct-Active Cycle

No Caricatures or No Caricatures or Straw-man ComparisonsStraw-man Comparisons

Direct caricature: Pure didactic presentation

with passive reception/absorption

Inquiry caricature: Open discovery (unguided chaos)

Hands-on alone does not make it inquiry

Lessons as CompositesLessons as Composites

Lessons have many constituent parts

All lessons are composites

Never 100% inquiry or direct throughout

To attempt this would be poor instruction generally

The essential difference?The essential difference?

What then is the essential difference between Inquiry and Direct?

“How students come to the concept”

Through exploration or are they told upfront?

This is the Active Agent that differs between modes

Lessons may have other Common Constituents

‘‘Active Agent’Active Agent’Example from Dynamics

How do students come to the force-motion law?

i. By exploring system behavior and proposing a law, OR

ii. By being given the law and confirming system behavior.

Added value, beyond just content knowledge?

Besides acquiring knowledge of the law, what else do students learn or gain, in one mode or the other?

B.B. THE INSTRUCTIONAL THE INSTRUCTIONAL UNITSUNITS

The heart of the study – Two important science topics

– Significant conceptual development sequence

Each written in Inquiry and Direct modes

Student and teacher booklets for each

Two Approaches (Two Approaches (InquiryInquiry//DirectDirect))

Essential Similarities Content

Objectives/goals

Equipment/materials

Practice problems

Assessment

Essential Differences

SequenceEvidence before claimsClaims before evidence

Teacher’s roleAsks… Tells…

Student’s experience … finds out … confirms

INQUIRY (investigative approach)

Exploration

Guided Concept Development

Application

Student learns by experiencing scientific inquiry, guided toward developing and applying scientific concepts & laws

(based on Karplus Learning Cycle)

DIRECT (confirmatory approach)

Delivery/Explanation

Verification

Application

Student learns by receiving, verifying, and applying scientific concepts & laws

InquiryInquiry andand Direct Direct Instructional Instructional ModelsModels

C.C. THE ASSESSMENT THE ASSESSMENTNature and quality of assessment is crucial

The project data depends on it

Tests understanding of the main science concepts

Problem-based Ability to apply concepts in (relatively) new

situations Bloom taxonomy levels 2 and 3 Conceptual MCQ form ‘Assessment as curriculum’ - examples as

indication

Assessment: examplesPrediction question

Amy is pushing on a wagon containing some packages. The wagon has good wheels so that friction is very small and the floor is level.

If Amy keeps pushing continuously, with a constant pushing force, and friction can be ignored, then the wagon will…

A. not move. B. move with a constant speed. C. speed up steadily. D. slow down.

How confident are you of your answer? i. Very confident. ii. Somewhat confident. iii. Not confident at all

Why?

14. An ice skater first gets up to speed, then stands on one skate and keeps going steadily across the ice without any apparent effort. The reason for her keeping moving like this without effort is that…

A. the ice must be sloping slightly downwards. B. the air is pushing her forward. C. this is the natural behavior of objects with

no net force on them. D. the force that got her moving is still acting on her.

How sure are you of your answer? i. Very sure. ii. Somewhat sure. iii. Just my best guess.

Assessment: examplesExplanation question

6a. The diagram below shows the earth relative to the light coming from the sun at a particular time of the year.

At this particular time of the year, what season(s) will be occurring in the northern and southern hemisphere? Circle A, B, C, or D.

Answers Northern Hemisphere

Southern Hemisphere

A Summer Summer

B Summer Winter

C Winter Winter

D Winter Summer

Assessment instrumentsAssessment instruments

22-question set for each topic unit 2 to 4 questions on each central

concept Identical Pre- and Post-tests Ascertain gain

“Prepare and Verify”PrepareMonitor Evaluate

FidelityFidelity

• Teachers blind to the assessments — no teaching to the (known) test

• Independent evaluators marked the assessments blind to student group assignments

• Independent observers rated teaching: to nature and degree of inquiry or direct

ObjectivityObjectivity

Transparency of research: We make available details of what the research involved, thus facilitating possible replication.

Our work is made available in detail at http://www.wmich.edu/way2go/

TransparencyTransparency

AnalysisAnalysis

Data aggregated over 2 years of trials (2007, 2008)

N=180 students (72 Direct, 108 Inquiry… to date)

Pre- and Post- assessments yielding each student’s raw percentage gain scores

Student gain scores normalized

Comparisons of both raw and normalized gain scores across various groups (modes of instruction, teachers) (t-test, ANOVA, α=.05)

Findings Findings (Example of raw % (Example of raw % gains)gains)

Total 07-08 LIGHT gain 13.6%, SD=15.3, is statistically significant (t(179)=11.934, p< .001), with an effect size (Cohen’s d) of .69(effect size of 1.4 for normalized gain)

Gain 14.1%, SD=16.4, is statistically significant (t(107)=8.925, p< .001), effect size (Cohen’s d) .67 Total 07-08 DYNAMICS gain 9.7%,

SD=13.5, is statistically significant (t(179)=9.655, p< .001), with an effect size (Cohen’s d) of .54 (effect size of 1 for normalized gain)

Findings Findings (Light Unit (Light Unit Summary)Summary)

Findings Findings (Dynamics Unit (Dynamics Unit Summary)Summary)

FindingsFindingsLIGHT Unit – Two Trials (2007, 2008)

Direct vs. Inquiry normalized gain mean difference of 3.8%

was not statistically significant (t(178)=.755, p=.451)

(std. error diff. 5.1, effect size Cohen’s d=.12)

Ann (Direct) and Tom (Inquiry) had a mean difference in raw gain of 7.6%, which was statistically significant (t(73)=2.132, p=.036), but the mean difference between their normalized gain scores was not (t(73)=1.857, p=.067)

DYNAMICS Unit – Two Trials (2007, 2008)

Direct vs. Inquiry mean difference of 3.1%was not statistically significant (t(178)=.717,

p=.474) (std. error diff. 4.4, effect size Cohen’s d=.11)

ConclusionsConclusions• Given natural class and teacher variation in

realistic classroom situations, good inquiry and direct instruction led to similar understanding of science concepts and principles in comparable times.

• Thus advocacy of either method cannot be based on science content acquisition alone.

• Inquiry-based instruction offers significant potential advantages for science education by modeling scientific inquiry during concept learning: these concomitant benefits would need to be studied in research for that purpose.

• However for science concept understanding, expertly designed instructional units, sound active-engagement lessons, and good teaching are as important as whether a lesson is cast as inquiry or direct.

http://www.wmich.edu/way2go/

Funded by the National Science Foundation’s Interagency Education Research Initiative (IERI/NSF 04-553) Award #0437655

The opinions expressed in this report are the sole responsibility of the researchers.

Controlled comparative study Controlled comparative study Treatment and control groupsTreatment and control groupsActual classroom situationsActual classroom situations

General Design ConsiderationsGeneral Design Considerations

Substance

Coherent development

Conceptual

Challenging

Experiential

Standards

Learning objectives

Clear main focus

‘Hands-on / minds-on’

Engagement

Reinforcing examples

Application

Reflection

Length?