Upload
leonard-klein
View
27
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Assessing student competencies in experimentation Marcus Hammann Westfälische Wilhelms Universität, Münster & Maike Ehmer, Phan Thanh Thi Hoi, IPN, Kiel. Outline. Introduction Theory: SDDS-Modell ( Scientific Discovery as Dual Search ) Research Questions Hypotheses - PowerPoint PPT Presentation
Citation preview
Assessing student competencies in experimentation Marcus HammannWestfälische Wilhelms Universität, Münster
&
Maike Ehmer, Phan Thanh Thi Hoi, IPN, Kiel
• Introduction
• Theory: SDDS-Modell (Scientific Discovery as Dual Search)
• Research Questions
• Hypotheses
• Design und Methods
• Results
• Discussion
Outline
How well do students experiment?
„In Biology as well as in Physics instruction, students‘ understanding of the method of experimentation and the principles of evidence-based argumentation remain underdeveloped. According to TIMSS, only 10-15 % of the students understand simple experimental designs at the end of grade 8.“
BAUMERT (1997)
How do students plan experiments?
A student hypothesizes that minerals are important forplant growth
TIMSS 1997
In order to test her hypothesis, she needs one more plant. Which of the following should she choose?
Students do not distinguish between different types of variables . They choose an
incorrect experiment as experimental control.
Grade 7: 60%
Grade 8: 58%
How do students plan experiments?
Hammann, M. et al. (2006) Fehlerfrei experimentieren. MNU 59/5, S. 292-299
How do students plan experiments?
Water
The seeds need water in order to grow.
Air
The seeds need air in order to breathe.
Nutrients
They are in the soil. The seeds feed on them.
How do students plan experiments?
„failure to seek disconfirmation“ (KLAYMAN & HA 1987)
„from an engineering model to a science model of experimentation“ (SCHAUBLE & KLOPFER 1991)
I put the seeds ca. 3 cm deep in the soil. The seeds need water to grow, that‘s why I add water. The seeds use the air and the nutrients and grow.
How convincing is an experiment without an experimental control?
• Can a student say after this experiment, which of the following factors are necessary for seed germination?
Selected student answers (n=15, age 10-12)
• S 1 No, because each variable must be tested individually. • S 2 No, because the air is invisible.• S 3 No, because the seeds also need light and warmth. Then the
experiment would turn out well. • S 4 No, because the seeds need also warmth. • S 5 No, a plant needs no nutrients.• S 6 Yes,because the student added all the things that are necessary for
seed germination.• S 7 Yes, because the seeds grow and when the plant grows, he sees
that the plant needs the things that he added.
Hammann, M. et al. (2006) Fehlerfrei experimentieren. MNU 292-299.
OECD PISA 2006
Assessing Scientific, Reading and Mathematical Literacy: A Framework for PISA 2006.
www.pisa.oecd.org
TIMSS 1997
butter
whole meal
white floursugar
honeymargarine
The original recipe for a cake
John‘s recipe:
(Tschirigi 1980)
How do students plan experiments?
Great cake?Hypothesis: =
Which one experiment can be used to test John‘s idea?
Experiment 2
Experiment 3
+
+
+
+
+
+
Experiment 1
„change all“
„HOTAT = hold on to one thing at a time“
correct
How do students test hypotheses?
Klayman & Ha (1989)
Restricted search in the hypothesis space,
Confirmation bias
Do student relate hypotheses logically in a series of experiments?
Schauble et al. (1991)
Do students revise hypotheses if confronted with disconfirming data?
Zelltyp Mutiertes Gen
Lactose Input
[Microgramm]
-Galactosidase-Output
[Microgramm]haploid kein 100 50
haploid P 200 100
haploid I 200 876
haploid O 200 527
Dunbar (1993). Concept Discovery in a Scientific Domain. Cognitive Science 17, 397-434.
How do students analyse data?
Mike: „Because the seeds in tray 2 did not have any light and only 6°C they did not germinate, [...] because, I think, it was not warm enough for them, only 6°C, not very warm and also no light. And in tray 3, the seeds also did not have any light, but instead they had a temperature of 22°C, and that was enough for the beans to germinate. Well, this proves that [...] bean seeds need light as well as
warmth for germination.“ conclusion is in conflict with the evidence
Mike: „well, what is written here is not quite correct. He should write [...] that bean seeds need light as well as ... no ... light as well as ... no ... One can‘t say ‚as well as‘, but light and perhaps also warmth.“I: „But the seeds do need light, or not?“Mike: „They need light. But in this experiment [tray 3] they don‘t need light. [...] exceptions prove the rule It works with light, but also with warmth. So it does not matter.“
I: „So, do you think that this experiment shows that the seeds need light or not? Mike: „Well, not really. In fact they only need special warmth of 22°C [...]. But only if this photo is correct.“ [...] calls into question that the representation of the experiment may be incorrect
How do students analyse data?
•Oliver: „And in tray 3 ... there was no light ... but instead there was warmth, and the seeds germinated anyway ...“ (makes a face)•I: „ And this does not seem to make you happy? •Oliver: „No, well, I‘d rather say that warmth and light are necessary for this.“ explicit statement of student conceptions
•I: „And this can also be seen in the experiment?“•Oliver: „In tray 1 (laughs), but not so much in tray 3 ... (thinks)“•I: „And is it enough, if the results show in tray 1?“•Oliver: „Yes, I would say so, yes. One must add light and warmth.“• looks at individual components of the experiment
that confirm his expectations
How do students analyse data?
Theory: SDDS
Top-level components of the SDDS-Model
SDDSScientific Discovery
as Dual Search
search hypothesis
space
test hypothesis
evaluate evidence
David Klahr (2000). Exploring Science: The Cognition and Development of Discovery Processes. MIT Press.
Hammann, M. (2007). Das SDDS Modell. In: Handbuch der Theorien der Biologiedidaktik. Springer Verlag.
Research Questions
1. testability
Can the three dimensions of the construct “experimentation as problem solving” (SDDS Modell) be tested with paper-and-pencil test Items?
2. number of dimensions
How many dimensions should be tested? Do the three dimensions correlate?
3. domain-specifcity of the dimensions
Do the three dimensions correlate with the biological pre-
knowledge?
Design und Method
Brot backen
Kleine Küken
Samen-keimung
Apfel-wein
Knowledge
Test
8-10 Items (SMC)
8-10 Items (SMC)
8-10 Items (SMC)
8-10 Items (SMC)
Competency test
„Search in the hypothesis space“
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
Competency test
„Search in the experiment space“
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
Competency test
„Data analysis“
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
2 Items (SMC)
Design und Method
pretest 1 pretest 2 pretest 3
Main study
Test knowledge-testCompetence-
test (2 Vers.)
knowledge-test knowledge-testcompetence-
test (1 Vers.)
knowledge-TestCompetence-
test (1 Vers.)
students (n) Version 1: n = 252Version 2: n = 547
Version 1: n = 122
Version 1: n = 77
Version 1: n = 1006
Test booklet 111 (n = 73) 121 (n = 82)131 (n = 97)211 (n = 131)221 (n = 216)231 (n = 94)241 (n = 106)
111 (n = 122) 111 (n = 77) 111 (n = 1006)
Answering format
knowledge-Test: CMC
competence-Test: SMC
knowledge-Test: CMC
knowledge-Test: SMC
competence-Test: SMC
knowlegde-Test: SMC
competence-Test: SMC
Design and MethodeSample unit: Seed germination-Version 2
1. Search in the hypothesis space
A student named Jan did an experiment about seed germination. He used two pots with soil (pot1 and pot 2) and one pot with cotton wool instead of soil (pot 3). He sowed bean seeds in the three pots and put all three pots in the sunlight at a temperature of 22°C. He watered pot 1 and pot 3, but did not water pot 2.
Why does Jan do this experiment ? A Because he wants to make the seeds germinate faster. (level 0)B Because he thinks that light and warmth are necessary for seed germination. (level 1)C Because he thinks that water, soil, light and warmth are necessary for seed germination. (level 1)D Because he thinks that water and soil are necessary for seed germination. (level 2)
2. Data analysisAfter some days Jan had the following results: The seeds in pot 1 and pot 3 germinated, whereasthey did not germinate in pot 2.
Which one is the best explanation of the findings?A The experiment did not work because the seeds in pot 2 did not germinate (level 0)B The experiment showed that seeds need light and warmth to germinate (level 1)C The experiment showed that seeds need soil and water to germinate (level 1)D The experiment showed that seeds need no soil, but water to germinate (level 2)
soil / water/ light/ 22°C
soil / no water/ light/ 22°C
no soil / water/ light/ 22°C
soil / water/ light/ 22°C
soil / no water/ light/ 22°C
no soil / water/ light/ 22°C
Sample unit: Samenkeimung-Version 2
3.Search in the experiment space
Jan thinks bean seeds will germinate faster if they are put at a warm temperature. He plans an experiment to test this idea. This is Jan’s experiment. He puts the bean seeds in a pot with soil (pot 1), waters them, and keeps the pot at a temperature of 22°C in the sunlight.
Design and Methode
Soil/ light / 22°C
Pot 1
Jan needs another pot in order to compare with pot 1
Which one in the following pots (A-D) should he chose?
Pot B Pot C Pot D
soil / light/ 10°C
no soil /no light / 10°C
Pot A
no soil / light / 10°C
soil / no light/ 10°C
Pot A (level 0)
Pot B (level 1)
Pot C (level 1)
Pot D (level 2)
Item difficulties
Item difficulty for
items
Mean Item difficulty for
units
Mean Item difficulty for
booklet
Item difficulty for knowledge
test10.3% -84.6% 43% - 60% 51.7%
Item difficulty for competency
test46% - 77% 54% - 69% 62.3%
reliabilities
Reliability at unit level
Reliability at booklet
level
Unit 1: Seed
germination
Unit 2: Chicken
eggs
Unit 3: Apple wine
Unit 4: Baking bread
Reliability for knowledge
test .3201 .2742 .4845 .4097 .6307
Reliability for competency
test.6895 .6303 .7724 .7110 .8840
Cronbach’s alpha for scale “forming
hypothesis”(8 items)
Cronbach’s alpha for scale
“data analysis”(8 items)
Cronbach’s alpha for scale “planning
experiment”(8 items)
.7822 .6879 .7122
Reliability of the three scales of the competency - test
Correlations between the biological pre-knowledge and the three dimensions
Biological pre-
knowledge
forming hypotheses
analysing data
planning experiments
0.385*
0.353**
0.400**
* Correlation is significant at the 0.01 level (2-tailed)
** Correlation is significant at the 0.05 level (2-tailed)
0.785**
0.615**
0.607**
Correlations between the three dimensions of experimentation
Forming hypotheses
Planning Experiments
Analysing data
Latent class analysis
Item profile of each model
Two-class model
50
60
70
80
90
100
1 2 3
Three dimensions in experimentation (1: Search in the hypothesis space; 2: Data
analysis; 3: Search in the experiment space)
Per
cen
tag
eClass 1
Class 2
Three-class model
50
60
70
80
90
100
1 2 3
Three dimensions in experimentation (1: Search in the hypothesis space; 2: Data
analysis; 3: Search in the experiment space)
Per
cen
tag
e Class 1
Class 2
Class 3
Thre relationship between three groups of students and three dimensions in experimentation
50
100
Search in the hypothesisspace
Data analysisSearch in the experiment
space
Class 1
Class 2
Class 3
Findings und Discussion
• Assessment of student competencies in experimentation should take into consideration well-documented problems / student conceptions.
• According to an empirical model of experimentation as problem solving, three dimensions should be assessed: forming hypotheses, planning experiments, analysing data.
• In the paper-and-pencil test presented, the three dimensions test form reliable scales that are fairly highly correlated.
• A possible reason for this may be a common factor that influences performance in the three dimensions (e.g. intelligence, methodological knowledge)
• The role of the biological pre-knowledge cannot be assessed with a great amount of certainty in this study.
• Qualitative studies yielded important insights into students’ thinking and argumentation when they form hypotheses, plan experiments and analyse data.
Thank you for your attention.