Embed Size (px)
Citation preview
Experimental researchtwo examples
Christian SpannagelPädagogische Hochschule HeidelbergInstitut für Mathematik und [email protected]://www.dunkelmunkel.netTwitter: @dunkelmunkelFacebook: /dunkelmunkel
2
Experiments: researching cause-and-effect-relationships
• design– research questions and hypotheses– independent variables and experimental design– dependent variables and tests– determination of sample size
• performance• data analysis
– parametric and nonparametric tests– univariate vs. multivariate– with and without repeated measurements
• internal and external validity
planning
control
manipulation
data analysis
repeatability
3
Research hypotheses and experimental design
„teacher training with reflection on lesson videos leadto higher performance than teacher training without videos.“
Method samplevideos a1
no videos a2
one-factorial design
Control confoundingvariables
ramdonization
parallelization
4
Research hypotheses and experimental design
„In addition, the type of the task has an influence on performance.“
Method Task A Task B No taskvideos a1b1 a1b2 a1b3
no videos a2b1 a2b2 a2b3
Two-factorial design (2x3)
5
Research hypotheses and experimental design
„The software for video discussion supports subjectswith high computer self-efficacy more than subjects withlow computer self-efficacy.“
Methode CS- CS+videos a1b1 a1b2
no videos a2b1 a2b2
two-factorial design (2x2)
ATI (aptitude treatment interaction; Cronbach & Snow, 1977)
6
Dependent variables and decision for tests
• e.g. learning success, motivation, ...• statistical tests
– parametric tests (t-Test, analysis of variane (ANOVA), …)
• Assumptions / preconditions
– metric scale
– normal distribution
– Homogeneity of variance
– …
– nonparametric tests (Mann-Whitney U, Wilcoxon, Bredenkamp, …)
• Less preconditions
7
Data analysis
Method CS- CS+Videos μ1,1 μ1,2
no videos μ2,1 μ2,2
Results PCK
Analysis of variane → main effect method, main effect computer self-efficacy, interaction effect
8
Main effectsM
eth
od
en
wis
sen
Me
tho
de
nw
isse
n
9
Interaction effect
Me
tho
de
nw
isse
n
10
multivariate anaylsis of variance
Method CA- CA+videos (μ1,1, ν1,1) (μ1,2, ν1,2)
No videos (μ2,1, ν2,1) (μ2,2, ν2,2)
Results (PCK, PK)
Außerdem beachten: VA mit Messwiederholung!
11
Internal and external validity
• Internal Validity: IV → DV?– Maturation
– History
– Testing
– Selection
– …
• External Validity– Generalizability to
• other persons?
• other situations?
• …
(Campbell & Stanley, 1963)
12
Two examples
using complex softwarein schools
eye tracking experiment
Using complex software in schools
Relevant theories
Cognitive Load TheoryTheory of Multimedia LearningManuals with text and picturesMinimal Manuals & Guided ExplorationScreen casts & animated demonstrationsTraining wheels (reduced user interfaces)Computer self-efficacy expectations……… ...
Research questions
Do animated demonstrations and training wheels support students while learning with a complex software in math lessons?
What effect does computer self-efficacy have on the students' performance in this context?
What effect do the different learning environents have in interaction with computer self-efficacy?
Hypotheses
Animated Demonstrations and training wheels support students while learning with a complex software in math lessons. (main effect treatment).
Students with high computer self-efficacy are more successful than students with low computer self-efficacy (main effect computer self-efficacy).
Students with low computer self-efficacy profit more by animated demonstrations and training wheels than students with high computer self-efficacy (interaction effect).
Experimental design
TEXT ANIM
CSW-
CSW+
ANIM+
Sample
TEXT ANIM
CSW-
CSW+
ANIM+
N = 1727 classes (8th grade)
27 29 31
30 28 27
Randomization!
Learning environment
Written manual (TEXT)
Animated demonstration (ANIM)
+ gesprochener Kommentar
ANIM + training wheels
ANIM ANIM+
training wheels
Plan and dependent variables
pretest on mathCUSE
Post test Mathpost test CALC
IMIevaluation
Follow-Up FI
3-5 Wochen 4-8 Wochen
Treatmentmax 90 min.
Process variables
Some results
0
100
200
300
400
500
600
700
TEXT ANIM ANIM+
Treatment
Bea
rbei
tun
gsz
eit
(Sek
.)
CSW-
CSW+0
1
2
3
4
5
TEXT ANIM ANIM+
Treatment
Erg
ebn
is
CSW-
CSW+0
1
2
3
4
5
TEXT ANIM ANIM+
Treatment
Erg
ebn
isCSW-
CSW+
0
50
100
150
200
TEXT ANIM ANIM+
Treatment
Hil
feze
it (
Sek
.)
CSW-
CSW+
Some results
0
100
200
300
400
500
TEXT ANIM ANIM+
Treatment
Leh
rmit
telz
eit
(Sek
.)
CSW-
CSW+ 0
100
200
300
400
TEXT ANIM ANIM+
Treatment
Ein
füh
run
gsz
eit
(Sek
.)
CSW-
CSW+
0
2
4
6
8
10
12
14
TEXT ANIM ANIM+
Treatment
Rü
ckg
riff
shäu
fig
keit
CSW-
CSW+
Motivation (IMI) results
Bifaktorielle, multivariate Varianzanalyse
TREATMENTCSWINTERAKTION
7,39
0,68
0,65< 0,001
> 0,05
> 0,050,15
0,02
0,02
F Sig. part. η²Effekt
27
Two examples
using complex softwarein schools
eye tracking experiment
28
Training Wheels
Research questions
Do training wheels increase the findability of icons?
Do full interfaces increase the awareness of icons neveruser before?
30
Eye Tracking
Eye Tracking
31
Areas of Interest
32
Eye movements
1200
383
4216
5233
6158
7757
8166
9233
1048211416 12
399
13316
14441
15541
162586
33
Research plan
Full interface
reduced interface
Reduced & rearrangedinterface
Full interface
search task
3 seconds
Full interface
10 seconds
reduced interface
reduced & rearranged interface
Interface in the post test (full interface)
39
results
51 teacher students (43 female, 8 male)
40
Discussion
Do you have any questions?
