Upload
karsten-d-wolf
View
1.657
Download
2
Embed Size (px)
DESCRIPTION
Keynote Young Researcher's Workshop at ITK07, Finland Abstract: Online settings provide non-reactive ways to observe learners' activities. While the learning itself is not directly visible for example in a logfile or within the history of a Wiki, the growing expressiveness in online learning environments takes us a step nearer to witness and analyse learning processes. In this presentation four questions will be discussed: (1) What is the relation between possible user activities in recent E-Learning environments and actual learning processes? (2) What data is available to observe the users' activities? (3) What data analysis methods are fitting to model these activities? (4) What research designs do we need to get more insight into learning processes? As an example, some ideas, experiences and results from an ongoing research project about online user participation and motivation will be presented and discussed, focussing on a multimethod approach.
Citation preview
cc by Karsten D. Wolf 2007
Karsten D. WolfDidactical Design of Interactive Learning Environments
interaktiivinen tekniikka koulutuksessa 2007Wednesday, 18.04.2007
Online Learning Processes –What Users Do and Why it is Important
1
Karsten D. WolfFaculty educational science
Didactical design ofinteractive learning environments
2
cc by Karsten D. Wolf 2007
www.everlearn.eu
User Generated ContentDidactical PrototypingCollaborative Learning
3
4
© Karsten D. Wolf 2007
roadmap
1. Why educational research is hard
2. How can we produce evidence in educational research?
3. Multimethod approaches in educational research
4. Alternative methods: Intra-individual analysis of micro process timeseries
5. Educational technology as a research frontier in learning science
6. Some applied results
5
cc by Karsten D. Wolf 2007
soft sciences are hard
6
cc by Karsten D. Wolf 2007
7
cc by Karsten D. Wolf 2007
btw: this is not a real sim.rocks do not decelerate before hitting the ground!
8
cc by Karsten D. Wolf 2007
what if rocks would behave?
9
cc by Karsten D. Wolf 2007
10
cc by Karsten D. Wolf 2007
covariates hell
pic by Don Gato on Flickr11
cc by Karsten D. Wolf 2007
3 strategies
control themlive with themr&d approach
Fischer/Waibel/Wecker (2006): Nutzenorientierte Grundlagenforschung im BildungsbereichArgumente einer internationalen Diskussion
12
cc by Karsten D. Wolf 2007
experimental researchdesign based research
engineering approaches
13
cc by Karsten D. Wolf 2007
14
cc by Karsten D. Wolf 2007
75%15
cc by Karsten D. Wolf 2007
Educational Researcher, Vol. 33, No. 2, 3–11
Educational Researcher, Vol. 32, No. 9, 3–14
16
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de
Simpson
paradox
17
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de18
cc by Karsten D. Wolf 2007
Problems with quasi-experiments
Steyer 2007 - www.causal-effects.de19
© Karsten D. Wolf 2007
Random Assignment & Propensity Score Matching
• Random Assignment
• is the population large enough?
• can the assignment truly random?
• doesn‘t it feel like a lottery?
• Propensity Score Matching
• need to collect covariates
• matching at an individual level between treatment and control
20
© Karsten D. Wolf 2007
• self-reports (behaviors, psychological contructs, forms of behaviors)
• informant assessment
• ecological momentary assessment
• web based / computer based methods
• ability tests
• implicit methods (performance measures)
• sequential observational methods
Multimethod approaches: assessment methods
21
© Karsten D. Wolf 2007
Multimethod approaches:assessment methods
• quantitative text analysis
• multilevel analysis: physiological & biochemical measures
• brain imaging
• experimental methods
• nonreactive methods
22
© Karsten D. Wolf 2007
Experiments vs. Assessments
Why is Explanandum?
If (A1 and A2) then Explanandum (general law, explanans)
A1 (antecedent condition 1)
A2 (antecedent condition 2)
Therefore: E (explanandum)
Experiments aim to develop and test general lawsAssessments aim to show that the antecedent conditions are met
Erdfelder & Musch 2006
23
© Karsten D. Wolf 2007
Level of nonreactivity
Low High
Type 0 Type 1 Type2 Type 3 Type 4 Type 5
Setting initially designed or selected for research yes yes yes yes yes no
Participants are aware of the research setting likely likely likely likely no no
Participants are aware of the research question likely likely likely no no no
Participants are aware of the research hypothesis likely likely no no no no
Participants are aware of the measures‘ manipulability likely no no no no no
Examples Participative expert interview
Bogus pipeline technique
Personality questionaire
Cover story experiments
Lost letter technique
Analysis of archival data
Fritsche and Linneweber 2006
24
© Karsten D. Wolf 2007
Methods for analyzing multimethod data
• Multitrait-Multimethod Matrix (Campbell and Fiske 1959)
• categorical variables analysis for asessing multimethod association
• multimethod item response theory
• multilevel analysis
• structural equation models
• longitudinal methods
25
cc by Karsten D. Wolf 2007
26
27
28
29
30
31
32
I feel taken seriously (EG1 TN1 SoLe III)
33
1 2 3 4 5 6 7 8 9 10 11 12 13 14
34
Taken seriously Cognitive Load Participation
I feel good I understand I am interested
35
I am interested for19 students in EG1 SoLe III
36
I feel taken seriously (EG1 TN1 SoLe III)
37
I feel taken seriously (EG1 TN1 SoLe III)Linear Trend
Trend
38
I feel taken seriously (EG1 TN1 SoLe III)3rd Polynom
Trend
39
I feel taken seriously (EG1 TN1 SoLe III)Spline λ=10.000
Trend
40
I feel taken seriously (EG1 TN1 SoLe III)Spline λ=100.000
Trend
41
I feel taken seriously (EG1 TN1 SoLe III)
Decomposition
Trend Microprocesses
42
ARMA-Analysis
43
AIC BIC R2 Φ
AR(1) 1201,8 1208,0 0,094 Φ1 = 0,307
AR(2) 1190,2 1200,9 0,139Φ1 = 0,238Φ2 = 0,223
AR(3) 1192,1 1206,5 0,139Φ1 = 0,237Φ2 = 0,222Φ3 = 0,007
AR(5) 1194,2 1215,8 0,145
Φ1 = 0,236Φ2 = 0,241Φ4 = -0,077Φ5 = -0,018Φ3 = 0,030
Model parameters
44
AR(2)-Model
45
Multivariate Analyis:Cross Correlations
FG
I
P
CL
t-3 t-2 t-1 t
I 0,1220,120
-0.143
0.146
46
Multivariate Models: GMDH
FG
I
P
CL
t-2 t-1 t
0,097
-0.15
0.13
47
“As the complexity of a system increases, our ability to make precise and significant statement about its behaviour diminishe until a threshold is reached … the closer one look at a real-world problem the fuzzier becomes its solution” (Kosko, 94)
Fuzzy Modelling (Zadeh)
48
Fuzzification1
0,5
0Negative_X ZERO_X Positive_X
49
IF
NOT_Participation (t-1) & TakenSeriously (t-2)
OR
NOT_Participation (t-1) & CognitiveLoad (t-2)
Fuzzy Rule InductionTakenSeriously (t-1) & NOT_Participation (t-2)
OR
NOT_Understand (t-1) & CognitiveLoad (t-2)
THEN FeelGood TN1 (prewhitened)
50
Better communication?
Different interrelationships at different levels? (Thresholds)
non-linear models?
basis for building hypotheses?
Advantages?
51
time consuming
lot of rules
no selection criteria
no „test of significance“
disadvantages
52
cc by Karsten D. Wolf 2007
Online learning as a research frontier?
53
cc by Karsten D. Wolf 2007
54
cc by Karsten D. Wolf 2007
55
cc by Karsten D. Wolf 2007
what is happening here?
how to measure this?
56
cc by Karsten D. Wolf 2007
???
57
cc by Karsten D. Wolf 2007
???Learning accompanying actions
BrowseSearchWrite…
58
cc by Karsten D. Wolf 2007
???creation
59
cc by Karsten D. Wolf 2007
???
construction
60
cc by Karsten D. Wolf 2007
???
communication
61
cc by Karsten D. Wolf 2007
???
cooperation
62
cc by Karsten D. Wolf 2007
???
collaboration
63
© Prof. Dr. Karsten D. Wolf 2007
Some results from a prior online seminar
ZfE Cover
64
© Prof. Dr. Karsten D. Wolf 2007
Activity
# chat msg# forum entries
# created words
femalen=23
46.2(57.92)
7.0(7.44)
2301(363.5)
malen=13
24.7(23.43)
3.0(3.08)
1919(419.0)
1-tail significant test, n=36
65
© Prof. Dr. Karsten D. Wolf 2007
Activity and Post-Interest
# chat msg# forum entries
# created words
Post-interest
.11(.258)
.19(.135)
.34*(.022)
1-tail significant test, n=36
66
© Prof. Dr. Karsten D. Wolf 2007
Pre-Interest and Activity
# chat msg# forum entries
# created words
Pre-interest.30*
(.043).28
(.053).36*
(.019)
1-tail significant test, n=36
67
© Prof. Dr. Karsten D. Wolf 2007
Activity and Post-Interest
# chat msg# forum entries
# created words
Post-interest
.11(.258)
.19(.135)
.34*(.022)
1-tail significant test, n=36
factors out by pre-interest!
68
© Prof. Dr. Karsten D. Wolf 2007
Activity and Social Relatedness
# chat msg# forum entries
# created words
socialrelatedness
-.08(.319)
.30*(.036)
.31*(.035)
1-tail significant test, n=36
69
© Prof. Dr. Karsten D. Wolf 2007
Pre-Knowledge and Activity
# chat msg# forum entries
# created words
pre-problemsolving
.34*(.021)
.47**(.002)
.54***(<.001)
1-tail significant test, n=36
70
© Prof. Dr. Karsten D. Wolf 2007
Activity and Knowledge
# chat msg# forum entries
# created words
post-problemsolving
.03(.425)
.28*(.048)
.43**(.005)
1-tail significant test, n=36
71
© Karsten D. Wolf 2007
Which means: constructivism is right after all!
72
© P
rof.
Dr. K
arst
en D
. Wol
f 200
5
http://www.karsten-d-wolf.de/
73