Embed Size (px)
Citation preview
Multilevel Modelling of Learning Gains: The Impact of Module
Particulars on Students’ Learning in Higher Education.
Jekaterina Rogaten
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise Whitelock. Please contract [email protected] if you want to know more about ABC learning gains
Research Team
Dr Bart RientiesDr Jekaterina Rogaten
Prof Denise Whitelock
Dr Simon Cross
Prof Allison Littlejohn Dr Ian ScottProf Rhona Sharpe
Dr Simon Lygo-Baker
Prof Steven WarburtonProf Ian Kinchin
Learning gains
We define Learning Gains as:
Growth or change in knowledge, skills, and abilities over time that can be linked to the desired learning outcomes or learning goals of the course
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (Submitted: 07-07-2016). Reviewing the concept of learning gains in higher education: an affective, behaviour and cognitive perspective. Educational Research Review. Impact factor: 3.860
ABC project to measure learning gains?Affect
Behaviour
Cognition
Academic
performance
VLE
Satisfaction
How are learning gains measured: a meta-analysis
• The concept of learning gain is primarily used to examine the effect of any particular educational ‘intervention’
• There is a gradual increase in studies examining learning gains all across the world
• All learning gains can be classified into ABC
53%
16% 21%
10% Behaviour-Cognitive
Learning Gains
Affective-Cognitive
Learning Gains
Affective-Behaviour-Cognitive
Learning Gains
Cognitive Learning Gains
Rogaten, J., Rienties, B, Cross, S., Whitelock, D., Sharpe, R., Lygo-Baker, S., Littlejohn, A. (Submitted: 07-07-2016). Reviewing the concept of learning gains in higher education: an affective, behaviour and cognitive perspective. Educational Research Review. Impact factor: 3.860
Do students make learning gains, and what is the power of LA to predict learning gains?
• Learning gains are usually measured using pre-post standardised testing
• It is resource intensive and becomes even more so if one wants to estimate learning gains across various disciplines and number of universities
• Using assessment results for estimating learning gains has number of advantages:• Assessment data readily available
• Widely recognized as appropriate measure of learning
• Relatively free from self-reported biases
• Allows a direct comparison of research finding with the results of other studies
Rogaten, J., Rienties, B, Whitelock, D. (2016). Assessing learning gains, TEA Conference, Tallinn, Estonia
Modelling learning gains
• Pre-tests and post-tests are a standard method
• Raw gain
• True gain
• The normalised gain
• Normalised change
• ANCOVA on pre-test and post-test scores
• ANOVA on raw gain scores.
• ANOVA on residuals
• Repeated measures ANOVA
Three-level Growth Curve Model
Level 1
Level 2
Level 3
TMA1
Student1
TMA3 TMA1 TMA2 TMA3 TMA1 TMA2 TMA3TMA2
Student2 Student3
Module1 Module2
TMA1 TMA2 TMA3
Student4
TMA1 TMA2 TMA3
Student5
Module3
Rogaten, J., Rienties, B, Whitelock, D. (2016). Assessing learning gains, TEA Conference, Tallinn, Estonia
Current Study
• Participants• STEM - 10,038students
• 74.4% males and 25.6% females
• Business and Law - 11,154 students
• 45.3% males and 54.7% females
• Academic performance for each Tutor Marked Assessment (TMA)• TMA for 200 modules was retrieved from our university database for 2012/13, 2013/14
and 2014/15 academic years.
Math and Computing
(M=71.6; SD = 22.2)
Math and Computing
Business and Law
(M=64; SD = 15.2)
Business and Law
Basic ModelSTEM Business and Law
Variance in students’ initial
achievements46% 16%
variance in students’ learning
gains77% 51%
Module-level intercept-slope
correlation-.69** -0.43**
Student-level intercept-slope
correlation-.15* -.1
VPC module 28.3% 9.6%
VPC student 33.8% 50.4%
VPC TMAs 37.9% 39.9%
Students in courses with the low initial achievements show higher learning gains than students in courses with the high initial achievements
Students with low initial achievement show higher learning gains.
Key messages
• The module that any one student student is enrolled in accounts for a substantial portion of variance, not just in the student’s initial academic achievements, but also in the learning gains within the module.
• Multilevel modelling is a more accurate method compared with simple linear models to estimate students’ learning gains because simple models are not able to detect differences between modules when looking at the faculty level performance
• Low learning gains - or negative learning gains of high achieving students -does not imply that students are losing knowledge, however it highlights the complexity of factors that have to be taken into account when using students’ academic performance as a proxy for learning gains.
Multilevel Modelling of Learning Gains: The Impact of Module
Particulars on Students’ Learning in Higher Education.
Jekaterina Rogaten
https://twitter.com/LearningGains
https://abclearninggains.com/
The results of the ABC project were made possible due to Jekaterina Rogaten, Bart Rienties, Simon Cross, Ceri Hitching, Ian Kinchin, Simon Lygo-Baker, Allison Littlejohn, Ian Scott, Rhona Sharpe, Steve Warburton, and Denise Whitelock. Please contract [email protected] if you want to know more about ABC learning gains
