Affective behaviour cognition learning gains project presentation

Affective-Behaviour-Cognition (ABC) Learning Gains

ProjectOpen University 15-02-2016

https://twitter.com/LearningGains


ABC approach

Project management

• Project team:• Three universities bring together their expertise to the ABC project

• Project management: • Project team meets monthly over skype• Everyone has access to shared folders• Within universities meetings are held every week • 2nd face-to-face meeting of all partners (Sep)

Open University Team• Strength• Learning analytics• Quantitative data analysis• Assessment• Online learning and learning design• Cross-sector thinking and working

• Responsibilities• Overall leading the project• Leading the Phase 1 of the project

(secondary data analysis)• Phase 1 ethics applications• Developing models of learning gains

Dr Bart RientiesDr Jekaterina Rogaten

Prof Denise Whitelock

Dr Simon CrossProf Allison Littlejohn

Open University1. Learning AnalyticsLargest VLE datasets of affect, behaviour and cognition of 200K students, with 2 large scale implementations of predictive modelling

2. Learning DesignExtensive modelling of learning designs (OULDI)

3. Student Experience on a ModuleExtensive datasets of students’ experiences with blended and online learning

Oxford Brooks University Team

Prof Rhona SharpeDr Ian Scott

StrengthExpertise in learners’ experienceMixed-methods approachMeasures of learning and engagementLearning analytics

ResponsibilitiesCo-leading Phase 2 of the projectIn depth-interviews and diary data collectionTesting validity of self report measures of learning gainsTriangulation

Oxford Brookes University1. Academic Performance Tracking Tool Presents staff with data from student information system, NSS and module evaluations in form of dashboards used for module and programme reviews.

2. Grade Point AverageProvides students with information about their progress. Provides staff with a more nuanced indicator of attainment.

3. Survey of Student EngagementDeveloped own scales for monitoring student engagement with Assessment Compact, Academic Advising and Graduate Attributes.

University of Surrey Team

ResponsibilitiesCo-leading Phase 2 of the projectTesting the validity of the self-reported measures of learning gains (+OB)Testing the validity of learning analytics modelling (+OU)Dissemination: website and Twitter

Prof Ian Kinchin Prof Steven Warburton Dr Simon Lygo-Baker

StrengthUniversity pedagogy and concept mapping Educational technologiesExpertise in innovative teaching approaches development

University of Surrey

• Business Intelligence - Progression Analysis project has built / is building:• Live data on students who are at risk, to act as an early warning indicator so that active support

measures can be put in place;• Visualisation of data via dashboards for academics to put in place appropriate actions;• Extension into WP & Outreach domain with specific data feeds, modelling, and a WP progression

dashboard.

• Institutional success: • Ability to makes links between what gains our students make in correlation with an institution which

has high levels of satisfaction and has enhanced learning and teaching across all Faculty.

WP5 – social media presence: Wordpress blog and Twitter feed

Learning gain literature review – in progress

1) Self-reported measures of learning gain 2) Average Normalized gain (post-pre)/(Nq-pre) 3) Percentage change (Post-Pre)/Pre4) Raw change (Post-Pre)

• Literature review shows that the concept of learning gain primarily used to examine the effect of any particular educational ‘intervention’

• Web of Science core collection • The time frame of the search 1995 - 2015. • The key word search included: “learning gain*”, higher education, college, graduate, not school, not child. • In total 188 studies were identified. • Review criteria:

1) Does the study assesses learning gain? 2) What learning gain was examined i.e., behavioural, affective, cognitive? 3) How was it measured?

Triangulation

Quantitativ

e

Academic performance

VLESatisfaction

Interviews

Diary

Module design

Structure of the ABC project Socio-Demographic

Timeline

Current progress• Phase 1• Ethics approval

• Ethics are now obtained for the Phase 1 at the OU

• Secondary data analysis • Collected (OU)

• Demographics data• Academic performance data• Module information data

• In progress (OU)• VLE data (e.g., discussion

participation, number of visits, length of visits)

• Students satisfaction data

• Secondary data analysis• Data cleaning and descriptive

statistics

Current progress: Example Arts AA100• The arts past and present

• 30 week, 60 credit, Level 1 module

• Introduces to university-level study in the arts across a range of subject areas - art history, classical studies, English, history, philosophy, music and religious studies.• Assignment 01 (10%)

• Assignment 02 (10%);





• Assignment 07 (10%)

• 25% - BA (Honours) History, 19% - BA (Honours) Humanities, 16.2% - BA (Honours) English Language and Literature, 12.5% - BA (Honours) English Literature

• Design: 75% Assimilative, 20% Assessment, 4% Productive

Participants

3262 students from years 2013/14 and 2014/15

2147 Females (age M=33.2, SD=12.8) & 1115 Males (age M=35.5, SD=14.2)

Motivational orientation:

36.8% - employment and personal development, 13.6% - employment, 23.9% - personal development

Occupational status:

43.9% in full-time employment, 24.2% part-time employment, 13.4% not in paid work, 10.9% unemployed

93.2% White; 19% reported study disability

50% have A level or equivalent, 30.9% have below A level, 14.3% have HE qualification

81.4% were only taking AA100 module

Current progress: Example Arts AA100



Next step: Multilevel Modelling

Level 1

Level 2

Level 4

Level 3

ABC1

Student1

ABC3 ABC1 ABC2 ABC3 ABC1 ABC2 ABC3ABC2

Student2 Student3

Course1 Course2

University1 University2

ABC1 ABC2 ABC3

Student4

ABC1 ABC2 ABC3

Student5

Course3

Dissemination

• Inside government - Measuring Learning Gain in Higher Education (May)• Presentation on ABC project

• EARLI conference (August)• Presentation of the 3-level model for OU students sample

• SRHE conference (December)• Presentation of the 4-level model for OU+OB+US students samples

• Stakeholder dissemination event (Sep) • [BART TO ADD DETAILS]

• Literature review (April)• Meta-analysis to be submitted to Educational Research Review or Review of Educational

Research.

Affective-Behaviour-Cognition (ABC) Learning Gains

ProjectOpen University 15-02-2016

