Expert Guided Crowdsourced Learning Content - STEM 2014

!bit.ly/batestalks

Expert-guided crowd-sourced learning content:

a pilot study

Simon Bates Firas Moosvi @simonpbates @firasm

a"web&based"MCQ"repository"created"by"students"

Context

http://goo.gl/MHvDfH

Last year….

‘Seeking’ behaviour

Challenge: Can we enhance the pre-reading activity to engage more students more deeply with the material?

• Requirements:

• Outputs: distributed over the whole term

• Load: at least 2h of student time

• Assessment: Submitted online, marked using a rubric

• TA Load: No increased marking load

!

Pilot study in 3 sections of PHYS101 Jan-Apr 2014

PHYS101: Energy and Waves

http://goo.gl/OgP1Mt

PHYS101: Energy and Waves

http://goo.gl/OgP1Mt

PHYS101: Energy and Waves

http://goo.gl/OgP1Mt

PHYS101: Energy and Waves

http://goo.gl/OgP1Mt

PHYS101: Energy and Waves

http://goo.gl/OgP1Mt

Sample Learning Objects

Ted-Ed: A video tutorial and a series of questions

linked to time-stamps

Screencast: A slideshow with annotations and

voice narrations

Prezi: A visual slideshow format that can be used to break down problems

and many more…

Implementation - Logistics

• Cohort split into 4 groups, 2 cycles per semester

• Required to submit two (2) learning objects over the course of the term, each worth 2.5% of course grade.

• In addition to the two assigned weeks, students can submit additional LOs

• The two highest scoring learning objects will be used to determine themark.

• LO topic must be on pre-readings for the next week

Implementation - Submission & Survey

• Students submitted attachments or links to LOs online through the Blackboard LMS

• Upon submission, students complete a short survey:

• How did you come up with your LO topic?

• Time Spent

• How much did you understand the topic before/after LO creation

• Other feedback

physics.lo.open.ubc.ca

Learning Objects are showcased here:

Sample 1 - Were Astronauts really on the moon?

Sample 2 - Standing Wave in a bowl

Sample 2 - Standing Wave in a bowl

Sample 3 - Colour Loss Underwater

Sample 4 - “Bathysphere” (a submersible)

Results

2. Time spent on Learning Objects

1. Participation Data

3. Benefits for student learning

LO submissions were constant through the term

0 100 200 300

LO 1

LO 2

LO 3

LO 4

LO 5

LO 6

LO 7

LO 8

Submitted LOs

AssignedOptional

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

Grade (%)

Num

ber o

f stu

dent

sLO Grade Distribution

These students didn’t even try!

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

Grade (%)

Num

ber o

f stu

dent

sLO Grade Distribution

Which students aren’t submitting LOs?

Non-participants are from all walks of life!

0 10 20 30 40 50 60 70 80 90 1000

5

10

15

20

25

Final Exam (%)

Num

ber o

f stu

dent

sFinal Exam marks of students that did not participate in LOs

0 100 200 300 400

Less than 0.5h

0.5 to 1 h

1 to 2h

2 to 3h

3 to 4h

4 to 5h

More than 5h

Number of students

Time spent on Learning Objects

Most students tended to work on each LO for at least 2 hours

0 200 400 600 800

None

Little

Moderate

Good

Excellent

Number of students

0200400600800

Number of students

Student self-reported understanding of LO content before and after LO submission

0 200 400 600 800

None

Little

Moderate

Good

Excellent

Number of students

0200400600800

Number of students

Students report that they understand their content MUCH better after creating the LO

Student self-reported understanding of LO content before and after LO submission

0 200 400 600 800

None

Little

Moderate

Good

Excellent

Number of students

• Major Challenge: How to determine whether students that did an LO on a specific topic are actually primed to understand content better?

• [Reminder] Requirements of new activity:

✓ Output: distributed over the whole term

✓ Load: at least 2h of student time

✓ Assessment: Submitted online, marked with rubric

★ TA load: No increased marking load on existing TAs

Future Directions

★ TA load: No increased marking load on existing TAs

Option 2: Crowd-source LO assessment by assigning LOs to be marked by students

Only two choices:

Option 1: Redistribute resources to reallocate TA hours to marking and curating

0 100 200 300

LO 1

LO 2

LO 3

LO 4

LO 5

LO 6

LO 7

LO 8

Number of students

AssignedOptional

10 100 200 300 400

Less than 0.5h

0.5 to 1 h

1 to 2h

2 to 3h

3 to 4h

4 to 5h

More than 5h

Number of students2

0 200 400 600 800

None

Little

Moderate

Good

Excellent

Number of students

0200400600800

Number of students3

Participation Time Spent

Understanding