Upload
hoangnhi
View
217
Download
0
Embed Size (px)
Citation preview
% o
f stu
dent
s
0
20
40
60
80
100
1 18
Inspiration and early findings
only attend
only view online
neither
F
M
W
MM
M M
MM
M
W
WW
W WW
W
Lecture #
Inspiration for this project came from self-reported attendance and online viewing data from the 2011 iteration of CS50 (n=452). Something important changes over time that affects attendance on all time scales: from freshman to senior year (not shown), from the start to the end of the semester, and from Monday to Wednesday. By the end of this very popular course, 25% of students reported neither attending or watching lectures online. Lecture attendance was negatively correlated with online lecture viewing.
Weak extant literature
Undergraduate lecture attendance has not been comprehensively or rigorously measured. Past research is limited by one or more of the following:
• self reported attendance• self reported reasons for attending or missing lectures• biased sample of included courses• single course analyses
Undergraduate lecture attendance has not been systematically related to course variables, course ratings, or the availability or use of digital resources (e.g., lecture videos).
Methods• Four GoPro cameras installed visibly at front of four lecture halls
with existing video recording equipment
• Imagery data collected every minute (slowest time lapse possible with GoPros) from all courses that met in these lecture halls
• Encrypted data retrieved regularly and stored securely
• Machine learning algorithm developed iteratively based on emerging methods from computer vision
• Imagery data paired with specific course lectures based on manual reconciliation of Harvard-accessible (behind PIN) data from course syllabi, course sites, and published meeting times/locations
• One summary statistic derived from multiple estimates per lecture
• Researcher given administrative access to course sites, which included aggregate site and lecture video usage statistics
Development of algorithm
• The algorithm was developed and refined over seven months• No “out of box” solution existed• Small sample of images used initially to explore different strategies
for estimating attendance• Algorithm was “trained” to distinguish between empty seats and
seats with people in them (but not other things, such as coats or bags)
• Most accurate strategy took advantage of color profiles of empty seats versus those with people in them
• Iterations required to accommodate slight changes in camera angles• Discrepancies reconciled during week of Daylight Savings change,
when image time stamps diverged from actual times
Validation of algorithm
• Algorithmic estimates compared with random sample of manually coded images
• Correlation extremely high (r = .99)• Estimates plotted over time for each course, and potentially
aberrant estimates identified• Outliers scrutinized further based on input from faculty and
inspection of imagery data
Course A (n≈150)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140 T
examexam
Th
T
Th
TTh
T
Th
T
ThT
Th
T
Th
T
Th
T
Th ThTh
TT T
start endAt present time, attendance data from 10 courses have been analyzed fully and were presented at the HILT Conference 2014. Enrollment numbers and lecture dates were obscured for reasons of anonymity.
Course B (n≈200)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110T
Th T Th
Th
T
Th
Th
Th
Th
T Th
Th
Th
Th
Th
exam
T
T
TT
T
exam
T
optional
start end
Course C (n≈125)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110
120
130
exam
quizquiz quiz quiz
W
M
W
F
M
F
W
F
M
F
M W
F
F
M
W
F
F
MW
F
MF
M
W
F
MW
M
start end
Course D (n≈300)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110 M
W
F
M
W
F
M
W
F
W
F
M
F
MW
F
M
W
F
MW
F
M
F
M
W
F
M
W
F
M W
F
M
W
exam exam
start end
Course E (n≈250)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110exam
T
Th
Th
Th
Th
Th
T
T
T
TT
T
Th
T
T
T TT
Th Th
Th
Th
exam
T
start end
Course F (n≈300)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110exam exam
M
W
F
M
W
F
M
W
F
W
F
M W
F
WF
M
F
M
W
F
M
W
FW
F
MM
MW W
W
FF
start end
Course G (n≈150)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
guestspeakers
optionalmovie
T
Th
T
ThT
Th
T
Th
T
Th
T
Th
T
Th
T Th T
Th
T
Th
T
start end
Course H (n≈250)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110
120
exam examexamM
W
M
M
W
W
M
M
W
M
W
M
W
M
W
M
W
M
W
M
start end
Course I (n≈300)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
110
start end
exam exam
movie
movie
WM
MW
W
M W
WM
W
MW
W W
M
W
M
W
W
Course J (n≈100)A
tten
danc
e (%
)
0
10
20
30
40
50
60
70
80
90
100
exam
M
M
M
MM
M
M
M M
M M
MW
W
W
W W W
W W
W
W
start end
Courses A-J
Att
enda
nce
(%)
0
10
20
30
40
50
60
70
80
90
100
110
Course ACourse F
Course ECourse G
Course D
Course B
Course CCourse H
start end
Course J
Course I
Courses A-J
On the average, 60% of students attended any given lecture.
Moreover, there was incredible variability between courses in their average attendance: Some courses had as low as 38% attendance (over the semester), whereas others had as high as 94%.
Lecture attendance declined over the semester, starting at 79% and ending at 43%.
There was also loss of attendance over the week: 5% from Monday to Wednesday or Tuesday to Thursday, and 10% from Wednesday to Friday.
0%0%
0%0%0%
5%
10%4%
start end
1%9%
1%0%0%
85%
56%83%
attendance grade
premed requirement
Courses A-J
Att
enda
nce
(%)
0
10
20
30
40
50
60
70
80
90
100
110
0% 0%
Courses A-J
With this preliminary set of 10 courses, two noteworthy effects emerge.
First, courses that measured and graded attendance had higher attendance than those that did not (87% vs 49%, respectively). Second, courses in which students self-reported enrolling to fulfill a premed requirement had much higher attendance than other courses.
Other reasons for taking the courses (e.g., elective vs General Education requirement) did not show significant effects, nor did time of day, day of week, published Q ratings, or the availability of lecture videos.
Course I stands out as unusual: It had high attendance yet did not grade or measure attendance, nor did it fulfill a premed requirement. One distinguishing aspect of this course is that the instructor restricted access to the lecture videos until immediately before exams.
Machine learning algorithms can be used to accurately measure attendance in large lecture courses.
Lecture attendance declines over the week and over the semester.
There is a huge range in average lecture attendance across courses.
These differences appear mostly structural in nature.
Preliminary findings(10 courses)
For most courses, site use (# unique authenticated users per week) correlated positively with lecture attendance. The more students attended lectures, the more they visited the course site.
Analysis of lecture video usage is problematic—most courses make their videos available to anyone at Harvard or anyone on the internet. Nonetheless, preliminary results show a positive correlation for some courses and a negative correlation for others.
Site and video use
How do we interpret attendance in high-enrollment courses?
What evidence-based advice can we offer instructors who wish to promote attendance in high-enrollment courses?
How does the availability and use of online course videos (or other digital resources) affect attendance in high-enrollment courses?
Will the present findings replicate in a larger sample of high-enrollment courses? What other factors explain variability between courses in lecture attendance?
Open questions