Online Course Evaluations: Nonresponse and ClassEval in Fall 2009

COURSE EVALUATIONSPaper versus Online

Ratings

Can they?

Why?

How?

METHODS OF ANALYSIS

• Study measured likelihood of response• Hierarchical modeling techniques• 30 variables

– Students (demographics, housing, number of surveys students completed)

– Course (if department of course and student’s major matched; grade earned)

RESULTS

Statist ical ly significant characteristics associated with nonrepsonse:

– Gender (Male)– Ethnicity (African-American,

Asian)– Housing (off-

campus/commuters)– Athletes

– Grades (Ds, Fs, ungraded students/courses)

– Age (traditional age)– Nontransfers– Class rank (sophomores &

juniors)– Students with more than

10 SETs to complete

48% response rate(overall)

RESULTS (continued)

• Using Holland’s six major academic types– Realistic majors were more likely to respond than social,

artistic, conventional, enterprising, and investigative disciplines.

– Social majors were less likely to respond than all other students if the course was in the same department as the student’s major.

• Most variables were no longer statistically significant when the course was in the same department as their major.

DISCUSSION

• Mostly aligned with previous research and theories of survey participation*– Exceptions = Environment of major/course, class rank

• Survey Fatigue – Are we oversurveying?• Introduced new potential influences on participation

– Campus housing, athletes, transfer status,

* Avery et al., 2006; Cohen, 1981; Clarksberg, et al., 2008; Dey, 1997; Dillman et al., 2002, 2009; Fidelman, 2007; Groves et al., 2004, 2009; Johnson et al., 2002; Jones, 2009; Kaplowitz et al., 2004; Lepkowski & Couper, 2002; Marsh, 2007; Moore & Tarnai, 2002; Porter et al., 2004; Porter & Umbach, 2006a; Porter & Whitcomb, 2005; Sax et al., 2003, 2008

TO INCREASE RESPONSE RATES

• Faculty attitudes and efforts are key to obtaining higher response rates*

• The instrument and administration of it• Target students unlikely to respond• Evaluate the environment

– Are course evaluations online accessible?– Is the instructor encouraging response? Is the discipline?

* (Ballantyne, 2003; Dillman, 1978; Dillman et al., 2002, 2009; Dommeyer et al., 2004; Groves et al., 1992)

ADDITIONAL RECOMMENDATIONS

• Obtain more information about the course, the faculty, class size, etc.

FOR MORE INFORMATION

meredith_adams@ncsu.edu

Full study available online at http://repository.lib.ncsu.edu/ir/handle/1840.16/6364