Stefani Dawn Assistant Director of Assessment Office of Academic
Program, Assessment and Accreditation eSET Slide 2 Decisions for
Faculty Senate HIGHEST PRIORITY DECISIONS Minimum class size for
evaluations Combined or uncombined reporting eSET-RELATED ITEMS and
PROCESSES Demographic data collection Courses to include or exclude
in eSET Financial, resource and technological limitations of eSET
to be configured for other evaluation approaches (formative
evaluations or department-centric evaluations) Slide 3 Minimum
Class Size for Evaluations OAR 576-003-0070 If Oregon State
University solicits or accepts student survey evaluations of the
classroom or laboratory performance of a faculty member, the survey
evaluations shall be conducted anonymously History Requests were
made to have separate reports for slash courses. That was
implemented last term and quickly put on hold when it was
discovered that some of the cross listed sections had only a few
students in it, even if all of the sections combined was a large
number. Took the issue to the Faculty Senate EC to make some
interim decisions to ensure that the OARs are upheld. Slide 4
Minimum Class Size for Evaluations Interim Decision Courses with
fewer than 6 students will not be evaluated. OSU guidelines
University of Oregon does not conduct evaluations in courses with 5
or fewer students. Question What should the number be to protect
student anonymity? Factors to Consider Student input Slide 5
Reporting: Combined or Uncombined There are advantages and
disadvantages to combining reports. Combining reports preserves
data Example: 2 section slash course one has 5 students, one has 4
students combining would allow the faculty members to receive the
evaluations. Some faculty and departments prefer the combined data
others do not Differences in P & T forms/data per department
Graduate versus undergraduate Having separate reports and combining
data after the fact is not straight forward Median calculation is
not a standard approach and cannot be done easily in an Excel
spreadsheet by the department or individual instructor The Office
of APAA and Enterprise Computing does not have the resources to
combine reports that have been separated. Slide 6 The Median
Calculation Selected by the Faculty Senate in 2004 Based on
"Measurement and Evaluation in Psychology and Education", Fourth
Edition, by Robert L. Thorndike and Elizabeth P. Hagen. This
calculation assumes that multiple values within a range are
distributed evenly over the range. Say you have the following
distribution: Score Freq Cumulative Frequency 6 29 106 5 29 77 4 38
48 3 10 10 The total responses is 106, so half of that is 53. 53
falls between the cumulative frequency of 48 and 77 corresponding
to the scores 4 and 5. 53-48 = 5 more cases in the 5 range are
needed. Since there are 29 responses for 5, we need 5/29 of them
or.20 (rounded from.17). We look at the range 4.5 5.5 and add.2 to
4.5 to get 4.7 as the median. Slide 7 Demographic Data Previous
demographic data collected: Students are still potentially
identifiable by: Gender class status (e.g. the only graduate
student in an undergraduate course) Grade: Audit Gender: The reason
you are enrolled in this course: Grade you expect to receive in
this course: Class Status: Is this course your major? Percent of
this class you attended: Your overall grade point average: Slide 8
Demographic Data Factors to Consider Do faculty use the data?
Survey fatigue by the students Slide 9 Everything is An If-Then If
you decide to combine reports: then how will that impact
departmental processes for P & T? If you decide to have
separate reports: Then what data will be lost? (Which is more
important the data or the separate reports?) Saving data and
accumulating it over time is not an option for two reasons OAR
576-003-0070 - Survey instruments from which evaluation data are
obtained shall be delivered to the faculty member. There are over
4000 courses and >1500 faculty evaluated per term. Tracking this
types of data is not feasible with the current FTE arrangements.
Then how would reports be recombined (e.g. a person has 7 different
slash versions of a course)? Could change the median calculation to
a standard calculation If you decide to keep the demographic data
Then how might it compromise anonymity? Is it information really
used and needed? Are there potential issues in the way questions
are asked and student response to those questions? Then how might
it impact survey fatigue?
