Faculty Evaluation Systems:Student Evaluations of Faculty

What are we measuring?

“The evaluation of teachers is a mark of a good college.”

Ernest Boyer, 1987, p.155

Introduction:Community College Faculty Evaluation • Community College –Teaching Mission

– Diverse mission of a comprehensive community college

– As a Community College Teaching is at the Center of our Purpose.

– RSCC a Teaching Institution – First & Foremost

– Can we Measure (Evaluate) Teaching Excellence?

Defining Teaching Excellence: Three Major Emphases

• 1) Input– Student, teacher and course characteristics

• 2) Process– Classroom atmosphere, teacher behavior,

student learning activities, course organization, & evaluation procedures

• 3) Product– End-of-course learning, affective/cognitive

change, skills acquisition– Long-term learning, affective/cognitive change,

skills acquisition

Presentation Outline

• Part 1: Faculty Evaluation Systems (Purpose)

• Part 2: Faculty Evaluation System: Student Evaluations of Faculty (Student Ratings - One Piece)

Part 1: Faculty Evaluation- What’s the purpose? Overall institutional purpose is key toproper evaluation. • Formative & Summative

– Improve Pedagogical Methods & Student Learning

– Faculty, Course & Curriculum Development

– Excellence in Teaching– Tenure & Promotion

Faculty Evaluation Purpose: Accountability Renewed attention to undergraduateinstruction…

– State Legislature, Coordinating and Governing Boards

– Accrediting Bodies – SACS

• "The institution regularly evaluates the effectiveness of each faculty member in accord with published criteria, regardless of contractual or tenured status."

• "Standards for all educational programs include all on-campus, off-campus, and distance learning programs."

– Source: The new SACS Comprehensive Standards governing Educational Programs.

Evaluation of College Faculty• First student ratings took place in the

late 1800s (Sioux City). • Last 2 decades, evaluation of college

faculty has increasingly become a way of life for those in higher education.

• Today, virtually all American colleges assess faculty performance using student evaluations along with other instruments.

Faculty Evaluation Systems

• Comprised of multiple measures– Student Evaluations of Faculty– Peer Evaluations– Self Evaluation– Supervisor Evaluation– Exit Interviews & Alumni Surveys– Teaching Portfolios/Dossier– Classroom observation

Part 2: Student Evaluation of Faculty (Ratings)• Most widely used structured method of

evaluating faculty in higher education• Integral to most systems is the

“instrument” by means of which student opinions & observations are collected.

• Student appraisal of instruction seems appropriate because, students are the only ones who observe teaching for an entire course.

Student Evaluations of Faculty (Ratings)• Do these ratings accurately portray

teaching effectiveness?

Student Evaluations of FacultyQuality of Ratings? Legitimacy? • Higher education tends to question

the quality and legitimacy of the info. collected through student ratings of faculty ….

Concerns about Evaluations (student ratings)

No surprise a significant # of studies focused on objectivity and validity of such evaluations –2,000 + published studies

• Concerns include: – Low validity & reliability– Appropriateness of items (instrument)– Correlation between grades & evaluation – Measure “How good an entertainer rather

than how good his/her teaching skills are”– Popularity contests that reward classroom

entertainers & easy graders – Students don’t take these seriously

Concerns Continued…

• Student opinion can be affected by variables beyond the faculty member’s control..– Class size– Course content

Student Evaluations of Faculty – Grades & Ratings• Research findings support the

notion that instructors CANNOT purchase FAVORABLE student ratings through easy grading.

• Research also indicates the relationship between grades & ratings is a function of the better achieving students’ greater interest and motivation.

Despite reservations…

• More than half of the higher education institutions in the nation continue to use student appraisals of faculty within their faculty evaluation systems.

• Student perceptions of end-of-course evaluations indicate that do take these seriously, but they need to understand how the results are used…

The Quality of Evaluative Information • Reliability – Free from error (Computing &

Measurement- accurate and consistent results)

• Validity – Meaning (Measuring what you think you are?)

• Generalizability – Representativeness (Whose opinions are reflected by data?)

(Does sample of info. portray the totality of the person’s teaching?)

• Utility (What purposes can the data serve?)

The Instrument: Asking the right questions• Definition of teaching quality (input,

process & product)• Purpose of evaluation• Usefulness of evaluative info.:

Information should be detailed, diagnostic, and focused on specific teaching behaviors and course characteristics to derive maximum benefit of information.

The Instrument: Evaluative Questions • Two types of evaluative questions found

in most student evaluation instruments – 1) Student reaction to instructor traits or

behaviors; characteristics of course material; and social and physical environment

– 2) Student outcomes (Progress toward general or specific educational goals)

– 3) You might find a third type: Demographics

The Instrument: Format of Questions • Whether outcomes or trait

questions… – Multiple Choice/Scaled Responses

• (Best # of points for a particular scale –empirical question)

• Some better than others– Open-ended

• Augment multiple choice items • Helpful & constructive recommendations

for improvement

Areas of Measurement: Grouping the Questions • Global: require high inference (making judgments)

– Instructor– Course– General (instructional climate)

• Specific Items: (descriptive & diagnostic)

– Course management– Instructor characteristics & style– Reading material/assignments– Exams– Student behavior (effort, involvement, etc.)– Student outcomes of instruction

• Open-ended:

High Inference Questions vs. Specific Questions• Specific multiple choice/open-ended

probing questions• Diagnostic information• Most useful for course/instructor improvement

• Very general or high inference rating scale items

• How would you rate this instructor’s overall teaching ability?

– Little helpful diagnostic information – Most useful for administrative decisions,

advisement of students…

RSCC Faculty Evaluation Instrument - completed by students

• Examples:• Question #9: The Instructor’s examinations and/or

assignments reflected the content and emphasis of the course.

• Question #10: The instructor returned examinations and/or assignments within a reasonable time and with appropriate explanations.

• Interpretation? – Self or course improvement? – Exam or assignment improvement?

• Question #9 reworded: The instructor’s examinations reflected the course content.

Flexibility of Instrument

• Many institutions tried a single institutional instrument – Criticism from faculty and students as

irrelevant to the specific needs of particular programs and/or courses

– Solutions: • Leave room on the questionnaire for the instructor

to add his/her own items• Module approach – different instruments for

different programs/courses (labs, distance learning courses, clinicals, small or large lecture etc.)

Flexibility of Instrument

• Does one size fit all? Different instruments for different types

of courses?– Lab– Clinical– On-line– IDEA Room– Small lecture or large lecture– Seminar– Other

Utility of Assessment Information depends on…

• Quality of instrument• Reporting • Relies on faculty perceptions of

evaluation process –quality issue• Student perceptions of the

evaluation process

Summary

• In sum, then, provided that data are gathered carefully, reported appropriately, and interpreted judiciously, student evaluations appear to make a useful contribution to course improvement, faculty/self development, personnel decisions, and, possibly to student learning.

Remember

• Research tells us that Students’ early evaluations are quite stable and instructors should be aware that the first few days of class may be very important in determining the eventual image students evaluate at the end of the course.

Questions?

Can we measure teaching excellence?

Future direction?(Planning/Implementation Time)