Main components of effective teaching

SRI validityA measuring instrument is valid if it measures what it is supposed to measure

SRIs measure:Student opinions about/perceptions of effective teaching, their satisfaction with instructionCIOS: Course/Instructor Opinion Survey

SRIs are valid because they do measure student satisfaction with instruction

However, in my talk I’ve shown that:Student satisfaction with instruction (ratings on the Overall Teaching item) does correlate in a moderate to high extent with: 1. Student learning2. The conceptual structure/main general behaviors of

effective teaching3. Other measures of effective teaching (i.e. by alumni,

peers, experts, trained observers, and by self ratings)And:4. There are almost no proven factors that bias SRI validity

Factors controllable by the teacher, found not to bias validity

1. Expressiveness/enthusiasm2. Course (perceived) difficulty/overload3. Course grades

Illustration: no relationships between rating of the items: course difficulty and Overall Teaching

TAU: All University undergraduate lecture courses (n = 1,770),

Fall Semester 2007, r =-.02

Risks with factors controllable by the teacher:

Faculty may manipulate these factors hoping to get higher ratings—by entertaining students, lowering course demands, grade inflation and altogether, watering down the course…

There is evidence that this already happens…

Factors uncontrollable by the teacher, found not to bias validity

1. Class size (smaller classes are rated higher)2. Discipline (“soft” disciplines are rated higher)3. Student GPA (higher GPA tend to be rated higher) 4. Student motivation (elective courses tend to be rated

higher)5. Student level of studies (graduate courses tend to be

rated higher) 6. A variety of extraneous factors

Extraneous factors (should not affect teaching or student learning) that were studied (to discredit SRIs)

Faculty: academic rank, age, gender, years of teaching experience, personal characteristics (other than enthusiasm or caring), physical attractiveness, with Asian accent, research productivity

Students: age, gender, personality

Course: the time of day it is offered, length of class meetings, number of rows in the classroom

Conclusion (Marsh, 2007): SRIs are valid

SRIs correlate only minimally with extraneous variables but correlate reasonably with effective teaching behaviors and with student learning

SRIs are relatively valid against a variety of indicators of effective teaching and relatively unaffected by a variety of variables hypothesized as potential biases

SRI reliabilityReliability is how accurately an instrument measures whatever it measures

SRI reliability refers to the stability and consistency of the measurement data

Consistency: Repeated measurements should give almost the same result. To illustrate with measurement of same issue in 2 semesters:

Stability: Measurement of same issue in different times and by different students provides stable/consistent results: Illustration with teacher ratings profile

Conclusion by Marsh & Baily (1993)Instructors appear to have distinct profiles of strengths and weaknesses that are highly consistent across sets of ratings obtained for the same instructor over a 13-year period and across undergraduate- and graduate-level courses.

Conclusion: SRIs are valid and reliable to a large extent