Successful Online Implementation

Depends on a Culture of Trust

Bethel University College of Arts and Sciences

Dr. Richard SherryDean of Faculty Growth and Assessment

IDEA Use at Bethel, 2006-07

Type Sections Percentage

Diagnostic, Paper 39 4.5%

Short form, Paper 117 13.5%

Diagnostic, Online 272 31.4%

Short form, Online 437 50.5%

Total 865

Shift to Online EvaluationIncreasing Online Participation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

fall 2005 spring2006 fall2006 spring 2007

DiagP ShortP

DiagO ShortO

Normal Use of IDEA

• Context of faculty evaluation– History of use at Bethel

• General policies for use– Known policies that allow flexibility– General confidentiality– Use in a formative context– Evaluation and multiple voices– Acknowledgment of value and consequences

Who speaks on teaching?

Annual Reappointment• Faculty self-

assessment• Department chair

observation, review• Course evaluations• Additional

observations if desired

Promotion/Tenure• Extended self-

assessment• Department chair• Course evaluations• 3 peers, including

teaching• 6 alumni• Additional if desired

Faculty Development and IDEA

• New Faculty Orientation*• Guidance to faculty on interpretation and

application• Approaches by personnel committees to

interpretation* CRI • Follow up with Chair, Faculty Development

Coordinator, or Dean• Direction for specific sessions on

pedagogy, curricular discussions

Committees Reading IDEA

• Understand the scale• Take the long view (trends, types of

classes)• Understand the context (kind of course)• Consider response rates and distribution• Look for key indicators• Consider the objectives• Read the comments FDI

Faculty Concerns about Ratings

• Earlier skeptical research repeatedly cited (google: misconceptions "student course evaluations")

• IDEA’s approach and “comparison”• Student ratings of instruction and the

“amenity” culture 9• “One bad course”• Time invested• Ceiling effect• Exploring a new course and “grace”

NO OTHER PROFESSOR uses the so called "university wide grading scale". A’s are generally accepted to be a good grade, and C’s are not good. No matter what you might think. This is a completely ridiculous grading system, as it makes the GPA go down in relation to other classes that don’t use this same scale.. . . Everyone else I know in other college writing classes is getting an A. I’m not saying that everyone deserves an easy A, but I know that I, and many other students work really hard to improve paper after paper, and it just doesn’t happen. FCR

Pilot of IDEA Online and Growing Acceptance

• The context for IDEA Online: reasonably robust technology

• Recruiting pilot cases

• Reasonable lead time

• Massive follow-up

• Successful reporting

Engagement with Faculty on Concerns

• Response rates

• Ratings differences

• “Ganging up” and “unsupervised” evaluations

Continuing to respond to technology and faculty

• Improvements in our processes• Customizing group reports• Customizing central questions

– How has this course helped you connect this area of study to a Christian perspective?

– How has this course contributed to your development as a thinking Christian?

– How do you feel this course is influencing your life as a believer?

Anatomy of a Failure

Implementation at College of Adult and Professional Studies/ Graduate School

• Neglect of the context

• Inattention to politics

• Failure to prepare students and faculty and communicate well

Reminders

• Communicate policies clearly

• Be appropriately flexible about evaluation

• Clarify “voice” and “weight” of ratings

• Create formative context

• Provide guidance on implementation and interpretation

• Provide guidance on committee use

Reminders (2)

• Communicate to students on value of ratings

• Manage response rates through monitoring technology

• Prepare students for the forms

• Reassure supervisory staff