The FUS Faculty Evaluation System

IDEA WorkshopCleveland Ohio October, 2008

The Purposes of the The Purposes of the FUS Faculty Evaluation FUS Faculty Evaluation

SystemSystem

Provide an opportunity for self-reflection, self-direction, and practical reasoning in departments for both faculty and program improvement Provide feedback from Chair and VPAA for faculty developmentProvide evidence of distinction in performance areas – for merit award decisionsProvide comparative data for enquiry seeking standards of internal practices of FUS faculty

FUS a Catholic and FUS a Catholic and Franciscan University Franciscan University

is:is:

Faithfully Catholic and purposefully seeking to operate from the Heart of the ChurchComprehensive; Associates, Bachelors, and Masters liberal arts University located in Steubenville At about 2400 students and 40+ programs Has a structure with a strong VPAA and rotating Department Chairs as well as Graduate Program Directors who are teaching and researching while managing 2 Decade User of IDEA

Since 2002 at FUS:Since 2002 at FUS:

Fr Terrence Henry was named as President Dr. Max Bonilla took the position as acting VPAA and then in 2004 was named VPAACompleted a NCA/HLC PEAQ Team visit resulting a progress report on Assessment due in Fall 2009Saw Dr. Bonilia commit to the faculty in 2004 that he would seek appropriate ways to widen the evidence used in Faculty Evaluation at FUS New CORE Program Development starts in 2007A revised set of forms for faculty evaluation was used, starting in 2007, which asks for more quantitative evidence and allows participation in The University of Delaware Study (UDELII)

Practical Reasoning

Franciscan University, in its very Mission, as a part of its self understanding as a community, is committed to a wide conception of rationality with a rich understanding of practical reason.Department Chairs and Faculty have been individually and collectively encouraged to consider the connections between Department Planning (Program Review), Assessment of Student Learning data and evidence, and individual Faculty Performance Evaluation - especially in setting future goals It is through the qualitative narrative in these documents that Chairs and faculty communicate the situational contexts in which their performance efforts are intentionally made. The arguments made for funding, curriculum changes, faculty development or other items needing approval are made in these reports. The argument is part of our praxis, practical reasoningThis practical reasoning of the faculty and chairs closest to the Internal Goods (in MacIntyre’s Sense) of the practices of Teaching/Learning, Scholarship and Service

IDEA is Great - but not for Measures and Evidence

of :

Instructional Design & Planning Instructional Delivery & Innovation Feedback Given to students on work Advising, thesis mentoring and other instructional departmental activity valued at FUS and your UniversitiesScholarship, Internal Service or External service

Evaluating Teaching and Learning is hard, but

Scholarship and Service?

The University of Delaware Outside the Classroom Productivity Study (UDELII) Measures and definitions Operationally defining, and therefore understanding better many of the tasks, projects and processes which make up the practices of Teaching, Scholarship and Service at UniversitiesThe study aims at the widest generality and most exhaustive comprehension of measures prudentComparative data from other institutions can inform the practical reasoning demonstrated in evaluations and reports

Review, Consider and Comment on FUS Reports

• Participants at tables review sample Analysis Reports

IDEA and UDEL Reports at FUS

• The following IDEA analysis is like one given to each faculty member each evaluation at FUS

• Note: Average among courses for evaluation

• Comparative Z scores • Discipline vs. Overall T

score

• The following UDEL analysis gives data by CIP CODE from participating institutions These are not intended

and cannot be used through simple comparisons as norms

These can inform prudential argumentation about contribution

These can inform the dialogue about standards of practice

Triangulation and Practice

• Quantity is not Quality • But quantity is one

dimension or aspect of quality - just ask the working parent who tries intensely to have “Quality Time”

• Faculty at FUS are encouraged to put the quantitative aspects (UDEL) of faculty efforts in dialogue and in context through their narrative

• SoTL like projects and funded projects are asked for in quantitative sheets

These are Funded through mini-grants from Assessment Office

• They are accompanied by AQIP like action planning (NCA/HLC alternative accreditation path)

• These then become exemplary projects for assessment / accreditation and faculty development

• They are saved on the ASPIRE website at FUS and become artifacts in our Narrative of Practice at FUS

On Going Faculty Formation and

Input

Faculty may (and have) added items to the quantitative worksheet Faculty are encouraged and paid to engage in SoTL activities and advance their own development in Scholarly Teaching as well as Scholarship Proper (including SoTL)The assessment office helps with project preparation and seeks approval from VPAA as well as aids in getting SoTL work published or presented. Assessment office helps with the creation of the “Flight Plan” which is a larger Narrative or plan for Teaching Scholarship and Service

Appraisal and Action

The appraisal for Merit (or probation) level award is made by the VPAA in consultation with the Chair after reviewing evidence and commentary from the faculty narrative, the chair evaluation, and an analysis from a central office reader and preparer of the materials. The consistency of merit recommendations between the preparer and the VPAA is calculated via inter-rater reliability. Last year’s Kappa Value was at at 0.76 or “substantially similar” The Faculty member and chair are encouraged to create individual action plans and longer range “Flight Plans” The VPAA sends suggestions for goals and actions with his feedback to each faculty annuallyThe assessment officer aids faculty in achieving plans and goals

IDEA and UDEL help in achieving some level of objectivity in measures

but…

……by definition the by definition the personal judgment used in personal judgment used in making a performance making a performance appraisal is subjectiveappraisal is subjective

SOFIA: A Continuous Improvement Process

Seeking-Searching for Standards of our Practices at FUS IDEA, UDEL, Faculty Standards Committee at FUS, Best Practices from Peer Universities etc. Ongoing- Formation Feedback &Input - from Faculty through the assessment office to aid in creating excellent practices at FUS and when possible to create public SoTL projectsAppraisal (summative) Annual or other (Promotion Tenure) and Action PlanningSOFIA is also evolving and ongoing as we assess our appraisal system and its value to us at FUS

Like others, we at FUS Like others, we at FUS have found stages of have found stages of faculty resistance* faculty resistance*

Stage 1: Disdainful Denial. Stage 2: Hostile Resistance.

Stage 3: Apparent Acquiescence. Stage 4: Attempts to Scuttle.

Stage 5: Grudging Acceptance. After two (or more- we will see) years of operation, the faculty find that the system can

actually be of some value. This is as good as it gets! There is no subsequent stage where faculty are happy with the system.

*Taken and adapted from: http://ctl.stanford.edu/Tomprof/postings/610.html 7/7/08

Conclusion

• Thank you for your attention.

• Any Questions?