Taking Course evaluation back to basics

Just what are we evaluating and why?

Scene setting

Student population– Undergraduate 19,096– Postgraduate Taught 5,147

4 Colleges– Arts– Medicine, Veterinary and Life Sciences– Science and Engineering– Social Sciences

Total of 19 Schools and 9 Research Institutes

Course evaluation

Session 2012-13 - EvaSys piloted– 1 School per College

Working Group established to review

Recommendations:• EvaSys rolled out on a voluntary basis• Review of course evaluation

Issues

• Not compulsory to use questionnaires for course evaluation

• Previously introduced standard questionnaire – lasted one year!

• Large and varied means of evaluating courses

• When questionnaires were being used – a wide assortment of areas and questions were being asked.

Just what are we evaluating?

What do we use course evaluation to evaluate?

Areas evaluated

• The relevance of the course?• The student experience on the course?

• The teaching content?

• The quality of teaching?

• The assessment methods?• The ‘popularity’ factor?

Then, there is…..

Areas evaluated

• Assessment of dissertation/project supervisors

• Assessment of placements

• Assessment of learning and teaching support

• Assessment of accommodation

• Assessment of laboratories, seminars• Assessment of GTAs• Assessment of feedback received

What do we do wWhat do we do with the information?

• Management overview/auditing purposes• Provide feedback to lecturers

• Provide feedback to course organisers and programme directors

• Provide feedback to Head of School, Head of College

• Provide information for marketing purposes• Provide feedback to students!

Working Group on course feedback questionnaires

• To identify the purpose of course evaluation

• The nature of feedback processes

• Consideration of the use of questionnaire data

• Administration of feedback processes

Summative

• To provide a snap-shot of the past, a description of how the course ran

• To highlight existing good practice

• To highlight exceptional quality that can be referred to in marketing and recruitment activities

• To recognise teaching as evidence for teaching excellence awards or promotion.

Formative

• To help identify where improvements can be made, and where there are unexpected problems that need to be solved

• To ensure that the quality of teaching is sustained and improved

• To identify where teaching staff might need additional support or resources

• To demonstrate the use of feedback practices across the university for the purposes of audit

Purpose of Course Questionnaire evaluation

Reviewed 300 questions!

Discussed questionnaire design

Course feedback evaluation questionnaire

• 5 core questions to be used on all questionnaires

• Teaching and course evaluation to be reviewed separately

• Optional question sets to survey different aspects

Core Question Set

CORE1a. The lecturer explained things well. • Strongly Agree □ □ □ □ □ Strongly Disagree• or• CORE1b. My project/dissertation/placement supervisor was helpful. • Strongly Agree □ □ □ □ □ Strongly Disagree• CORE2. The course was intellectually stimulating. • Strongly Agree □ □ □ □ □ Strongly Disagree• CORE3. I am satisfied with the overall quality of the course. • Strongly Agree □ □ □ □ □ Strongly Disagree• CORE4. What was good about the course?• OPEN• CORE5. How could this course be improved?• OPEN

Clarify what is being evaluated: Other Question sets

• Teaching Quality set (9 qns)

• Course Quality set (9 + 5)

• PGT set (4)

• Expectation/Marketing set (5)

• Ad hoc set (up to max 22 qns)

What to do with the information?

• Course– Improving course design– Involving students with curriculum design and

development

• Teaching– Improving teaching – Provide source for recognition and award of

teaching

• Feedback– Improving student feedback

Linkage to other processes

• P&DR and Recognition of Excellent Teachers

• Annual Monitoring

• Internal Subject Review

• Marketing

Moving Forward

Establishment of Advisory Board– Strategic overview

– Review potential developments and linkages to other processes

– Identification and dissemination of good practice

– Update and revise policy as required

And Finally….

Any questions?

Catherine Omand

Senior Academic Policy Manager

Senate Office