Requirements for Student Evaluation of Teaching and Learning [Approved by the University Board for Teaching and Learning on 9 July 2013 for implementation from October 2013] [For the purposes of the processes described in this document, in Henley Business School the Programme Area Director will be fulfilling the functions of the School Director of Teaching and Learning in respect of programme evaluation and the Director of Studies will be fulfilling the functions of the School Director of Teaching and Learning in respect of module evaluation.]

Introduction and definitions 1 These Requirements for Student Evaluation of Teaching and Learning were informed by the

recommendations contained within the Final Report of the Working Group on Programme and Module Evaluation, which was approved by the University Board for Teaching and Learning on 5 November 2012.

2 The University seeks and obtains feedback from its students in a range of ways and at a

number of levels. This includes module questionnaires, Student/Staff Liaison Committees, programme evaluations, discussion and feedback sessions and informal dialogue.

3 In seeking student opinion, the University aims to provide useful information to relevant staff members, including module convenors, School Directors of Teaching and Learning and Associate Deans (Teaching and Learning). This information is essential in terms of both quality assurance and the enhancement of the student experience of teaching and learning.

4 In establishing the requirements for student evaluation set out below, the University aims to harmonise practices across Schools and Faculties in order to promote transparency, support quality assurance principles and enable comparison across the Faculties.

5 This document is primarily concerned with student evaluation at module and programme level. For further information on the operation of Student/Staff Liaison Committees (SSLCs), please refer to the policy on Student representation at the University of Reading.

6 This policy applies to undergraduate and taught postgraduate programmes. It also applies to taught modules offered within professional doctorate programmes, but does not apply to sessions offered in the Researcher Development Programme nor to the evaluation of doctoral programmes, which is fulfilled through processes developed by the Graduate School.

Module evaluation 7 Whilst supporting the appropriate use of a range of formal and informal feedback

mechanisms by Schools, the University’s policy is that formal end-of-module evaluation in the

Guide to policy and procedures for teaching and learning Section 4: Student representation, evaluation and complaints

form of module questionnaires is an essential component of the student feedback process. Each module should normally be evaluated on a bi-ennial cycle.

Method of evaluation 8 The University has established a standard Module Evaluation Template for use in formal end-

of-module evaluations (please see Appendix 1). The use of this form is not mandatory for the Session 2013/4, but will be for the Session 2014/5 by which time the University hopes to have in place a University-wide electronic system with additional centralised support for Schools. Section 1 of the Template comprises a series of questions grouped into four sub-sections (Content and Structure, Assessment and Feedback, Quality of Teaching and Learning and Learning and Teaching Resources), which must be included in all module evaluation questionnaires. Students are asked to respond using a five-point numerical scale and are also given the option of including qualitative comments under each sub-section.

9 Section 2 of the Template gives Schools the option of including up to 10 locally-determined, module-specific questions which might comprise a combination of numerical scale questions and more open-ended qualitative questions, as desired. Appendix 2 provides a list of possible open-ended questions for Schools to refer to as a guide, although this should not be seen as a definitive list.

10 The Template requires Schools to clearly state where and when students’ responses will be reviewed.

11 Adoption of this method is intended to ensure a transparent approach and a degree of harmonisation across Schools and Faculties, and to prepare a path toward a more automated system of module evaluation in due course. It enables cross-institution comparison of data obtained under Section 1, whilst recognising the need for flexibility to satisfy local requirements. Qualitative feedback is seen as complementary to the quantitative information gained from the numerical scale questions.

12 The Module Evaluation Template is available as a Word document (http://www.reading.ac.uk/nmsruntime/saveasdialog.aspx?lID=89913&sID=87193) and as a pro-forma on Blackboard.

Frequency of evaluation 13 Schools are not expected to evaluate each of their modules every year. Instead, they are

required to operate a bi-ennial cyclical evaluation programme for all modules, with the following exceptions where annual evaluation will be required for a period of three years:

(i) new modules; (ii) modules which have undergone significant changes; (iii) modules where significant concerns have been raised, for example by SSLCs, a

Periodic Review panel or by previous module evaluations.

School Directors of Teaching and Learning will be ultimately responsible for determining which modules fall within categories (ii) and (iii)

14 Boards of Studies will have oversight of module evaluations for programmes within their remit and will be responsible for maintaining a comprehensive record of the cycle of module evaluations and any underlying rationale. This information can then feed into the external

examining process as appropriate, the SPELT process and the six-yearly Periodic Review process.

Questionnaire Administration

15 Schools may choose to use either paper-based or electronic questionnaires depending on local circumstances and resources available, including cohort-size, the availability of software for collation/analysis of results (for example, Optical Mark Recognition software) and staff time available for data-inputting.

16 Whilst the use of Blackboard or other online survey tools such as Bristol Online Surveys (BOS) can ease the administrative burden on Schools in respect of data collation and analysis, it should be noted that electronic methods of module evaluation can lead to decreased response rates. Paragraph 17d below suggests a number of approaches which might be adopted in order to maximise completion rates when using electronic methods.

17 The University has established a set of good practice principles to guide Schools in respect of questionnaire administration as follows:

a. Where paper questionnaires are being used, questionnaires should be distributed towards the beginning of the lecture/seminar/workshop to prevent the tendency to complete questionnaires in haste at the end of a session;

b. Where paper questionnaires are being used, a Course Rep or a member of staff other than the module convenor/lecturer should be responsible for collecting the completed questionnaires and taking them to the relevant School Office in order to preserve anonymity and encourage honest and constructive feedback;

c. Similarly, where Blackboard or other online questionnaire methods are employed within a class setting, the module convenor/lecturer should ensure that he/she is clearly positioned out of the line of vision of students’ PC/laptop screens;

d. In order to maximise completion rates for electronic surveys, Schools may wish to adopt one or more of the following approaches: taking groups of students to IT labs towards the end of a lecture, encouraging students to access Blackboard via a smart phone or tablet, or using specialist polling software;

e. Schools should aim for a minimum completion rate of 50%. Where small cohorts are involved, it may be necessary to aim for a higher rate of completion.

Data collation and analysis of module evaluation

18 Schools are required to collate and analyse the results obtained from module evaluation questionnaires at the end of each Academic Session.

19 The results of module evaluation questionnaires should be presented as a series of data ideally in graphical or tabular format. Qualitative comments obtained should be included alongside quantitative data.

Reporting of module evaluation

20 School Directors of Teaching and Learning will maintain oversight of the analysis of student evaluation findings and will be responsible for submitting a results package, complete with

lecturer(s) responses, to SSLCs and Boards of Studies for review at the first meeting of the next Academic Session. It is recognised that evaluation results should be treated with caution since it may not be beneficial or reasonable for some comments to be in the public domain - personal comments about lecturers, for example. Names of staff and students should be redacted from student evaluation findings in advance of their submission to SSLCs and Boards of Studies. Personal issues should be dealt with privately by SDTLs in conjunction with module convenors. If issues cannot be resolved, then the Head of School will need to be informed and an appropriate course of action decided upon.

21 Boards of Studies are formally responsible for responding to student evaluation including results of module evaluations. They are required to have a standard item for report on their agenda on student evaluation, which should, depending on the timing in the academic year, include results of module evaluations. The Student representation at the University of Reading policy also specifies that SSLCs should receive and discuss the results of module evaluations and programme evaluations.

22 Schools are required to make SSLC and Boards of Studies minutes, incorporating student evaluation findings and actions, available to all students within the School via Blackboard. Names of staff and students should be redacted from published information, as appropriate. Chairs of SSLCs/Boards of Studies are responsible for ensuring that minutes are made available in this way.

23 As part of the annual programme reporting process, the Associate Deans (Teaching and Learning) (ADTLs) will ensure that any issues raised by module evaluation within their Faculties are dealt with appropriately and in a timely manner. A section within the Annual Programme Report template requires Boards of Studies to comment on the key outcomes of evaluation and review mechanisms, including programme and module evaluation, identifying key issues and any actions that will be taken in response, and examples of good practice. Faculty Annual Quality Reports provide assurance that Boards of Studies have considered, analysed and evaluated the relevant data and identify any particular issues which require further action at Faculty or University level.

24 Schools/Departments may wish to consider involving Course Reps in devising, distributing and reporting the outcomes of end-of-module evaluations, although ultimate responsibility will remain with the SDTL/module convenor. Where Course Reps are involved, they should be reminded of the potential sensitivities around the data collected and the need for confidentiality.

Providing feedback on module evaluation to students 25 Schools must ensure that students receive informative and timely feedback on the actions

taken as a result of module evaluations (and not just feedback on the data obtained from evaluations). Such feedback may be transmitted in a variety of appropriate ways but should be communicated by the relevant module convenor directly to those on the module and to a wider School audience.

26 To ensure that actions taken as a result of the module evaluations are communicated to students, a range of the following mechanisms could be considered:

a. publication on Blackboard of a summary of points raised and actions undertaken in response;

b. publication of SSLC and Boards of Studies minutes on Blackboard (see paragraph 22 also);

c. communication via Course Reps; d. use of School/Departmental noticeboards; e. oral updates during lectures by the module convenor, Programme Director or School

Director of Teaching and Learning; f. use of social media; g. dissemination via paper or electronic newsletter, for example making use of a “You

Said – We Did” format; h. publication of feedback actions in programme or module handbooks; i. email sent to relevant students.

Mid-module evaluation

27 In addition to the requirements set out above in relation to end-of-module evaluation, all Schools are strongly encouraged to undertake some form of informal, light-touch mid-module evaluation for all modules on an annual basis. This can be particularly useful in the resolution of practical and operational issues and provides an opportunity for module convenors to react quickly and efficiently to feedback from students and to make (minor) changes which will benefit the current cohort. Student feedback obtained via the Students’ Union and from research carried out by the National Union of Students and the QAA on the student experience in 2012 suggested that this would be welcomed by students and would encourage them to engage in future feedback opportunities.

28 Module convenors may wish to select from a variety of methods of mid-module evaluation, including (but not restricted to):

(i) verbal feedback gathered informally by an academic member of staff with no responsibility for the module under review or by a Course Rep, during a lecture;

(ii) use of the ‘Poll Everywhere’ tool administered using mobile phone responses during a lecture either via PowerPoint with no live feed or via a web connection with a monitored live feed;

(iii) Post-it and flip-chart method; for example, students are asked to note what is working well on the module and what, if anything, could be improved.

29 A trial of mid-module evaluation across a sample of different disciplines and module sizes was undertaken during the Spring Term 2013. A table summarising the effectiveness of, and resource required by, the various survey methods trialled is included as Appendix 3.

Programme evaluation 30 From 2013-14 onwards, all Schools will be required to carry out some form of programme

evaluation for all programmes on an annual basis, in addition to module evaluations. This can be done either at end-of-Part and/or on completion of the whole programme.

31 The University acknowledges the resource implications of carrying out formal, paper-based methods of programme evaluation and considers that formal methods of evaluation might not always be appropriate given the level of survey activity for final year students in particular. Schools are therefore encouraged to make use of alternative, less formal methods of programme evaluation, including (but not restricted to):

a. student end-of-year focus groups; b. informal feedback sessions held with ADTLs/Course Reps (in the absence of academic

staff members from the relevant School).

32 Schools may wish to refer to the good practice examples summarised in Appendix 4 in respect of end-of-Part/programme evaluation.

33 Schools must ensure that students receive informative and timely feedback on the actions taken as a result of programme evaluations, which can be transmitted in a variety of appropriate ways (please see paragraph 26 above).

Appendix 1- Module Evaluation Template 2013-14 [It is suggested that Schools/Departments include all questions in Section 1 below in all module evaluation questionnaires. In addition, they may choose to include up to 10 module-specific questions in Section 2. These may comprise questions with a quantitative scale (ranging from 1 – Strongly Disagree to 5 – Strongly Agree) and/or questions seeking a qualitative response, as desired.] Module title and code

Module convenor

Your degree programme

Please complete this short questionnaire on the module you have just studied. Your responses will be completely confidential and reviewed by the appropriate Board of Studies and Student-Staff Liaison Committee. You will be told of the outcome of the evaluation process, including any actions taken by the School in response to feedback from students, via [complete as appropriate, citing the method(s) by which the outcome will be communicated back to students e.g. via Blackboard, the School noticeboard, electronic newsletter]. Responses will be kept strictly anonymous (so please do not give your name). << comments are invited as to whether students should be given an opportunity to give their name if they want specific feedback, for example >>

Section 1 Content and Structure Strongly

Agree 5

Agree 4

Neutral 3

Disagree 2

Strongly Disagree 1

Not Applicable

The aims and objectives of the module were clear

The module was well structured and organised

The module content was at an appropriate level

I could see how the content of this module linked with other modules in my programme

Comments

Requirements for Student Evaluation of Teaching and Learning

Assessment and Feedback Strongly

Agree 5

Agree 4

Neutral 3

Disagree 2

Strongly Disagree 1

Not Applicable

There was a clear link between the type of assessment(s) and the aims of the module

The assessment criteria were provided when the work was set

Feedback I have received on my work was informative and showed me how to do better next time

I received feedback within 15 working days

Comments Quality of Teaching and Learning Strongly

Agree 5

Agree 4

Neutral 3

Disagree 2

Strongly Disagree 1

The module was generally taught effectively The methods of teaching (lectures, seminars, tutorials, workshops and /or practicals) were appropriate for the content of the module

The lecturer(s) were helpful and approachable Lecturer 1 [name] Lecturer 2 [name] Lecturer 3 [name] There was sufficient opportunities for interaction between students and lecturer(s)

Lecturer 1 [name] Lecturer 2 [name] Lecturer 3 [name] Comments

Learning and Teaching Resources Strongly Agree 5

Agree 4

Neutral 3

Disagree 2

Strongly Disagree 1

Not Applicable

The module content was effectively supported by seminars, practical sessions and/or on-line resources

Handouts and/or support materials such as books or journal/conference papers were useful and relevant

Comments

Section 2 Other Questions [Schools/Departments may wish to include up to 10 questions in this Section, either with a quantitative scale (ranging from 1 – Strongly Disagree to 5 – Strongly Agree) and/or questions seeking a qualitative response.] 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Thank you for your comments

Please add below any further comments on the module

Jennifer Lauren Chetcuti Section name

©University of Reading 2013 Friday 26 July 2013 Page 1

Appendix 2 - Suggestions for generic, open-ended

questions for inclusion in Section 2 of Module

Evaluation questionnaires

1. The best thing about this module was…………..

2. In what ways was your learning supported in this module?

3. What do you think were the main strengths of this module?

4. If you could improve one aspect of the ways in which this module is taught, what would it be?

5. If you could improve one aspect of the ways in which this module is assessed, what

would it be?

6. Please state one thing that was done especially well. Why was that?

7. Please state one thing that was not done well, and how it might be improved

8. For me the main learning outcome of this module has been…

9. Relative to other University modules you have taken, the intellectual challenge

presented was higher/average/lower

10. Relative to other University modules you have taken, the amount of effort you put

into this module was higher/average/lower

11. How could your own development and learning have been enhanced?

12. What percentage of lectures did you attend?

13. Which aspects of this module did you most enjoy?

14. Which aspects of this module did you least enjoy?

Requirements for Student Evaluation of Teaching and Learning

Section name

©University of Reading 2013 Thursday 18 April 2013 Page 1

Appendix 3 – Methods of mid-module evaluation trialled

during the Spring Term 2013

Approach Pre

evaluation preparation

Post evaluation effort

Delivered by Student feedback on method

Overall

Verbal feedback

Low Low Not the module

convenor

Very positive Good for small

groups

Poll Everywhere (by text message)

Medium/High

depending on experience

and whether using

PowerPoint or

live feed

Low Preferably not

the module convenor

Very positive Most effective

overall and suitable for

any size group

Post it & flip

chart

Medium Low Not the module

convenor

Course

Representative?

Positive Fair for small

and medium

sized groups

Paper and

Pen

Medium/High

depending on photocopies

required

High Module

convenor or other

Very positive Works with

large groups but can be

resource intensive

Requirements for Student Evaluation of Teaching and Learning

Section name

©University of Reading 2013 Thursday 18 April 2013 Page 1

Appendix 4 - End of Part/Programme Student Evaluation – Good Practice Examples School/Department Chemistry Maths and Statistics

Title of initiative Part One Review Staff- Student Forums

Primary evaluation method

Themed focus groups Focus groups

Led by Departmental academic staff, teaching fellows/ Postdoctoral staff

Departmental senior T&L academic staff

Rationale Enables more informal and reflective feedback from whole cohort on key T&L themes in a prioritised way.

To amplify the student voice within a large school, allowing more relaxed discussion of relevant issues beyond the formalised structure of the SSLC.

When? End of summer term in Part One Middle of Autumn and Spring terms

Reporting to Department T&L Group Staff-Student Liaison Committee

Examples of changes to practice resulting from feedback

More introductory lectures in difficult subject areas such as Physical Chemistry to help smooth the transition between A level and degree level work. Moving the Physical Chemistry module to the spring term so that students have already completed a term of Maths before starting this course. Introductory practical course to ensure all students are up to speed with laboratory techniques before starting the main course. Introduction of Part 1 module designed to encourage independent learning.

Some books were moved to the Course Collection in the library after students raised the issue of not being able to access popular texts when they needed them. Part 2 skills module will consolidate the generic skills into one module, and combine the mathematical skills development components to create another module, to help students see the relevance of these skills to their degree and the profession.

Completion of feedback loop

Participating students informed of actions taken via email & informal communication within the Department and through Staff Student Committee.

Minutes of the meetings are annotated with feedback regarding actions taken or to be taken, and are made available on Blackboard soon after the meeting, and are passed to the SSLC for further consideration if necessary. Later actions affecting module content are highlighted on the Blackboard Student Portal (in a ‘Student Voice’ box).

Requirements for Student Evaluation of Teaching and Learning

©University of Reading 2013 Thursday, 18 April 2013 Page 2

Contact Dr Elizabeth Page (e.m.page@reading.ac.uk) Prof Matthew Almond (m.j.almond@reading.ac.uk)

Dr Karen Ayres (k.l.ayres@reading.ac.uk) Dr Paul Glaister (p.glaister@reading.ac.uk)