Course Enhancement Questionnaire Management

Guide

V 6.0

Student Analytics, Insights & Modelling

31/08/2020

Index

Overview............................................................................................................ 1

Requesting access for School Survey Coordinators ......................................... 2

Course Enhancement Questionnaire Management in EUCLID ........................... 2

Searching Courses ........................................................................................... 2

Editing Course Questionnaire Details .............................................................. 3

Bulk Update .................................................................................................... 4

Individual Update ........................................................................................... 5

Incorrect Course Organiser ............................................................................. 6

Teaching Staff Lookup ..................................................................................... 6

Extracted Details ................................................................................................ 7

Eligible Courses and Students ............................................................................ 8

Access to Response Rates .................................................................................. 9

Access to CEQ Results ........................................................................................ 9

Access to CEQs for Students .............................................................................. 9

Contacts ........................................................................................................... 10

Requesting changes or new School Specific Question Sets ........................... 10

1

Overview The Course Enhancement Questionnaire Management screen in EUCLID is designed to help you manage the various aspects of the Course Enhancement Questionnaires (CEQs) process:

1. Ensuring that the Course Organiser details for the course are correct.

2. Choosing when the questionnaire should ‘Open’ (Monday date) and ‘Close’ (Sunday date).

CEQs close automatically at 23:59 on the selected Close date. If response rates are low and the deadline

should be extended, administrative staff in Schools would need to email the Student Analytics, Insights &

Modelling team at student.analytics@ed.ac.uk with the new closing date at least 3 days before the close

date that was entered in the EUCLID screen for CEQ.

3. Selecting the appropriate ‘Reporting Phase’ for the questionnaire i.e. Semester 1, 2 or Summer.

This relates to when the questionnaire closes not to when the course took place. If not sure, click on the

question mark icon next to ‘Close Week’. This will display the hyperlink to the calendar: check the latest

close dates.

4. Specifying the desired format i.e. paper or online.

5. If applicable, selecting School Specific question sets to be used in addition to Core questions.

The Core and Staff question sets won’t need to be added as part of this process. The Staff question set will

be used for the staff that you will include for each questionnaire.

6. Teaching staff names do not need to be included during 2020/21, this is optional. If the School has

decided to include staff names, remember that if the Course Organiser was teaching, their details will

need to be added using the ‘Teaching Staff Lookup’ and ‘Add Lookup’ button.

7. Confirming that the preparation is completed. IMPORTANT: If the Preparation Completed is set to ‘Yes’

and the information saved, an email notification will be sent to the Course Organiser with the details that

have been set-up. This will give Course Organisers an opportunity to contact administrators before the

CEQ is launched should there be any mistake in the set-up. CEQs cannot be amended after the Open date.

8. Saving the information.

IMPORTANT: Details should be prepared at least 2 weeks prior to a questionnaire opening date. The

Student Analytics, Insights & Modelling team will work on building CEQs for courses that have been set to

Preparation Completed ‘Yes’ on a weekly basis.

2

Requesting access for School Survey Coordinators School administrators dealing with the set-up of information in the Course Enhancement Questionnaire Management screen, will require the role of ‘School Survey Coordinator’ in EUCLID. Where School administrators do not already have their role recorded in EUCLID as ‘School Survey Coordinators’, this should be applied for and authorised by using the standard form available here:

Student Systems User Account Application Form

If an option to select the ‘School Survey Co-ordinator’ role is not available, enter ‘School Survey Co-ordinator’

in the free-text box for ‘other’ and select ‘Student lookup and immigration overview’ for one of the options.

Course Enhancement Questionnaire Management in EUCLID Log into EUCLID via the standard MyEd channel. The full list of options available on the screen will vary depending on your role and access rights to EUCLID.

1. Select the ‘DPTs and Course’ tab 2. Under ‘Course Creation, Approval and Maintenance’,

select ‘Course Enhancement Questionnaire Management’

Searching Courses The Search screen will default to display your School. If you are not sure what parameters to use, click on the ‘Search’ button at the bottom right of the screen to retrieve all courses within your School. You can then type into any of the blank fields under the column heading to filter.

For the parameters, you can use any of the following filters to search for courses. Click on the ‘Reset Search’ button if you want to clear the filters.

1. Course – start typing part of the course name or course code and a drop down list will appear for you to select the course. By using this filter, you will only be able to select one course

2. Study level – select either Undergraduate or Postgraduate 3. Period – select when the course is delivered e.g. Semester 1 4. Occurrence – choose from ‘Available to all students’, ‘Part-year visiting students only’ or ‘Not available

to visiting students’ 5. Subject Area – select one from the School specific drop down list e.g. Geography

The ‘Email Survey Team’ button can be used to contact the Student Analytics, Insights & Modelling team if:

you are having issues with the process

you would like to add a new School Specific question set that is not already available

the course is showing as ‘extracted’ and you would like to set up another questionnaire for the course

The button opens an email window pre-populated with the team’s email address student.analytics@ed.ac.uk

3

Editing Course Questionnaire Details

There are two ways you can edit and update details for Course Enhancement Questionnaires (CEQs). You can choose your preferred method depending on what you feel more comfortable using and what works for you. For example, you might prefer to update details using only the ‘Individual Update’. It is important to bear in mind that if you select a course (for example with a bulk update) that has already been edited, the new information you save will overwrite the old.

You can use a ‘Bulk Update’ for courses that require the same information for:

1. Open Week 2. Close Week 3. Delivery (i.e. paper or online) 4. Reporting Phase 5. School Specific question sets

After using the ‘Bulk Update’ - where possible and only for the courses that have the above five settings in common - use the ‘Individual Update’ to edit each single course for:

1. Checking if the Course Organiser details are correct 2. If needed, including teaching staff that should appear on the questionnaire 3. Setting the Preparation Completed to ‘Yes’

You can use the ‘Individual Update’ alone to prepare all the details. REMEMBER: Details should be prepared at least 2 weeks prior to a questionnaire opening date. The Student Analytics, Insights & Modelling team will work on building CEQs for courses that have been set to Preparation Completed ‘Yes’ on a weekly basis. The Course Enhancement Questionnaire Management screen in EUCLID will hold the information that you have saved, which means that going forward, for the next academic year, you will only need to review and update the courses where necessary and set the Preparation Completed to ‘Yes’. NOTE: During Semester 2, any YR or FLEX course that was surveyed in Semester 1 will appear as ‘extracted’.

Please email the Student Analytics, Insights & Modelling team at student.analytics@ed.ac.uk if a particular

course code needs to be surveyed again in Semester 2. The Student Analytics, Insights & Modelling team will

switch the course back to ‘edit’ mode so that you can enter the new details as necessary.

Why Online?

Online questionnaires are preferred to paper for a number of reasons:

Saving on printing costs

More environmentally friendly

More flexible – students can complete the questionnaires both in or outwith class hours and online reminders can be set up to encourage completion

Cutting on administration time – paper questionnaires require more time to be spent on logistics, distribution, collection and returning completed questionnaires for scanning

More detailed and readable free text comments – with online questionnaires, comments appears as typed text whereas with paper, a picture of the handwritten comment is scanned into the system

Students are less likely to be identified through their response

Faster reporting on questionnaire results – reports for online questionnaires are sent to the Course Organiser when closing a questionnaire whereas completed paper questionnaires need to be scanned into the system (this is outsourced at EvaSys) before reports can be sent

4

Bulk Update You can use the ‘Bulk Update’ if you have more than one course requiring the same information for the following settings:

1. Open Week 2. Close Week 3. Reporting Phase 4. Delivery 5. School Specific question sets

NOTE: Teaching staff cannot be selected in bulk but can only be included by editing each course individually. IMPORTANT: If you select a course that has already been edited, the information will be overwritten.

NOTE: During Semester 2, any YR or FLEX course that was surveyed in Semester 1 will appear as ‘extracted’. Please email the Student Analytics, Insights & Modelling team at student.analytics@ed.ac.uk if a particular course code needs to be surveyed again in Semester 2. The Student Analytics, Insights & Modelling team will switch the course back to ‘edit’ mode so that you can enter the new details as necessary.

1. Select the relevant courses by clicking on the box next to their code 2. Click on ‘Bulk Update’

Specify the following information:

1. Open Week – select a Monday date from the drop down list. The first email invitation with a link to the

questionnaire will be sent to students on the selected date at 10:00.

2. Close Week – select a Sunday date from the drop down list. The questionnaire will close at 23:59 on the

selected date.

3. Reporting Phase – select the reporting period e.g. Semester 1, Semester 2 or Summer. This relates to

when the questionnaire closes not to when the course took place. If not sure, click on the question mark

icon next to ‘Close Week’. This will display the hyperlink to the calendar: check the latest close dates.

4. Delivery – specify if this should be an online or paper questionnaire.

5. Question Sets – if you would like to include School Specific question sets, select all that apply. To view

the questions, click on the ‘details’ hyperlink for the set. If a set is missing, contact the Student Analytics,

Insights & Modelling team by clicking on the ‘Email Survey Team’ button.

6. Preparation Completed – IMPORTANT: This must be set to ‘No’ if you have partially updated the

information and need to return to the courses at a later time or need to include teaching staff by

individually updating each course record. If set to ‘Yes’ an email notification will be sent to the Course

Organiser with the details that have been set-up for the selected questionnaires.

7. Save the information.

5

Individual Update To edit a single course, under the Action column, click on the ‘Edit’ button.

NOTE: During Semester 2, any YR or FLEX course that was surveyed in Semester 1 will appear as ‘extracted’. Please email the Student Analytics, Insights & Modelling team at student.analytics@ed.ac.uk if a particular course code needs to be surveyed again in Semester 2. The Student Analytics, Insights & Modelling team will switch the course back to ‘edit’ mode so that you can enter the new details as necessary.

IMPORTANT: Check if the Course Organiser details against the course are correct. If the details are incorrect, please update them in the ‘Course Creation and Maintenance’ (CCAM) screen in EUCLID.

Specify the following information:

1. Open Week – select a Monday date from the drop down list. The first email invitation with a link to the

questionnaire will be sent to students on the selected date at 10:00.

2. Close Week – select a Sunday date from the drop down list. The questionnaire will close at 23:59 on the

selected date.

3. Reporting Phase – select the reporting period e.g. Semester 1, Semester 2 or Summer. This relates to

when the questionnaire closes not to when the course took place. If not sure, click on the question mark

icon next to ‘Close Week’. This will display the hyperlink to the calendar: check the latest close dates.

4. Delivery – specify if this should be an online or paper questionnaire.

5. Question Sets – if you would like to include School Specific question sets, select all that apply. To view

the questions, click on the ‘details’ hyperlink for the set. If a set is missing, contact the Student Analytics,

Insights & Modelling team by clicking on the ‘Email Survey Team’ button.

6. Teaching Staff – This is optional. If needed, type the staff name that should be included in the

questionnaire and select it from the Lookup. Click on ‘Add Lookup’.

REMEMBER: If teaching staff are being included and the Course Organiser was teaching, add their name.

7. Preparation Completed – if you have partially updated the information and need to return to the course

at a later time, select ‘No’. If the details have all been prepared and are ready for the questionnaire,

select ‘Yes’. IMPORTANT: When selecting ‘Yes’ and clicking ‘Save’, the Course Organiser will receive an

email notification with the details that have been set-up for the questionnaire.

8. Save the information.

6

Incorrect Course Organiser When a CEQ closes, the Course Organiser will automatically receive the ‘Course Organiser’ report with responses to all the questions that were included on the CEQ so it is important that the correct Course Organiser is recorded against a course record.

If the incorrect Course Organiser is displayed against a course, please update the details in the ‘Course Creation and Maintenance’ (CCAM) screen in EUCLID.

If the course is appearing as ‘Extracted’ in the EUCLID screen for Course Enhancement please notify the Student Analytics, Insights & Modelling team as soon as possible. This is because the CEQ has already been built and details must be updated by the Student Analytics, Insights & Modelling team too.

Teaching Staff Lookup Staff names added to the course using the ‘Teaching Staff Lookup’ and ‘Add Lookup’ button will appear on the CEQ for the 4 Staff questions:

1. was organised and well prepared 2. was good at explaining the subject 3. was approachable and willing to help 4. stimulated my interest in the subject

If a staff name is not appearing on the ‘Teaching Staff Lookup’, try to type either just the firstname or the surname, check if the person has maybe recently changed their surname or if there is a different spelling, for example John could be under Johnathan. If the name can’t be found, a staff record must be created by Student Systems.

Please note that a student record is separate from a staff record – if they are teaching on University’s premises, they should either have an UUN or a VRS (Visitor Registration System) number and a staff record can be created for them by Student Systems.

Contact Student Systems at studentsystems@ed.ac.uk with the staff UUN (or VRS number) and with their staff email (student emails cannot be used), ask for a new staff record to be created so that they can be included on Course Enhancement Questionnaires (CEQs). Please note that NHS staff can’t be included on CEQs.

Once Student Systems creates the staff record, they will appear on the ‘Teaching Staff Lookup’.

If there are two or more staff with the same name, select the first name in the Lookup and click ‘Add Lookup’.

You will then be able to see the School and the staff UUN. Click on the red cross next to the name that should NOT be included.

If the error ‘This record can not be added to the list as some details are missing’ is displayed when

clicking on the ‘Add Lookup’ button, please contact Student Systems at studentsystems@ed.ac.uk

and provide the staff email.

7

Extracted Details

Details should be prepared at least 2 weeks prior to a questionnaire opening date. The Student Analytics, Insights & Modelling team will work on building CEQs for courses that have been set to Preparation Completed ‘Yes’ on a weekly basis. Once the Student Analytics, Insights & Modelling team has extracted the details, you will not be able to edit them. If you have made a mistake (e.g. teaching staff was included by mistake or staff have been missed out and you would like to include them), please contact the Student Analytics, Insights & Modelling team by using the ‘Email Survey Team’ button. The Student Analytics, Insights & Modelling team will check if the CEQ has already been sent and if so, how many students have already responded. CEQs cannot be amended after the Open date. If confirming to go ahead, the Student Analytics, Insights & Modelling team will have to delete the CEQ and set the course back to ‘edit’ mode for you to update the details in the EUCLID screen for Course Enhancement. Responses already gathered will be lost and you will have to notify students that they will receive a new email invitation to complete the questionnaire. You can check details and sort courses using the columns in the screen:

1. Under the ‘Course Code’ column on the left-hand side, click on the course code to see the details that have been set-up

2. Click on the arrows in the ‘Prep. Complete’ column to sort for courses that are ready or not 3. Click on the arrows in the ‘Action’ column to sort for courses that have been extracted or can be

edited

NOTE: If you have a large number of courses to check, you can contact the Student Analytics, Insights & Modelling team to ask for a ‘Review’ file. This is an excel file listing the details for each course so that it is easier for you to check the information that has been entered. You can’t use the ‘Review’ file to provide the details to the Student Analytics, Insights & Modelling team as these must be entered in the EUCLID screen for Course Enhancement.

8

Eligible Courses and Students All taught, credit bearing courses (UG and PG) delivered by the University of Edinburgh should be surveyed using the Course Enhancement Questionnaires (CEQs) process described in this guide. Courses with 0 credit will not appear in the Course Enhancement screen and can’t be included as part of this process.

CEQs are sent to current students, in the current academic year, that are enrolled on the course with one of these Mode of Study:

Mode of Study Code Mode of Study

CLASS_ASSESS_E Classes & Assessment excl. centrally arranged exam

CLASS_ASSESS Classes & Assessment incl. centrally arranged exam

DIST_LEARN Distance Learning CLASS_ONLY Class Only

DIST_LEARN_NC Distance Learning (Non-Credit)

CEQs are not sent to students enrolled with Mode of Study ‘Assessment’ or ‘Exam Only’.

The eligible student cohort number is displayed in both the ‘edit’ window and the details window (accessible by clicking on the course code under the ‘Course Code’ column).

When students are enrolled, the default Mode of Study is pulled from what is recorded against the course record itself. If a course has been created with Default Course Mode of Study ‘Assessment’ or ‘Exam Only’, the enrolments might all appear with this status and the cohort on the Course Enhancement screen will be displayed as 0. It is important to select the correct default Mode of Study when a course is created as enrolments are affected. Each School structure is different, if you are not sure who deals with course creation, please check with your manager. If you notice that the cohort is 0 for a course that has students attending classes or distance learning, check the course record itself in EUCLID under ‘DPTs and Courses’, ‘Course Creation, Approval and Maintenance’ (CCAM), ‘View Course’. After locating the course, open the ‘Reference Data’ tab to check the Default Course Mode of Study.

If the Mode of Study against the course and/or student enrolments is incorrect:

Mode of Study should be updated against the course in EUCLID, see guidance here: http://www.studentsystems.is.ed.ac.uk/staff/Support/User_Guides/OCE/OCE_Mode_of_Study.html

Mode of Study should updated against relevant student enrolments in EUCLID/Students/Student Hub: you can use the ‘Full search’ to retrieve the course code, current year and current students e.g. 2020/21. In the Assessment tab for each student, you’ll be able to see the Mode of Study recorded against the student enrolment for the relevant course code. The Mode of Study can’t be changed in bulk for a whole class but has to be done for each student by clicking on the ‘Edit’ button.

9

Access to Response Rates

Any staff with an ed.ac.uk account can access weekly Response Rates (usually updated on Tuesdays) on

Power BI: CEQs Response Rates

The first email invitation with a link to the questionnaire is sent to students at 10:00 on the selected Open date. Course Organisers receive an automatic email notification when the CEQ is sent to students so that they know when the CEQ is open. This email notification will have a hyperlink to the webpage with information on how to enable the link to CEQs in Learn: Link to CEQs in Learn The link to CEQs can be enabled by anyone with ‘Instructor’ access to the course in Learn and can be done any time, staff do not need to wait for the CEQ to be set-up. By enabling the link, Course Organisers will be able to see information (real time response rates, opening and closing dates) for any CEQs that have been scheduled to launch for their courses. Students will be able to see information (link to questionnaire, opening and closing dates) for any CEQs that have been sent to them. CEQs close automatically at 23:59 on the selected Close date. If response rates are low and the deadline should be extended, administrative staff in Schools would need to email the Student Analytics, Insights & Modelling team at student.analytics@ed.ac.uk with the new closing date at least 3 days before the close date that was entered in the EUCLID screen for CEQ.

Access to CEQ Results

When a CEQ closes, the Course Organiser will automatically receive the ‘Course Organiser’ report with responses to all the questions that were included on the CEQ. At the same time, if a staff name was included on the CEQ, they will also automatically receive responses to the 4 Staff questions relevant to them. Directors of Quality will receive responses to the free-text Core questions at the end of each semester for

CEQs that have closed.

Quantitative results are published in Power BI: CEQ Results

CEQ Results are published weekly, for CEQs that have been closed, at different levels:

Core Questions

Core Questions - School level

Staff Questions - these are at aggregated level, no name is published There is a threshold of 10 respondents applied to CEQ Results. Anyone with an ed.ac.uk account can access

Power BI.

Access to CEQs for Students

Students can access CEQs via:

The first email invitation sent at 10:00 on the selected Open date. This email is delivered from student.surveys@ed.ac.uk on behalf of Lisa Dawson

Learn, if the link to CEQs has been enabled

The Notification portlet in MyEd: http://edin.ac/CEQ

10

Contacts To contact the team for support on using the CEQ screen in EUCLID, you can click the ‘Email Survey Team’ button, either from the home page or from the individual course editing screen. The ‘Email Survey Team’ button opens an email pre-populated with the team’s address student.analytics@ed.ac.uk

Alternatively, you can contact Cinzia Discolo (Surveys Support Officer) at Cinzia.Discolo@ed.ac.uk or via Microsoft Teams.

Requesting changes or new School Specific Question Sets The 8 Core questions are used for all CEQs automatically:

1. The course was well organised 2. The learning aims of the course were clear 3. Feedback so far has been helpful and informative 4. This course has been intellectually challenging 5. The course has developed my skills and abilities 6. Overall I am satisfied with the quality of the course 7. What did you find most valuable about the course? 8. What improvements, if any, would you make to the course?

The following 3 questions are also included automatically to all CEQs:

When using Learn it was easy to find the course-specific resources I was looking for Definitely Agree / Definitely Disagree

A Schedule of Adjustment (SOA) is an agreed change to your course such as extra time for assignments, smaller tutorial groups or learning materials provided in a different way. Some adjustments will happen automatically, and some adjustments will happen only when you need them and only if you ask (for example additional information on an assignment question). Adjustments are arranged by the Student Disability Service and implemented by the School responsible for the delivery of the course.

Did you have an SOA arranged by the Student Disability Service? Yes / No

Were the adjustments agreed in your SOA, either automatically or after you asked, implemented as expected by the School delivering this course? Yes / No / Not Applicable

The Senate Quality Assurance Committee (SQAC) are reviewing the wording of 2 new questions asking students about their experience of hybrid teaching and learning. These questions will be added to all CEQs in 2020/21 by the Student Analytics, Insights & Modelling team. In addition to the above questions, Schools can include School Specific sets. Changes and requests for School Specific Sets should be kept to a minimum. If you would like to include a new School Specific set that is not already available in the CEQ screen, you will need to complete a ‘School Specific request form’. The form is an excel file designed to provide the questions in a specific format to the Student Analytics, Insights & Modelling team. To request the form, contact the Student Analytics, Insights & Modelling team at student.analytics@ed.ac.uk The Student Analytics, Insights & Modelling team would create the set and make it available on the EUCLID screen for CEQ so that it can be assigned to the relevant course questionnaire(s).

NOTE: School Specific sets are normally agreed at School level. When requesting a School Specific set, please think of themes rather than course-by-course questions. Our general guidance is to have School Specific sets that can be assigned to a number of courses delivered in a similar way or that have the same themes the School would like to explore/ask feedback for. This way, each year, changes to sets should be minimal, with small changes or new sets for themes not already covered. For example, if the questions are too specific for a single course and the course content changes each year, staff would need to be careful in remembering to submit a new School Specific request form in future years.