1

Running a successful Athena SWAN survey January 2016

Introduction The Athena SWAN self-assessment team (SAT) in the Department of Health Sciences has run a number of successful surveys with high response rates, and with complete and high quality data. The purpose of this short report is to share our experiences to improve survey uptake in other departments across the University.

Methods for a successful survey There is no one-size-fits-all method for running a successful survey. The Health Sciences SAT drew on the survey literature and previous experience of team members to plan and administer their surveys. The processes are described below.

Developing the survey Developing a survey takes time. Its content and structure should be designed to encourage people to take part and to complete the survey, keeping drop-out rates low. Responsibility for developing the initial survey was assigned to a SAT sub-group, who met to discuss the content and to propose and agree suitable topics for the survey. Questions were drafted and shared within the sub-group, and changes were made in response to feedback. The survey underwent several iterations before it was shared with the wider SAT, when more feedback was provided and the survey was revised further.

As Athena SWAN (at the time) was primarily concerned with improving the working lives of research and academic staff and research students in STEM subjects (its remit has since widened to all staff in all subjects), the survey for research and academic staff was developed first. Staff-specific questions were removed for the PhD/MD student version, and student-specific questions were added. Questions pertaining specifically to research and teaching activities were removed for the professional staff version, with specific questions relevant to this staff group added in. Again, these surveys underwent several iterations before being shared more widely.

The final draft was transferred into SurveyMonkey. Care was taken to assure participants of confidentiality. Questions were worded in accordance with best practice (e.g. non-leading questions, question-specific labels, provision of a ‘don’t know/not sure’ option, a ‘middle’ category, etc) with signposting throughout. Skip technology was used to guide participants past questions that were not relevant to them. A variety of formats were used to engage respondents, e.g. closed and open questions, vertical and horizontal lists, a percentage complete bar, etc. Appendix 1 gives some examples, including the introductory paragraph on the survey, explaining its raison d’être.

Piloting the survey The three survey versions were piloted with people within each staff and student group. Up to six students and members of staff (for both research/academic and admin/technical) were approached and ask to complete a ‘dummy run’ of the survey, and to provide feedback (either verbally or by email) on the suitability of the questions and the ease of completion. These people were purposively chosen to include a

2

variety of gender and grades, and were carefully selected to be those who were likely to respond and who wouldn’t mind completing the survey for a second time once it was opened to the whole Department.

All versions were revised slightly in light of the feedback, including improvement of question wording, and the addition of explanatory text to clarify why some questions were being asked.

Inviting participants to complete the survey The Chair (co-Chairs in later surveys) of the SAT sent a formal email (tailored to each group: academic and research staff; PhD and MD students; administrative and technical staff) inviting members of the Department to complete the survey (Appendix 2 shows the first invitation email for the most recent survey, sent in early December 2015 to research and academic staff). The emails explained the purpose of the survey, the importance that everyone complete it, a link to the SurveyMonkey questionnaire, and a reassurance of confidentiality within the survey team and anonymity in reporting. A deadline for completion was also included with an invitation to contact one of the co-Chairs with any queries or concerns.

Reminders to complete the survey The co-Chairs sent a gentle reminder one week after the initial invitation (Appendix 3). This shorter email thanked those who had completed the survey, gave the response rate so far, and encouraged non-completers to take part.

A second reminder was sent one week later (Appendix 4). This email was shorter than the first reminder and again said thank you to completers, gave updated response rates, and encouraged non-completers to take part. A separate reminder was sent to research group leads, asking them to encourage their staff and students to take part.

A third and final reminder was sent to academic and research staff, and students in the New Year, to capture those who were on leave during December (Appendix 5). The professional staff response rate was so high, it was not deemed necessary to remind them again. An email was also sent to departmental postgraduate tutors, requesting that they encourage the students in their geographical area to take part (Health Sciences students are spread across several sites).

Appendix 6 shows the responses for the academic and research staff throughout the duration of the survey: note the spikes in number of responses on the days of each reminder.

Parallel promotional activities We try to launch the survey at the time of our annual departmental conference. This provides an opportunity for the Head of Department to verbally promote the survey during the introduction to the conference, and to stress its importance. At one conference, stickers were provided to those who had completed the survey saying “I've completed the Athena SWAN survey, have you?” This was ‘just for fun’ but provided a useful platform for informal conversation about the survey.

Soon after the first invitation, colourful posters were displayed in prominent areas around the buildings: by photocopiers, in the kitchen, on toilet doors (Appendix 7).

At the time of the second reminder, chocolates were available at all sites to encourage participation, and a revised poster replaced the first one (Appendix 8).

Verbal reminders were given by SAT members during group meetings and informal coffee breaks.

3

Reporting back Continued engagement with further surveys can be helpfully facilitated by comprehensive feedback of results. In Health Sciences, this has comprised: dissemination of a written report, including an action plan to detail changes that will be made in response to the findings; a departmental Athena SWAN event, part of which was to communicate the survey findings; presentations with Q&A at quarterly departmental meetings; and Athena SWAN sessions at our annual departmental conference.

Summary A variety of means can be used to develop and run a successful survey. Our experience with the Department of Health Sciences’ Athena SWAN survey may provide some useful learning points for others undertaking similar surveys. In summary:

1. Take time to develop the survey. Put together a small team to thrash through ideas, and work together through several iterations to produce the best draft possible. Run the draft past a wider team for further input. When adopting a ‘standard survey’, do take time to check that it is the best survey for your department.

2. Pilot the survey. However careful and consultative the development phase, there will always be further refinements that need to be made. Choose a variety of willing respondents who can complete the draft survey and provide feedback.

3. Launch the survey with a fanfare. Use multiple routes to make it known that the survey is open: email; face-to-face contact with colleagues; posters; conferences and meetings; getting managers and postgraduate tutors on board. Assure participants of confidentiality and anonymity in reporting.

4. Use reminders. Send multiple reminders, giving an update of the response rate, helping responders to see the impact their contribution has made to the response rate, whilst also encouraging non-responders to contribute. Use other means to encourage completion such as reminder posters, and incentives such as chocolates.

5. Report back. Reporting back to responders (and non-responders) ‘closes the loop’ in the process. People can see what others have said (in a summarised and anonymised format), and can see how their individual responses have contributed to the whole. Include action points and give regular feedback at departmental events on progress made.

6. Consider ethical issues throughout. Although formal ethical review is not necessary (unless the survey is for research purposes1), care should be taken to assure participants of confidentiality of data within the survey team and anonymity in reporting. Questions should be worded so that they don’t ‘lead’ respondents. Action should be taken as a result of the survey, and this should be reported back to the respondents to show them that their time and their contribution is valued.

1 Athena SWAN is not a research project. The survey is part of the self-assessment process, designed to improve the working lives of University staff. Questionnaires for research purposes should be submitted to a University (or NHS) ethics committee for review, along with other documentation.

4

Appendices

Appendix 1 Examples of questions and formats from the survey

5

6

7

Appendix 2 Initial e-mail (we sent a similar one to professional services staff and research students)

To all Research and Academic staff in the Department of Health Sciences

We would like to thank you for your continued engagement with Athena SWAN (AS) activity in the Department. Your enthusiasm and support for this agenda is hugely appreciated and means that we can progress activities and initiatives knowing we have the full support of colleagues.

As you know, the AS Charter recognises commitment to advancing gender equality in higher education and awards are conferred in recognition of efforts made to ensure gender equality. We are hoping to apply for an AS GOLD award in November 2016. This is the highest level of award and it will be challenging to gain recognition at this level, but with the continued support of the Department we hope this can be achieved. As part of the self-assessment process, we wish to undertake a survey of ALL research and academic staff. While the focus of Athena SWAN has been on women, the idea is that improvements made for women result in improvements for all.

With this in mind, the AS self-assessment team encourage you all to participate in this survey which will help us in the preparation of our submission. As you know AS awards are important as they are linked to funding provision.

It is vital that the Department builds on its Silver success and can submit a strong application and this survey forms part of that process. We are keen to achieve a really good response rate (hopefully we can exceed our 92% response rate from 2014). Please follow the link to the online survey https://www.surveymonkey.co.uk/r/HS-AS-res-ac-2015 which should take around 10 minutes to complete. Each respondent can access the survey once only. If you have any queries please contact Kate Williams ksw6@le.ac.uk, tel: 252 5432. The survey will close at midday on Tuesday 22nd December.

Thank you so much for your time

Kate Williams and Nicola Cooper

On behalf of the Health Sciences Athena Swan Self-Assessment Team

8

Appendix 3 First reminder email

To all Research and Academic staff in the Department of Health Sciences

Thank you so much to those of you who have already completed the Athena SWAN survey. So far we have had responses from 50% of research and academic staff.

The information that is collected is vitally important to the Department and will enable us to identify areas for improvement as well as celebrate areas of good practice. However, we can only do this if everybody takes the 10 minutes or so that are required to complete the survey. We are keen to achieve a really good response rate (hopefully we can exceed our 92% response rate from 2014). Please follow the link to the online survey if you have not already done so https://www.surveymonkey.co.uk/r/HS-AS-res-ac-2015. Each respondent can access the survey once only. If you have any queries please contact Kate Williams ksw6@le.ac.uk, tel: 252 5432. The survey will close at midday on Tuesday 22nd December.

Thank you so much for your time

Kate Williams and Nicola Cooper

On behalf of the Health Sciences Athena Swan Self-Assessment Team

9

Appendix 4 Second reminder email

To all members of Research and Academic Staff

As you know it's the Athena SWAN survey time of year again.

We know that most of you (78%) have already responded, so thank you. If you haven't had time yet we would be very grateful if you could complete it (https://www.surveymonkey.co.uk/r/HS-AS-res-ac-2015). It is really important that we get responses from as many people as possible; only then can we be certain that the issues identified are those that matter most to the people who work in the Department.

We achieved over a 90% response rate to our last survey, so are keen to reach or exceed that if possible.

Many thanks

Kate Williams and Nicola Cooper

10

Appendix 5 Third reminder email

To all Research and Academic staff in the Department of Health Sciences

Thank you so much for completing the Athena SWAN survey. So far we have had responses from nearly 90% of research and academic staff.

We achieved a 92% response last year, so will leave the survey open until the end of the week in case you haven’t had time to complete it yet and would like to nudge the response rate up to say 93%! Please follow the link to the online survey if you have not already done so https://www.surveymonkey.co.uk/r/HS-AS-res-ac-2015. Each respondent can access the survey once only. If you have any queries please contact Kate Williams ksw6@le.ac.uk, tel: 252 5432.

We will provide a short summary of the survey results at the next Departmental Meeting.

Thank you so much for your time

Kate Williams and Nicola Cooper

11

Appendix 6 Number of responses per day during the research staff survey

Survey launch: 10th December 2015 First reminder: 16th December 2015 Second reminder (and publicised deadline): 22nd December 2015 Third reminder: 6th January 2016

Launch

1st reminder

2nd reminder 3rd reminder

12

Appendix 7 Initial poster

13

Appendix 8 Reminder poster