Enhancing the student experience: working to improve student satisfaction (and NSS scores!)

University of Brighton 14th September 2012Sally BrownEmerita Professor, Leeds Metropolitan University,Adjunct Professor, University of the Sunshine Coast, University of Central Queensland and James Cook University QueenslandVisiting Professor University of Plymouth & Liverpool John Moores University

The current context

A changing HE context in a global economic downturn both for HEIs and our graduating students;

International issues including competition (e.g. many nations teaching programmes in English), global perspectives and trade agreements and UK Borders Agency issues;

Issues around how best to deliver the curriculum including using appropriate technologies;

Recognition that students nowadays can choose educational providers internationally, and that increasing numbers pick and mix to match their requirements regardless of physical location;

The challenge of how to met students high expectations of what they will receive in return for their fees.

How can we make changes to enhance learning and teaching in universities?

External stimuli (e.g. NSS scores) can be powerful triggers for change but are not sufficient for really making a difference;

It’s more important (and effective) to look at enhancing the student experience than at how to massage NSS scores;

In my view, firm direction and dictat are much less effective than passion, persuasion and people-centred approaches;

Evidence-based practice helps to change better than instruction;

Working within university systems is essential if changes are to be long-lasting.

What students think about assessment

Student evaluations frequently reveal poor assessment practices that:

Lack authenticity and relevance to real world tasks; Make unreasonable demands on students; Are narrow in scope; Have little long-term benefit; Fail to reward genuine effort; Have unclear expectations and assessment criteria; Fail to provide adequate feedback to students; Rely heavily on factual recall rather than on higher-

order thinking and problem-solving skills. (Flint and Johnson, 2011, p.2)

Interpreting NSS scores

NSS scores are an unreliable indicator as they can be skewed by all kinds of extraneous factors, but do give us some hints on what we should be doing;

Attempts to massage NSS scores usually fail; It is much more sensible and worthwhile to

concentrate on improving the student experience of assessment and feedback, since these are so central to enhancing the student experience.

But what we can do is: Scrutinise NSS scores and free response

feedback to see where enhancements can be made;

Carefully consider the context and the student cohort experience;

Honestly review where improvements can be made;

Develop a strategic implementation plan that doesn’t just contain pious hopes, but specifies concrete action.

Setting changes in place

Marshall and Massy argue that when leading in turbulent times, the first step to be taken is to:

“create a sense of urgency about the crisis. While it’s easy to scare people, the aim is to at the same time present a plan about how, by doing g things differently, the university can break the momentum taking it in the wrong direction and work its way out of the problem. The key is to create a sense of urgency without instilling a feeling of hopelessness.” (p.68).

What do we want to change?

A focus on engagement; Being more consistent and coherent; Orientation towards teaching for learning; Supporting students inclusively; Using assessment to promote learning; Fostering robust quality measures; Maximising efficiency in curriculum design.

Engagement: why talk about it? Because:

Academics and learning support staff report increasing levels of disengagement by students of the ‘iGeneration’;

Potentially the nature of student behaviour in higher education is changing radically in terms of academic and other literacies;

Institutions need to ensure that new students enter with, or have the opportunity to acquire, the skills needed for academic success;

HEIs must devise programmes in which the emphasis is on maximising students' development.

Engagement of international students: some important considerations

Is recruitment undertaken to ensure students have the potential to succeed?

Is induction framed appropriately to welcome international students?

Are steps taken proactively to ensure international students have a good chance of integrating with their study cohorts?

Are we training our staff to be aware of diverse international approaches to HE learning and teaching, or are we just expecting students to get on with our systems?

Is the right kind of support offered (language, crisis support, befriending etc.?)

Consistency and coherence: mapping the student experience Will students feel from the outset that they are

on the programme they signed up to? Do students feel that they are immersed in the

subject they have signed up to study from the outset?

Is induction a valuable and productive introduction to the course (or just the distribution of endless information)?

Do students have a positive and balanced experience across the programme?

Are there points in the academic year when there doesn’t seem to be much going on?

Teaching for learning

Is there a coherent model of progression across programmes?

Are there clearly way-marked sources of student support throughout their studies?

Are students using critical thinking and high levels of analytical thought?

Are students working autonomously? Do students have opportunities of working

together?

What does high quality teaching look like? Our views:

Students graduate with good degrees; Relatively few students drop out; Student evaluations of teaching are good; External scrutineers, including Professional and

Subject Bodies, are comfortable with the standard of work achieved by students;

Students are employable at the end of the learning process, and fit-to-practise where appropriate;

Teachers find the workload manageable, and gain satisfaction from their work.

What does high quality learning look like? My views:

Students are able to progress from high levels of support to high levels of independence in their approaches to learning;

Students develop a good tool kit of appropriate skills for learning, including information literacy;

Students learn flexibly, have high levels of self-efficacy and confidence;

Teachers and students engage in meaningful learning dialogues.

Supporting students inclusively: we need to:

Adopt a holistic approach to the development of skills, particularly information literacy, so that this is fully integrated into the learning programme;

Enable students to become self-aware and reflexive learners who become robust in the face of problems;

Help students build resilience through ‘a diet of early successes’ and positive reinforcement.

Inclusivity: it’s helpful to ask from the outset: Is the content of learning materials excessively

or exclusively focussed on materials from a single context?

Can students recognise their own experiences in at least some of the examples used?

Are students with disabilities enabled to participate fully in group tasks and assignments?

Are reasonable adjustments for students with disabilities identified from the outset?

Active learning: we can:

Explore how we can best use the early stages of a programme to induct students into good study patterns and practices to enhance learning and improve engagement early on (Yorke, 2009);

Reconsider the kinds so activities students engage with the maximum ‘learning by doing’;

Rethink the way in which we use all learning opportunities to include activity as well as delivery;

Consider how we can best make use of technologies to support learning and engagement.

What can we do in the first six weeks of the first semester of the first year?

Enable students to feel part of a cohort rather than a number of a list;

Help students acclimatise to the new learning context in which they find themselves;

Familiarise them with the language and culture of the subject area they are studying (Northedge, 2003);

Foster the information literacy and other skills that students will need to succeed;

Guide them on where to go for specialist help as necessary.

We can help students understand what is required with reading

Help them to understand that there are different kinds of approaches needed for reading depending on whether they are reading for pleasure, for information, for understanding or reading around a topic;

Help them to become active readers with a pen and Post-its in hand, rather than passive readers, fitting the task in alongside television and other noisy distractions;

Give them clear guidance in the early stages about how much they need to read and what kinds of materials they need to focus on.

We can help students write at the appropriate level: we can:

Devote energy to helping students understand what is required of them in terms of writing by showing them good examples;

Work with them to understand the various academic discourses that are employed within the subject/institution;

Help them to understand when writing needs to be personal and based on individual experience, such as in a reflective log, and when it needs to be formal and using academic conventions like passive voice and third person, as in written reports and essays.

Using assessment for learning

Assessment that is meaningful to students can provide them with a framework for activity;

“Students can escape bad teaching but they can’t escape bad assessment” (Boud);

Where assessment is fully part of the learning process and integrated within it, the act of being assessed can help students make sense of their learning;

Assessment should be formative, informative, developmental and remediable.

Five things HEIs can do to improve NSS scores on assessment and feedback

1. Clarify what students can expect in terms of feedback and stick to what you say;

2. Speed up the turn around of assessed work so that students have time to learn from it before they complete the next assignment;

3. Focus on developmental feedback which concentrates on helping students know what to do to improve;

4. Ensure the language of feedback comments is understandable, helpful and appropriate;

5. Find ways to ensure that students use the feedback they receive.

Boud et al 2010: ‘Assessment 2020’Assessment has most effect when...:1. It is used to engage students in learning that is

productive.2. Feedback is used to actively improve student learning.3. Students and teachers become responsible partners in

learning and assessment.4. Students are inducted into the assessment practices and

cultures of higher education.5. Assessment for learning is placed at the centre of

subject and program design.6. Assessment for learning is a focus for staff and

institutional development.7. Assessment provides inclusive and trustworthy

representation of student achievement.

Mapping assessment

Are tasks aligned to the learning outcomes? Is assessments part of the learning programme,

or is everything ‘sudden death’ end-point? Is there excessive bunching of the assessment

workload that is highly stressful for students and unmanageable for staff?

Are there plenty of opportunities for formative assessment, especially for students struggling to gauge the level of study?

Are students over-assessed? Are your assignments uninspiring / tame /

excessively traditional?

Making the most of formative and summative assessment

Are summative assessments undertaken throughout the course, or is everything ‘sudden death’ end-point?

Is there excessive bunching of assignments in different modules that is highly stressful for students and unmanageable staff?

Are there plenty of opportunities for formative assessment, especially early on?

Are students over-assessed? When you have introduced innovative

assignments, have they been as well as, or instead of, existing ones?

Making the most of feedback

Plan to maximise the impact of formative feedback. Release extra time helping students to understand the importance of feedback, and the value of spending some of their time after receiving work back, to learn from the experience.

Provide opportunities for students to respond to our feedback, for example, by giving students follow-up task or give them ‘feed-forward’ comments to improve their next piece of work.

Think about the means by which we deliver feedback, since this can be vital in determining how much notice students take of what you say.

Supporting those at risk of failure & falling short of their potential. We can

Monitor achievement regularly: Monitor student attendance/ engagement and take

action when students disappear and particularly when work is not handed in;

Notice and challenge students when their outputs don’t match their potential as judged at entry;

Personalise the learning experience as far as we can;

Provide extension tasks for coasting students.

Using student feedback to trigger enhancements Regularly and frequently review all student feedback

and NSS results; Use a reality and viability check; Prioritise issues and act on the really important

ones fast; Look for quick hits and rapid solutions where you

can; Make sure students, all colleagues, managers, QA

staff and external examiners know what actions have been taken for enhancement. (“You said, we did”)

Making effective use of student reps Reconsider the number of reps you have to improve

representativeness; Train student reps on how to present views strongly

and appropriately; Consider offering incentives for course reps (e.g.

academic skills credit) (Some HEIs pay); Don’t just use them for annual review but engage

with them regularly e.g. through open forums; Involve them in identifying outstanding staff.

Ensuring consistency of student experience across programmes.

We need to be consistent in: Marking (obviously) and the way we treat marks

when aggregating them at exam boards; The ways in which we offer extensions and accept

mitigating circumstances; The treatment of episodes of plagiarism and

cheating (which to me suggests limiting tutors’ individual discretion);

We also need to be consistent about: The help we offer to students seeking placements

and internship; The support we offer in identifying and locating

reference material; The amount of support we give individuals (which

for me suggests fewer signed-up for individual appointments and more mandatory group problem-addressing sessions).

Fostering robust quality measures: I argue for:

Rapid turnaround of assignments with detailed and useful feedback;

Proactive and positive initial training for teaching staff and ongoing CPD;

Peer Observation of various kinds; Teaching based on a supportive/ reflective model; Clear and widely publicised mutual expectations for

students and staff.

And also

Recognising and rewarding good teaching and learning support, and having obvious career pathways for those who dedicate their lives to enhancing the student experience;

Taking student feedback very seriously, and publicising widely action take as a result of feedback.

Maximising efficiency in curriculum design

Wise HEIs carefully model the costs of curriculum provision to ensure the best possible offer to students;

Reusable Learning Objects of all kinds build flexibility into what we can offer;

Open Educational Resources throughout higher education are increasingly being used (see particularly JISC projects e.g. Unicycle at Leeds Met).

Conclusions The changes we make to improve the student

experience need to be strategic and evidence-based;

It is possible to make significant improvements to promote high quality learning, but it needs ownership by staff at every level;

Strategic approaches aren’t worth a fig if individual staff don’t embrace the need to improve things;

Doing the same things we have always done in the same way we have always done them is doomed to failure.

References and wider reading 1 Assessment Reform Group (1999) Assessment for Learning : Beyond the black box

Cambridge UK: University of Cambridge School of Education Biggs, J. (2003) Teaching for Quality Learning at University, Maidenhead: SRHE &

Open University Press.Boud, D. (1995) Enhancing learning through self-assessment, London: Routledge.Bowl, M. (2003) Non-traditional entrants to higher education ‘they talk about people

like me’ Stoke on Trent, UK: Trentham Books.Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in

Higher Education, London: Routledge.Brown, S., Rust, C. and Gibbs, G. (1994) Strategies for Diversifying Assessment,

Oxford: Oxford Centre for Staff Development. Brown, S. and Glasner, A. (eds.) (1999) Assessment Matters in Higher Education,

Choosing and Using Diverse Approaches, Buckingham: Open University Press.Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London:

Kogan Page.Brown, S. and Denton, S. (2010) Leading the University Beyond Bureaucracy in A

practical guide to University and College management (Eds. Denton, S and Brown, S) New York and London: Routledge.

Brown, S. (2011) Bringing about positive change in higher education; a case study Quality Assurance in Education Vol 19 No 3 pp.195-207.

References and wider reading 2Carroll, J. and Ryan, J. (2005) Teaching International students: improving learning for

all, London: Routledge SEDA series.Dweck, C. (2000) Self-theories: their role in motivation, personality, and development

Philadelphia: Psychology press: Essays in Social Psychology Taylor and Francis.Falchikov, N. (2004) Improving Assessment through Student Involvement: Practical

Solutions for Aiding Learning in Higher and Further Education, London: Routledge.Flint, N.R. and Johnson, B. (2011) Towards fairer university assessment: recognising the

concerns of students, London: Routledge.Gibbs, G. (2010) Using assessment to support student learning, Leeds: Leeds

Metropolitan University.Kneale, P. E. (1997) The rise of the "strategic student": how can we adapt to cope? in

Armstrong, S., Thompson, G. and Brown, S. (eds) Facing up to Radical Changes in Universities and Colleges, 119-139, London: Kogan Page.

Knight, P. and Yorke, M. (2003) Assessment, learning and employability, Maidenhead, UK: SRHE/Open University Press.

Marshall, P. and Massy, W. (2010) ‘Managing in turbulent times’ in Forum for the Future of Higher Education, papers from the 2009 Aspen symposium Massachusetts Institute of Technology Cambridge USA.

References and wider reading 3Morgan, M. (2012,) Improving and Enhancing the Student Experience- A practical

guide, London: Routledge. Newton, J. (2003) Implementing an Institution-wide learning and Teaching strategy: lessons in managing change Studies in Higher Education Vol 28 No 4

Nicol, D. J. and Macfarlane-Dick, D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education.

Northedge, A. (2003) Enabling participation in academic discourse Teaching in Higher Education, Vol. 8, No. 2, 2003, pp. 169–180.

PASS project: Bradford http://www.pass.brad.ac.uk/ Peelo, M. and Wareham, T. (eds) (2002) Failing Students in higher education

Buckingham, UK, SRHE/Open University Press.Pickford, R. and Brown, S. (2006) Assessing skills and practice London: Routledge.Race, P. (2001) A Briefing on Self, Peer & Group Assessment in LTSN Generic Centre

Assessment Series No 9 LTSN York. Race, P. (2006) The lecturer’s toolkit (3rd edition) London: Routledge.Race, P. et al (2009) Using peer observation to enhance teaching, Leeds: Leeds

Metropolitan Press.Race, P. and Pickford, R. (2007) Making Teaching work: Teaching smarter in post-

compulsory education, London: Sage.

References and wider reading 4

Renfro, W. L. and Morrison, J. L. (1983) ‘Anticipating and managing change in educational organisations’, Educational Leadership Association of Supervision and Curriculum Development.

Robertson, C., Robins, A. and Cox, R. (2009) Co-constructing an academic community ethos- challenging culture and managing change in higher education: a case study undertaken over two years, in Management in Education Vol. 23 Issue 1.

Rust, C., Price, M. and O’Donovan, B. (2003) Improving students’ learning by developing their understanding of assessment criteria and processes, Assessment and Evaluation in Higher Education. 28 (2), 147-164.

Ryan, J. (2000) A Guide to Teaching International Students Oxford: Oxford Centre for Staff and Learning Development.

Sadler, D. R. (1998) Formative assessment: revisiting the territory Assessment in Education: Principles, Policy and Practice 5, 77-8.

Wisker, G. and Constable, J. (2005) Fellowship and Communities of Practice, SEDA, Anglia Ruskin University UK.

Yorke, M. (1999) Leaving Early: Undergraduate Non-Completion in Higher Education, London: Taylor and Francis.

Yorke, M. and Longden, B. (2004) Retention and Student Success in Higher Education, Maidenhead: Open University Press.