Glancing back, looking forwards: Learning from an institutional approach to change in assessment practice and culture Helen Parkin and Graham Holden Quality

Glancing back, looking forwards:Learning from an institutional approach to change in assessment practice and culture

Helen Parkin and Graham HoldenQuality Enhancement and Student SuccessStudent and Learning Services

Session Outline

• Overview of the Assessment for Learning Initiative at Sheffield Hallam University

• Discuss the Change Model adopted by the Initiative

• Outline deliberate steps taken by the Initiative

• Describe the approach taken to evaluate the initiative

• Share the initial headlines from the evaluation

HEA 201022nd June2010

The Assessment for Learning Initiative• TALI was established in 2006 focusing on the deliberate

actions required to support the development of::

• Assessment practices that are learner focused and promote student engagement and attainment.

• Regulations that are clear, consistent and student centred.

• Assessment processes that are efficient and effective and which enable the delivery of a high quality experience for staff and students.


A Joined Up Approach

Regulations and

Frameworks

Academic

Practice

Process and Systems


Key features of the approach to change• Acknowledge that changing assessment practice is complex and time

consuming. • Responsibility for changing the assessment culture and practices lies with

different groups of stakeholders • Advantages in promoting institutional change through key personnel at

ground level, (Newton, 2003). • Importance of emergent change and the need for connectivity across the

institution (Seel, 2005). • Model for an integrated approach to assessment (adapted from Joughin

and Macdonald, 2009) providing a clear framework within which the complex nature of assessment could be understood enabling the formulation of a quality enhancement strategy to move the University towards assessment for learning.


Deliberate StepsStudent: Where assessment matters: student experience of assessment, student involvement in assessment processes …For example: Feedback for learning campaign in collaboration with students union and student representatives,

Module Level : Where assessment happens: module design, teachers’ experience of assessment….For example: Targeted resources (blue folder) and development opportunities (e.g. Blackboard Grade Centre) Course/programme Level : Supporting good practice: departmental culture, programme design, innovation and improvement, staff development….For example: staff development opportunities supported by Faculty Teaching Fellows acting as 'local authentic voices'


Deliberate StepsFaculty Level : the Faculty context: Faculty culture, strategies and action plans, resources, support, funding For example: Staff development opportunities and faculty funded projects and initiatives that align to University priorities.

Institutional Level: the institutional context for good practice: principles, policies and regulations, resources/funding, recognition and reward (for good teaching, including assessment) For example: Assessment and Feedback Policy and guidelines for the module and course design, Changes to module template and requirements re course approval

Wider Level: the external context: reflecting political, economic and social forces impacting on HE, as well as initiatives from within the HE sector itself.For example: HEA funded research project - Technology, Feedback Action!


Key Changes to Regulations

• Calculation of Degree ClassificationIntroduction of an element of compensation at final level (taking best 100 at level 5 and best 100 at level 6 / or just best 100 at level 6)

• Passing a ModuleStudents must achieve an overall mark of 40 or above to pass a module

• Limited Re-registrations

A student can re-register on a module on only one occasion


Key Changes to Regulations

• Credit LoadingStudents are permitted to study mixed level to retrieve failure. Student progression will be determined by the maximum student loading regulations

• Retention and Attainment

Where in-module retrieval is available, a student may choose to take the assessment again if they achieve below 40% in the assessment task

• Acknowledging SuccessIntroduction of Pass, Merit, Distinction for Foundation Degrees and Postgraduate Courses and Distinction for Ordinary Degrees


Evaluating an institutional approach to change in assessment practice and culture

To evaluate the staff and student perceptions of the impact of the University's Assessment for Learning Initiative

Analyse the impact on key performance indicators such as:Good degree statisticsFirst time pass ratesStudent retention


Key Features of Implementation

• Academic Practice– Teaching Fellows

• Regulatory Change– Faculty Implementation Teams

• Systems and Processes– Cross disciplinary working

All initiated through two-day residentialfocussed on approach to change


Methodology

• An Appreciative Inquiry Approach

• Data mining exercise to explore existing data sets

• Workshop attended by 30 members of academic, administrative and managerial staff

• 40 semi-structured interviews


Academic Practice• Raised awareness of assessment for learning and feedback

across the University• Encouraged staff to 'rethink' assessment strategies and

provided an opportunity for module redesign• Opportunities to share best practice well received• Evidence that assessment now features in 'peer review' across

the University• Inconsistencies in practice remain• Implications for workload (or shift) particularly where in-

module retrieval has been adopted


Teaching Fellows

What worked well• 'Faculty face' rather than

'central body'• Seen as expert in

assessment matters• Engagement at subject

group level• Commitment and passion

for assessment and improving the student experience

Lessons Learned• Tensions between faculty

need and central need• Need for clearly defined

roles with expectations and accountability

• Need more time• Who makes the decisions?


Regulatory Change

• New regulations 'celebrate student success'– removal of 'hurdles'

• Acted as stimulus to encourage thought around assessment strategies

• Professional Body requirements added complexity• Implementation was inconsistent at faculty level with possible

consequences for the student experience of assessment• Greatest impact where implementation is consistent and

widespread


Faculty Implementation Teams

What worked well• Consisted of appropriate

people to make decisions for the faculty

• Every module though revalidation or minor modification process

• Maintained a dialogue around assessment

• Ongoing commitment to assessment

Lessons Learned• Variations between

faculties in how teams worked

• Need to achieve consistency at course level

• Tracking of data is different for each faculty and often difficult to retrieve if outside the faculty


Systems and Processes• Highly flexible regulations and practice require sophisticated

and complex bespoke developments of corporate systems• A number of 'work-arounds' have been developed at Faculty

level– Double entry of grades has implications for workload and the

accuracy of data– The use of Blackboard Grade Centre as an 'administrative tool'

detracts from its pedagogic benefits• System is perceived as dictating practice• Work ongoing through 'Improving Systems and Processes'

Project


Key Performance IndicatorsGood Degree Statistics

Overall remained fairly consistent but students awarded a First has risen from 8.2% (2006/07) to 12% (2008/09) and those awarded a 2:1 has risen from 48% to 49.1%

First-time pass ratesDifficulties with obtaining accurate data but the number of

modules 'passed' has been sustained despite growth and limited re-registrations

Evidence of improved pass rate in modules that adopted 'Model A'

Significant improvements in the faculty where adoption was widespread and consistent

Retention and ProgressionWithdrawals down from 9.1% to 8.5%


Future PrioritiesPractice• Authentic assessment - preparing our students for an uncertain future (tie

in with graduate attributes)• Technology enhanced learning - e-PDPPolicy / Regulations• Student entitlement (feedback on all assessed work within 4 weeks;

feedback on exams)• Refresh regulations and guidance / update ancillary regulations

(extenuation etc)Process and Systems• Move towards all modules to providing access to grades online• University wide system for the management of module marks


ReferencesHEFCE (2009) 'National Student Survey'. http://www.unistats.com/

[last accessed 18 January 2010].

Macdonald, R. & Joughin, G. (2009) 'Changing Assessment in Higher Education: A model in support of institution wide improvement' In Joughin, G. (ed.) Assessment, Learning and Judgement in Higher Education. Springer. pp.193-214.

Newton, J. (2003) 'Implementing an institution-wide learning and teaching strategy: lessons in managing change' Studies in Higher Education. 28 (4) pp.427–441.

Seel, R. (2000) 'Culture and Complexity: New insights on Organisational Change' Culture and Complexity - Organisation and People. 7 (2) pp.2-9.


http://www.unistats.com/

Thank you – any questions?

[email protected]


mailto:[email protected]

Documents

Glancing back, looking forwards: Learning from an institutional approach to change in assessment practice and culture Helen Parkin and Graham Holden Quality