The Northern Collaboration15 September 2016

Head StartLibrary-led innovation and collaboration

Nadine SunderlandAcademic Services and Retention Manager

Library and Student Services

Session outline

1. The Head Start programme

2. Library-led collaboration – timeline and success factors

3. Models of evaluation and impact measures

4. Lessons learned

5. Q&A

Head Start – bridging the gapPurpose and content• Insight into HE study and an opportunity to refresh/develop academic skills • 4 core elements: learning skills for HE, information skills, academic writing

and referencing• Delivered pre-entry via the university’s VLE

Head Start iterations• UG Generic - UG Health – UG Education – UG Psychology – all on VLE• PGCE Education – on VLE• L5/L6 Head Start Plus – Free MOOC• M level Head Start – Preparing for Postgraduate Study – Free MOOC

Head Start – bridging the gapTargeted groups• All full time UG applicants and WP students more specifically• All PGCE and School Direct applicants• Direct-entry and top-up applicants• Postgraduate applicants

Delivery• Online via VLE or MOOC (Open Education platform)• Pre-entry between May and September for UG and PGCE applicants• Non-credit bearing• Self-learning course• Monitored by Library and Academic Advisers

Library-led collaboration - timeline2011- Originated from

business module – campus delivery

Extensive input from Library

advisers

2012 - Library-led project group to scope scalability

Development of online modules by LAAs using

Articulate software

Professional services and academic

departments buy-in and processes

2013- First promotion of Head Start to all FT UG applicants (670)

Annual reporting to Student Engagement

and Retention Committee

New iterations developed in 2014

and 2015

THELMA Award in 2014

1015 students in summer 2015

Over 1000 on MOOCs

New iterations planned

including Law

Collaboration - success factorsSuccess factors

1 Evidence from initial campus-based programme

2 Commitment of a number of Library and Academic Advisers – content development

3 Steer and leadership from Learning Enhancement Managers – Library and Academic Skills

4 Partnership work with other services – Admissions, IT Services and Learning Technologists

5 Selling the benefits to obtain engagement of academic staff

6 Support from senior managers

7 THELMA Award and “snow ball” effect

Programme evaluation

Monitoring

Measures of activitye.g. no of studentse.g. completion rates

Ev

aluation and I

mpa

ct

Measures of performancee.g. impact on levels of confidence and learning

Val

ue

Measures of worthe.g. value to learnerse.g. value for money

The Intervention Logic Model (ILM)

No of eventsNo of participants

Short & long term effectsLearning gained, attainment, graduate employment.

Intended or non-intendedIntervention Logic Model – developed by Victoria Goodwin – UOC, 2013

Kirkpatrick’s evaluation model

Image from Kirkpatrick Partners available at http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel Accessed 18 May 2016

Kirkpatrick model – Applied to Head Start(HS)

Level What is measured? How1. Reaction • Do students engage with the

programme?• Do WP students engage with the

programme?• How do they feel about their

experience?

• Initial HS completion survey – e.g. satisfaction rates

• Engagement rates• WP profile of HS students• Access to HS resources during 1st semester

2. Learning • Do students feel more confident about HE study?

• Does it benefit their learning on their main programme of study?

• Initial HS completion survey• Follow-up survey

3. Behaviour • How far has learning been applied?• Has is resulted in personal change?

• Initial HS completion survey• Follow-up survey• No of students who revisit HS learning

resources• Observations from advisers and tutors

4. Results • How did the programme impact on retention?

• Comparison of retention rates between HS and non-HS students

Models in action – Head Start programme (2015)

Context Input Output Outcomes and impact

60% WP intake at UoC

Lack of preparedness for academic study

Lack of confidence in navigating HE study

350 hrs content development hours

150 hrs student monitoring each year

50 hrs admin hours e.g. communication and registration

1015 UG applicants registered on Head Start

Increase of 19% on 2014

53% WP students (LPN, Disability, BME)

50% complete the whole programme

70% complete core elements (academic writing and referencing)

93% would recommend the programme to others

94% feel more confident about starting at university

One semester on, 88% say that HS helped them manage their course assessments

66% indicate that HS gave them an advantage compared to their peers who didn’t complete it

43% indicate that HS helped them remain on course

47% continued to access HS resources during the 1st semester

Higher retention of HS students versus non-HS – e.g. Nursing (+3%), Forestry and Conservation (+6.9%)

Lessons learned• Library and Academic Adviser blended role

• Find your champions

• Start small and build up

• Design the evaluation activity alongside delivery

• Use evidence to build business case for expansion

• Build staff capacity for development, monitoring and evaluation activity – e.g. OFFA

• Greatest difficulties with adapting Admissions and IT processes for pre-entry registration and IT log on

Any questions?

BibliographyGoodwin, V.(2013) Evaluation Report: The North West Graduate Employability Support Project.Available at: http://www.cumbria.ac.uk/Public/Enterprise/Documents/GraduateEmployabilitySupportProgramme/GraduateEmployabilitySupportProjectEvaluationFullReport.pdf (Accessed: 23 /05/2016)

Kirkpatrick, D. L. (1994) Evaluating Training Programmes ,San Francisco: Berrett Koehler Publishers.

The Kellogg Foundation Logic Model Development Guide (2006).Available at: https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide (Accessed: 23/05/2016)

The Kirkpatrick Model (2016). Available at: http://www.kirkpatrickpartners.com/OurPhilosophy/TheKirkpatrickModel (Accessed: 18/05/2016)

Value & Impact Toolkit (2011), AMOSSHE.Available at: http://www.amosshe.org.uk/sites/default/files/u3152/Value_and_impact_toolkit_June_2011.pdf (Accessed: 20/06/2016)