Eval Framework For Presentation

Evaluation Framework

Don’t Panic!

• Why do we need it?

• How was the Sing Up Evaluation

Framework developed?

• How are we going to use it?

Why do we need it?

To provide: • Sing Up with consistent and organised

evidence to tell the story of Sing Up and capture the learning generated by the campaign

• A system to summarise existing evaluations and carry out evaluation of new Sing Up projects and activities

How was the Framework developed?

• A Sing Up Evaluation Working Group included representatives from Funded Programmes, Workforce Development, Beyond the Mainstream, The Institute of Education, Music Services & Sing Up Advisors

• The group held a series of meetings to look at: Evaluation across the Sing Up programmeHow to collate different methods of

evaluation being used across Sing Up

How are we going to use it?

Area Leaders and Project Managers of all Sing Up strands will use the Evaluation Framework to:

Evaluate new Sing Up activities and resources

Identify whether we are collecting relevant evidence

Sing Up Evaluation Framework

We have identified:

Six areas of evaluation

The people involved

Sing Up Evaluation Framework

We have created:

An Evaluation Framework

A method for using the Framework

• The Evaluation Framework is a flexible system that can be used to gather a full report on all activity.

• The pyramid shows this using Workforce Development as an example.

The four Sing Up Strands (WFD,

Campaign, Funded

Programmes and Resources

Each of the Workforce Development programmes

.(e.g. Vocal Force, BTM)

A longer project that might include the one-off session (e.g. Area

Leader activity)

One workshop or session, either as a one-off, or as part of a longer term project

Evaluating Sing UpSingUp

Programme

six areas of evaluation

The Evaluation Working Group has identified six areas of evidence to collect from all parts of the Sing Up programme:

ContextEngagement, Enjoyment and SatisfactionLeadership and Learning: KnowledgeLeadership and Learning: Impact Internal ImpactExternal Impact

Context

Engagement,

Enjoyment, Satisfaction


• Who

• What

• Where

• When

• The experience of the activity/resource for

individuals

• How they felt during and after the

activity/resource

• What they remember

• What they feel worked well and less well

Leadership and

Learning Knowledg

e

Leadership and

Learning Impact


• What skills and competencies are acquired

• How were they learnt

• What further knowledge needs to be acquired

• How will new skills be used

•How learning has affected them

•Practical examples

Internal Impact

External Impact


• Impact of the activity on the school

/organisation / group

• Identify areas of future development

• Impact of the activity on the, wider

community and the wider sector

• Development of new partnerships

• Identify areas of future development

People Involved• Lead Organisations or Project Managers

• Adults Audience Participants Trainers

• Children and young people Audience Participants Young singing leaders

•Organisations overseeing programmes/projects

•Individuals who project manage

Lead Org. & Project Managers

•Who have attended non training activity

•Who have participated in training activity

•Who have delivered to Adults, Young Singing Leaders and Young People

Adults

•Who have attended a performance event

•Who have been involved in direct delivery lead by an external Adult Teacher or Internal Teacher and Young Singing Leader

•Young Singing Leaders

Children & Young People

People Involved

Is it clear so far?

virtual evaluation grid

• We constructed a grid using the six areas of evaluation and the people involved, showing how evidence for each activity/resource can be brought together

• The virtual evaluation grid is a visual aid

• The grid should not to be used as a template

virtual evaluation gridareas

People↓

Context Engagement, Enjoyment

and Satisfaction

Leadership and

Learning knowledge

Leadership and

Learning impact

Internal Impact on workplace

, colleague

s

External Impact

on others,

community

Lead Orgs. & Project

Managers

Adults asTrainers

Adults as Participants

Children & Young People as Singing Leaders

Children & Young People as Participants &

Audience

How the grid works

• There is a different table for each group of people involved

• In each table there are questions under the headings of the six areas of evaluation

LEAD ORGS. & PROJECT MANAGE

RS

Context Engagement, Enjoyment and

Satisfaction

Leadership and Learning knowledge

Leadership and Learning

impact

Internal Impact on workplace,

colleagues etc

External Impact on

others, community etc

What is your role and responsibilities?

What are the aims of the project?

Who are the partners in your project?

When did your project take place?

Where did the project take place?

Which SU strand?

Type of project/resource (one off event etc.)?

What areas of organising this project/ resource did you enjoy most?

…and least?

In what ways have you felt supported and engaged with Sing Up?

In what ways have you not felt supported and engaged with Sing Up?

What have

you learnt through delivering on this project

What worked well?

Where did you struggle?

I understand the qualities of good Vocal Leadership.

The following qualities of good Vocal Leadership were demonstrated in the activity.

What are you

inspired to do differently as a result of the activity/ resource?

How will this project inform you/ your organisations current and future work?

Have any new partnerships and relationships in the immediate working environment been developed as a result of the activity/ resource?

What would you like to see happen next outside of your own organisation as a result of this project/ resource?


ADULTS AS

TRAINERS

Suitable for evaluating one

activity, but also for

evaluating their experience of

working across the programme


Satisfaction



impact


colleagues etc

External Impact on


When did the singing activity take place?

Where did the singing activity take place?

Who led the singing activity?

How long did the singing activity last?

What were the aims of the activity?

Who were the participants?

What did you enjoy most about leading this activity?

What did you enjoy least about leading this activity?

In what ways did you feel supported by the lead org/ project manager?

In what ways did you not feel supported by the lead org/ project manager?

What have

you learnt through delivering on this project?

What worked well?




What are you inspired to do differently as a result of the activity/ resource?



ADULTS AS

PARTICIPANTS


Satisfaction


knowledge

Leadership and Learning impact

Internal Impact on workplace, colleagues etc

External Impact on


How did you hear about the activity/ resource?

What was the venue like?

Where did you come from?

What is your job title?

Are you involved in other Sing Up projects?

If so, which one(s)?

What did you like about the activity?

What are your personal impressions about the activity?



What stood out in the activity?

What was the turning point for you in the activity?

What helped you learn?

What did you learn in the activity?

What will you use from what you have learnt?

To what extent has this activity been beneficial to you?

To what extent will your participation in this activity be beneficial to others?

Are you singing more often as result of the activity/ resource?

I would attend future similar activities.

I will share the skills & knowledge gained through the activity.



What are you inspired to do differently as a result of the activity/ resource?

CHILDREN AND YOUNG PEOPLE

AS SINGING LEADERS


Satisfaction



impact


colleagues etc

External Impact on


How did you hear about the workshops/ session?

What was the venue like?

Where did you come from?

How old are you?

If you attend a school/college which one is it?

If you are taking any qualifications which ones?


How does leading make you feel?

How do you get to lead singing?

What have you learnt through leading the singing activities?

What worked well?



When will you

use the skills you’ve learnt again?

How has leading the singing been useful to you?

I am a good role model

How will you share your new skills with others?

Who will you share your new skills with?

What will you tell other people about what you’ve learned?

Have you made any new friends through your new singing activities?

Where else do you lead singing?

CHILDREN AND YOUNG PEOPLE

AS PARTICIPAN

TS


Satisfaction



impact


colleagues etc

External Impact on


How did you hear about the activity/ resource?

How old are you?


And if so, which ones?

How does singing make you feel?

Was the singing fun?

Would you choose to do singing again?

What did you learn?

What can you remember?

What was the best part of the singing activity? And why?


How has this activity/ resource helped you?

I sing more often.

How much singing do you do in school?

Does your teacher enjoy singing?

Do you sing everyday at school?

Where else do you sing?

Who else do you sing with?

How do we work with this?

• If you have an existing evaluation process in place: Keep calm: our systems can work together Many of you are doing this already

• If you do not have an existing evaluation process This will provide you with one

• We will provide you with Evaluation Forms. Young Singing Leaders Adults as Trainers Adults as Participants

Evaluation Form

Evaluation Form

Evaluation Form

Next Steps for Evaluation Working Group

• Evaluation Working Group meeting to agree final implementation across all Sing Up strands

• To be implemented at the beginning of Phase 4

Questions?