Embed Size (px)
Citation preview
Evaluating Capabilities: Building consensus around
capability measures and indicators
Jason PollardThink:Learn:Do
Background
• Integrative Development Model (Kia-Keating, 2009)
• Young Foundation Report
• Adoption by commissioners
• Challenge for service deliverers
• Opportunity for the sector
The challenges of evaluation
• Bringing capabilities into a single tool
• Developing a set of (valid) indictors
• Scoring a capability measure
• Getting the right data out of the service
Meeting the Challenge
• Local evaluation network
• Feasibility study –
– Asked can operationally useful information be collected to support operational decision making
Evaluation Network
• Recruited six youth providers through VAL
• Jointly developed a set of indicators
• Pilot evaluation and pre-test of data collection tool
• Full evaluation to ‘road test’ tools
Lewisham – Capability Measures
1. Being Creative2. Communication3. Relationships and Leadership4. Resilience5. Confidence and Agency6. Managing Feelings7. Planning and Problem Solving8. Strengthening Citizenship9. Making the most of London
Delphi – Agreeing Definitions
Example – Being Creative
Content Category What we mean by....
Learning about how to express their feelings. in expressing emotional
life Using Creativity to express how I
feel
A process and a way of channelling emotions, confidence, insecurities and confidence
Skills [and] tools to illustrate emotions
Music and poetry as a tool to deliver a message to communicate with
others Discussing friendships
Development of communication
Delphi – Agreeing Measures
How to measure Being Creative
What we mean by ... How we will measure it ...
Using creativity to express how I feel
The young person has the ability to express themselves creatively
Exploring learning through creative activities
The young person has the ability to use his or her creativity to set and pursue learning goals
Creative thinking and practical problem solving
The young person can test and apply learning to new and existing problems they encounter
Delphi – Agreeing Indicators
What would we expect to see?
Creativity
The young person can express themselves in a range of activities
The young person can articulate what they mean, and use collaborative activities to communicate this to others
The young person can identify and set personal learning goals
The young person is able to challenge their view of the world, and look for new solutions to problems they face
The young person takes the lead in finding solutions to problems
The young person is able to adapt their plans on taking advice
The Indicator Set
• 27 indicators
• Equally split over 9 outcomes
• What we would expect to see
• Conceptually linked to outcomes
Developing the Data Collection Tool
• A ‘composite’ tool – brings together several capability scales
• Valid and reliable (in pre-test)
• Soft Data – questions relating to the development of the young person and their engagement with their project
• Hard Data – age-specific questions directly relating to the capabilities
Next Steps
• Pilot - Scale-up: 10+ organisations to collect common data
• Explore Thematic indicators (talent; financial capability, etc)
• Establish evaluation communities of interest (Co-ops; CiCs)
