Developing adult social workeffectiveness evaluation

THL, FinSoc-team 2

Goal-oriented adult social work as a target for evaluation

THL, FinSoc-team 33

The characteristics of goal-oriented adult social work• The goal of social work • The object of social work • Social work methods • Contextual and situational factors • Mechanisms

THL, FinSoc-team 4

Adult social-work effectiveness evaluation project

THL, FinSoc-team 5

The EEA-project (2011-2012)

Coordination:The National Institute for Health and Welfare

Seinäjoki: Developing an effectiveness evaluation measure for adult social

work

The Centre Of Excellence On

Social Welfare In The Ostrobothnian

Area (SONet Botnia)

Helsinki: Focusing on data from client

monitoring forms at West Helsinki Social Centre

Tuusula:

Evaluating rehabilitative social work

THL, FinSoc-team 6

Design: single-case evaluation

THL, FinSoc-team 7

• Single-case design allows social workers to evaluate how well the social work’s goals have been reached, what methods and procedures have been used, and how important different contextual and situational factors and mechanisms are for the goal attainment

• The method was originally developed as a practical tool for social workers rather than as a research method

• Single-case evaluation also seems to be suitable because it follows the basic idea of empirical research, but without a control group

• It is possible to gain information about single clients’ goals but also to obtain quantitative data. It is also possible to track the client’s situation using repeated measurements.

THL, FinSoc-team 8

Example of how the goals could be evaluated ordinally

Goal number 5: Subsistence or financial situation management

Question: Managing the subsistence or financial situation. Choose one of the following options:

Options:

• The goal is to improve the situation

• The goal is that situation remains unchanged

• The item is not a goal

THL, FinSoc-team 9

A framework for the measure: realist evaluation

THL, FinSoc-team 10

• Generative mechanisms• Generative causality

THL, FinSoc-team 11

Developing the KEY-measure for adult social-work effectiveness

evaluation

• Questions about the goals, methods and procedures of social work and about the client’s situation.

• Testing period: November 2011 – August 2012 (10 months)

• Single-case design: Evaluation phase I & Evaluation phase II

• Evaluation I: 209 client-cases

• Evaluation II: 172 cases

THL, FinSoc-team 12

THL, FinSoc-team 13

Basic information form

Reasons why evaluation is not

done

THL, FinSoc-team 14

Results from adult social-work effectiveness evaluation

15THL, FinSoc-team

Concern about the close personal relationshipsLeisure activities and opportunities for participation

The consequences of anti-social behaviourPhysical problems caused by illness

Child's needsSubstance abuse and consequences

Problems related to mental illnessess Rental housing situation in the area

Substance useSocial skills

Client's relationshipsMental health

Development of service systemProceeding with further education

Client's possibility to access housingInclusion and participation (eg. hobbies)

Self-esteemAwareness of the problems due to the society

Getting in to further educationFluency of everyday life

Life managementAccess to housing; keeping itClearing up the debt situation

Promoting the employment situation (for long-term unemployed)Supporting client's occupational capabilities

Supporting client's problem-solving capabilitiesService management for a clientEmployment, searching for a job

Client's control over his/her economic situation

0 10 20 30 40 50 60 70 80 90

Women (%) (N=110)

Goals set by clients; stratified by gender (%)

Kivipelto, Blomgren & Suojanen 2013, 43

THL, FinSoc-team 16

Goal reached admirably; percentage of levels of achieving goals (%)

Kivipelto, Blomgren & Suojanen 2013, 44

Client's possibility to access housing

Rental housing situation in the area

Leisure activities and opportunities for participation

Physical problems caused by illness

Promoting the employment situation (for long-term unemployment-people)

The consequences of anti-social behaviour

Concern about the close personal relationships

Mental health

Clearing up the debt situation

Client's relationships

Problems related to mental illness

Awareness of problems due to the society

Employment situation, searching for a job

Inclusion and participation (eg. hobbies)

Development of service system

Client's control over his/her economical situation

Life management

Proceeding with further education

Social skills

Getting into further education

Substance use

Substance abuse and consequences

Supporting client's occupational capabilities

Self-esteem

Fluency of everyday life

Access to housing; keeping it

Supporting client's problem-solving capabilities

Child's needs

Service management for a client

0 10 20 30 40 50 60

05

182121

232424

2526

282930

3333

353636

383939

41414242

454949

53

Situation improved

THL, FinSoc-team 17

Controlling the client

Supporting client's participation

Supporting client's awareness of social problems

Case management

Dealing with the client's problems

Service plan for the client

Solution focused work

Supportive discussion

Supervision and guidance

Needs assessment

Decision making

Social assistance

0 10 20 30 40 50 60 70 80 90 100

Women (%) Men (%)

Most used social work methods. Percentage of all methods; stratified by gender (%)

Kivipelto, Blomgren & Suojanen 2013, 44

THL, FinSoc-team 18

Values and attitudes in the living areaAccess to the environmentClient's cultural background

The demographic structure of living areaSubstance abuse of family members

Concern about the close peopleSubstance abuse / addictions

Client's mental healthMental health and well-being

Client's employment situationSatisfaction of relationships/adequacy

Client's lifestyleClient's economic situation

Client's physical healthPossibility to access health related technology

Awareness of the bacground determinants behind the problemsPossibility to influence

Leisure activities and opportunities for participationClient's ability/motive to plan his/her economic situation

Client's occupationSeeing solutions and alternatives

Employment and occupational situationsInternet connectivityClient's self-esteem

Living conditionsClient's motivation

Possibilities to be heardService provision / availability of services

Client's attitudes towards the services

0 10 20 30 40 50 60 70

Men (N=99)

Women (N=110)

Kivipelto, Blomgren & Suojanen 2013, 46

Factors contributing to reaching goals. Stratified by gender (%)

THL, FinSoc-team 19

Supportive methods were not used * Used 1-2 supportive methods)* Used 3-4 supportive methods*

25.8

10

47.8

13.8

27.3

43.8

Men (%) Women (%)

Effectiveness of supportive methods to the client's problem-solving capabilities. Comparison of men and women %

Kivipelto, Blomgren & Suojanen 2013, 50

THL, FinSoc-team 20

Single-case evaluation fits well for evaluating social-work with

adults

THL, FinSoc-team 21

Advantages

• It was seen as important to integrate systematic inquiry within adult social work

• The evaluation measure development was also an attempt to make the data collection easier and more systematic

THL, FinSoc-team 22

Difficulties

• Social workers found it difficult to incorporate the use of the measure into practical social work, even though the information required for the measure was largely the same information they needed in client work

• Single-case evaluation is best suited for those situations that require long-term social work

• Many clients just “disappeared” and the second phase of evaluation could not be undertaken

• More work is still needed to develop a programme theory that works well

THL, FinSoc-team 23

Towards evidence-based social work

THL, FinSoc-team 24

Processing the data into evidence-based knowledge

THL, FinSoc-team 25

References

• Blomgren S, Kivipelto M (2012) Valtaistus. Aikuissosiaalityön valtakunnallinen kartoitus [National Survey of Adult Social Work], Report 27, Helsinki: National Institute for Health and Welfare (THL).

• Bloom M, Fischer J, Orme J G (2009) Evaluating practice: Guidelines for the accountable professional, 6th edition, Boston: Allyn & Bacon.

• Cohen B J (2011) Design-based practice: A new perspective for social work, Social Work, 56(4): 337–346.

• Fischer J, Corcoran K (2007) Measures for Clinical Practice and Research. A sourcebook. Volume 1. Couples, Families and Children, Fourth Edition, New York: Oxford University Press.

• Gray M, Plath D, Webb S (2009) Evidence-based social work. A critical stance, London: Routledge.

• Howe D (2009) A brief introduction to social work theory, Basingstoke: Palgrave Macmillan.

• Juhila K (2008) ’Aikuisten parissa tehtävän sosiaalityön yhteiskunnallinen paikka’ [Social Locus of Social Work With Adults], in A Jokinen and K Juhila (eds), Sosiaalityö aikuisten parissa [Social Work With Adults], Tampere: Vastapaino: 48–81.

THL, FinSoc-team 26

• Julnes G, Mark M M (1998) ‘Evaluation as sensemaking: Knowledge construction in a realist world’, in G T Henry, G Julnes and M M Mark (eds) Realist Evaluation: An Emerging Theory In Support Of Practice: New Directions For Evaluation 1998(78): 33–52.

• Kazi M, Wilson J (1996) ‘Applying single-case evaluation in social work’, British Journal of Social Work 26(5): 699–717.

• Kivipelto M, Blomgren S, Suojanen R (2013)’ AVAIN-mittarin kehittäminen Seinäjoen sosiaalivirastossa’ [Developing the KEY-measure in Seinäjoki social security office]. In M Kivipelto, S Blomgren, P Karjalainen and P Saikkonen. Vaikuttavaa aikuissosiaalityötä – arviointimalleista mittareihin. [Effective adult social work – from evaluation models to evaluation measures] Research and development project; final report. Report 8, Helsinki: National Institute for Health and Welfare (THL).

• Kivipelto M, Blomgren S (2012) ‘Social work with adults as a tool for tackling exclusion’, in S Karvonen, I Keskimäki, M Kuronen and K Wilskman (eds) Annual review, Helsinki: National Institute for Health and Welfare (THL), 21–23.

• Pawson R, Manzano-Santaella A (2012) ‘A realist diagnostic workshop’. Evaluation 8(2), 176–191.

• Pawson R, Tilley N (1997) Realistic Evaluation, London: Sage.• Payne M (2005) Modern Social Work Theory, 3rd edition, Basingstoke: Palgrave

Macmillan.• Pedersen L, Rieper O (2008) ‘Is realist evaluation a realistic approach for complex

reforms?’ Evaluation 14(3), 271–293.• Wong S E (2010) ’Single-case evaluation designs for practitioners’, Journal of Social

Service Research, 36(3): 248–259.•