Slide 1

FACULTY OF HEALTH SCIENCES CENTRE FOR DISABILITY RESEARCH AND POLICY INTERNATIONAL DEVELOPMENTS IN MEASURING OUTCOMES Roger J Stancliffe Eric Emerson 1

Slide 2

Monitoring progress in achieving disability equality: Fulfilling Potential

Slide 3

F ULFILLING P OTENTIAL

Slide 4

R EPORTS http://odi.dwp.gov.uk/fulfilling-potential/index.php

Slide 5

D OMAINS & I NDICATORS 11 domains containing -1 or 2 headline indicators -up to 11 supporting indicators 5

Slide 6

K EY M ESSAGE 1 Co-production is key to balancing the interests of -Government -DPOs -(People with disabilities)

Slide 7

K EY M ESSAGE 2 Monitor the inequality gap -over time -and for at risk groups

Slide 8

L EFT B EHIND Framework -v1 (2009) UN Convention on the Rights of Persons with Disabilities -v2 (2011 onwards) Australias Social Inclusion Indicators Framework Data -Annual survey of Household Income & Labour Dynamics in Australia (HILDA) -Indicators matched to 44% of Framework indicators 8 Monitoring Changes in the Wellbeing of Young Disabled Australians

Slide 9

L EFT B EHIND 2014 Between 2001 and 2012, the gap between young Australians with disabilities and their non-disabled peers has grown in 11 (of 22) areas including -Not being employed -Being long-term unemployed -Having low economic resources and financial stress -Having low subjective well-being -Not having someone to turn to in times of crisis -Not having a voice in the community -Experiencing entrenched multiple disadvantage. It has not narrowed in any area at all

Slide 10

L EFT B EHIND 2014 10

Slide 11

Capturing the views of service users: Englands Adult Social Care Survey

Slide 12

A DULT S OCIAL C ARE S URVEY Annual survey (began 2010/11) How effectively are services helping users to live safely and independently? What is impact of services on their quality of life? Cross-sectional stratified random sample from every Council with Adult Social Services Responsibilities (CASSR, n=154) 2012/13 sample > 50,000 Service users aged 18 and over in receipt of services funded wholly or in part by Social Services Questions about -Subjective wellbeing -Social relationships -Leisure activities -Safety -Food quality -And more . http://www.hscic.gov.uk/article/2215/User-Experience-Survey-Adult-Social-Care- Guidance-2012-13

Slide 13

S ELECTED R ESULTS 1

Slide 14

S ELECTED R ESULTS 2

Slide 15

W HAT I S G OING O N ? The data are valid, English disability services (especially for people with intellectual disability) are excellent and getting better People with intellectual disability have low expectations Supported responding introduces significant positive bias Concentrating on mental characteristics (such as pleasure, happiness or desires) can be particularly restrictive when making interpersonal comparisons of well-being and deprivation. Our desires and pleasure- taking abilities adjust to circumstances . deprived people tend to come to terms with their deprivation [as such] .. the deprivation of the persistently deprived may look muffled and muted

Slide 16

S ELECTED R ESULTS 9%56%

Slide 17

R ESEARCH INTO A CTION Implications 1.Invest in annual data collections that can be used to monitor all key aspects of disability equality 2.Use subjective measures of wellbeing with extreme caution (if at all) 3.Invest in developing supports for survey completion that are independent of service provider agencies 4.Develop alternative ways of capturing the voices of people who use disability services

Slide 18

FACULTY OF HEALTH SCIENCES CENTRE FOR DISABILITY RESEARCH AND POLICY International developments in measuring outcomes: US National Core Indicators 18

Slide 19

Outline 1.Describe National Core Indicators (NCI) 2.Examples of policy-relevant NCI analyses 1.Choice of Living Arrangements 2.Wellbeing and Choice of Living Arrangements 3.ASD Eligibility Policies 3.How is NCI Data Used by US States? 1.Example from Kentucky 4.Possible Applications in Australia

Slide 20

1. National Core Indicators (NCI): Features, development, current use and future expansion

Slide 21

Who participates in the NCI? Annual survey on a random sample of service users (400+ per state) -Longitudinal (multi-year) data on specific individuals not available by design (to avoid survey fatigue) States opt in -Some states add questions to the standard NCI instrument to investigate issues of specific local policy interest -Some states focus on different parts of their service system in different years by oversampling different subgroups Used only by the intellectual disability and developmental disability (ID/DD) service system in each participating state

Slide 22

Who has access to NCI data? Benchmarking: For each indicator the NCI provides for: - state-by-state comparisons, -comparisons with the national average, -year-by-year comparisons within states Summary data publicly available via the NCI website http://www.nationalcoreindicators.org/ http://www.nationalcoreindicators.org/ Individual state annual reports available via NCI website Deidentified NCI data shared with (selected) university researchers for independent secondary analysis

Slide 23

NCI Indicator Framework Individual Outcomes Employment Choice & Control Relationships Community Inclusion Family Indicators Information & Planning Access to Supports Community Connections Choice & Control Health, Welfare, & Rights Health & Wellness Safety Respect & Rights System Performance Service Coordination Incidents & Mortality Staff Turnover = Adult Consumer Survey

Slide 24

NCI Participating States 2010-2013 2010-11 24 States 2011-12 29 States 2012-13 35 States HI WA AZ OK KY AL NC PA ME MA SD TX AR GA NM NJ MO NY LA OH NH DC CA FL IL OR WI IN MI MS SC VA MD CT RI UT

Slide 25

2. Examples of policy-relevant analyses using NCI data

Slide 26

Choice of Living Arrangements Overall What percentage of adult service users living outside the family home choose where and with whom they live? POLICY IMPLEMENTATION QUESTION

Slide 27

Overall NCI Choice Results 2008 6778 adult developmental disabilities service users living in non-family-home service settings in 26 US states (Stancliffe et al., 2011)

Slide 28

CONCLUSION Most people have no choice of where to live (55%) or whom to live with (59%). Policies endorsing choice of living arrangements are not being implemented satisfactorily.

Slide 29

Choice of Living Arrangements Does choice of living arrangements vary by residence type and level of disability? POLICY IMPLEMENTATION QUESTION

Slide 30

Choosing Whom to Live With (person chose) by Level of Disability and Residence Type

Slide 31

CONCLUSION People with severe/profound intellectual disability had little or no choice of whom to live with, regardless of residence type.

Slide 32

Choosing Whom to Live With (person chose) by Level of Disability and Residence Type

Slide 33

CONCLUSIONS For people with mild and moderate intellectual disability, choice of living companions varies dramatically by residence type: -own home (73.5% and 57.3% chose) -group home (9.5% and 9.7% chose) These findings support policies promoting individualised settings, such as ones own home or an agency apartment. -These settings do provide substantially more choice about living arrangements, as intended.

Slide 34

Wellbeing and Choice of Living Arrangements

Slide 35

Choice of Living Arrangements Does exercising choice of living arrangements lead to greater wellbeing? (Stancliffe et al., 2009) POLICY IMPLEMENTATION QUESTION

Slide 36

NCI Wellbeing Outcomes Loneliness Feeling happy At Home Feeling afraid at home Feeling afraid in your neighborhood Home staff nice and polite Liking home

Slide 37

Self-Report Data Only Well-being items come from Section I of the NCI Consumer Survey, which may only be completed by interviewing the person receiving services. Due to communication difficulties, some service users could not take part in the interview. Only included participants who were judged by interviewers to have given valid and consistent interview responses. These selection criteria yielded predominantly people with mild or moderate ID.

Slide 38

Loneliness the most widespread problem

Slide 39

Results Summary ItemChose Who to Live with Chose Where to Live Loneliness Feeling happy Afraid at home Afraid in neighbourhood Home staff nice Like home Personal characteristics controlled statistically in all comparisons.

Slide 40

Conclusion Choosing where to live and whom to live with each are associated with: -multiple wellbeing benefits and -no wellbeing detriments.

Slide 41

Policy Analyses and Outcomes: ASD Eligibility Policies Grouping states by common policies to evaluate the impact of these polices on service provision and client outcomes. EXAMPLE Hewitt et al. (2011) compared the proportion of state ID/DD service users with and autism/ASD diagnosis by state autism/ASD service eligibility policies: ASD Eligibility PoliciesNo. of States % service users with ASD diagnosis None66.6% Related condition (RC)148.4% RC + autism specific HCBS59.3% x 2 (2, 12,382)=17.39, p