Kirsty Wells, Scotland Manager, HouseMark

Angela Currie, Director, SHBVN

Influences behind STAR in Scotland Key survey questions Building a questionnaire Survey methods Timing Census, sampling and checking for bias Benchmarking

Outline

STAR Social Housing Charter

Satisfaction data for

ARC

Satisfaction benchmarks across the

UK

Developed in Scotland with SHBVN

Established framework for satisfaction measures

Over 200 landlord comparators

Historic timeline of data – 2011 and before

Inclusive development process involving:◦ Landlords, tenant groups, market research

companies, other survey experts

Free to access guidance & support from HouseMark and SHBVN

What’s good about STAR?

‘Conducting a comprehensive satisfaction survey – a guide’ from SHR / Ipsos MORI

Very similar approach to STAR on most aspects of the survey

Some close-to-identical key questions

Consistency with the Scottish Household Survey

What’s good about the Social Housing Charter satisfaction measures?

Aspect of survey framework

Checklist

Flexible questionnaire

Response options except overall sat question

Permitted survey methods

Frequency / timing adapted slightly to match

Margin of error adapted slightly to match

Representative samples / weighting

Wording of core questions and

Original STAR alignment with Charter requirements

STAR in Scotland

code

Charter

Reference

Question

CorS1 CI1(B) Taking everything into account, how satisfied or dissatisfied are you with the service provided by [your landlord / landlord name]?

CorS2 CI10 Overall, how satisfied or dissatisfied are you with the quality of your home?

CorS4 CI29 Taking into account the accommodation and the services [your landlord / landlord name] provides, do you think the rent for this property represents good or poor value for money?

STAR / Charter equivalent questions

Some Charter questions had significantly different wording from the same themed question in STAR

Decided sufficiently distinct for each version to be included in a survey

So for each themed pair of questions you can:◦ Comply with the Social Housing Charter indicator

◦ Benefit from an additional STAR question enabling direct comparisons with the rest of the UK

The problem for STAR …

Theme Origin Question

Neighbourhood

STAR Overall, how satisfied or dissatisfied are you with your neighbourhood as a place to live?

CharterOverall, how satisfied or dissatisfied are you with [your landlord / landlord name]’s management of the neighbourhood you live in?

Involvement

STARHow satisfied are you that [your landlord / landlord name] listens to your views and acts upon them?

CharterHow satisfied are you with opportunities given to you to participate in [your landlord / landlord name]’s decision making process?

Repairs

STARGenerally, how satisfied or dissatisfied are you with the way [your landlord / landlord name] deals with repairs and maintenance?

CharterThinking about the LAST time you had repairs carried out, how satisfied or dissatisfied were you with the repairs service provided by [your landlord / landlord name]?

Keeping you informed

STARHow good or poor do you feel [your landlord / landlord name] is at keeping you informed about things that might affect you as a resident?

CharterHow good or poor do you feel [your landlord / landlord name] is at keeping you informed about their services and decisions?

Same theme – different wording

Charter ref

Question Service user

CI9(B)

Thinking about when you moved in, how satisfied or dissatisfied were you with the standard of your home?

New tenants

CI33

Taking everything into account, how satisfied or dissatisfied are you with the factoring services provided by [your landlord / landlord name]?

Owners of factored properties

CI28

How satisfied or dissatisfied were you with the overall quality of the temporary or emergency accommodation you were provided?

Users of local authority temporary or emergency accommodation

CI37

How satisfied or dissatisfied are you with [your landlord / landlord name]'s management of your site?

Users of gypsy / traveller sites

Charter questions not in original STAR

Use the exact question wording contained in STAR Questions for Scotland

Use the specified response options – as important as the question wording

Remember – for core and charter questions

Have you decided when you will conduct your large scale survey?

If so, what methodology do you plan to use?

Discuss

Build your own survey

• Starting point is the core and charter satisfaction questions

• Always ask the overall satisfaction question first

• Add to the core/charter questions with:• From optional questions from STAR

Questions for Scotland• Your own questions

• Add demographic questions as necessary

• Group questions by theme

As long as it’s statistically robust and representative, STAR in Scotland can be done by:◦ Post◦ Telephone◦ Face-to-face

On-line not acceptable for SHR: “extremely unlikely” to be representative

Permitted survey methods

Neither HouseMark, SHBVN nor SHR want satisfaction data which is too far out of date

Survey data should not be more than three years old

That is a maximum – you may conduct surveys more frequently if you wish

Take account of:◦ Data validity for comparisons◦ Utility value for business decisions◦ Cost

Keep data fresh!

Census – sent out to all tenants

If under 2,500 units, you should conduct a census and do not need to select a sample

For larger landlords, see section 3.3 of SHR / Ipsos MORI guidance on sample selection

Whether census or sample, always check make-up of respondents for evidence of bias e.g. proportionally more older tenants responding than in your tenant population?

Weighting can correct problems of bias – see section 4.4 of SHR / Ipsos MORI guidance

ARC return asks for unweighted and weighted results

Census, sampling and checking for bias

Will you conduct the survey in-house or externally commission the work?

Who in your organisation will be responsible for leading the project?

Discuss

Current E-form relates to STAR

A STAR in Scotland E-form will be introduced this year

Comparisons of STAR results will be available via:◦ PI Tracking◦ STAR benchmarking◦ Core and dashboard – alongside HouseMark cost

and performance measures

HouseMark Benchmarking

Annual benchmarking

Results available on query tool

Results available to use within report builder

Results used to inform VFM ranking

Sharing Good Practice

SHBVN Benchmarking

Unlikely all survey/feedback needs will be met by STAR in Scotland

Audit and review your whole framework for feedback and surveys

Avoid duplication and survey fatigue

Helpful to have a good base of tenant profile information◦ provides a variety of baselines for checking for bias◦ avoids having to repeatedly ask demographic questions

in surveys

Other considerations

www.housemark.co.uk/starscotland A Guide to STAR in Scotland STAR Questions for Scotland List of survey suppliers STAR in Scotland logos Links to relevant SHR documents

Coming soon Frequently Asked Questions Register for notification of updates

More info on STAR in Scotland