Conducting social researchCarolyn Black

Ipsos MORI – More than just opinion polls

Leading, world-wide research company Work with the world's leading businesses, governments and

institutions We specialise in researching:

– Advertising (brand and communications); – Loyalty (customer and employee relationship management); – MediaCT (media, content and technology research); – Marketing (understanding markets and building brand portfolios) and – Public Affairs (social & political issues)

Ipsos MORI Scotland focuses on Public Affairs research

Who are our clients? Public sector

My background Higher Modern Studies

Degree in Psychology, University of Edinburgh (but employees have diverse academic backgrounds e.g. Politics, History, Journalism)

Started as a Trainee Research Executive at Ipsos MORI in March 2008

There is no one path!

Some current projects

Behaviour in Scottish Schools Research 2012 – Scottish Government

Evaluation of the ‘You First’ programme – Scottish Government

Glasgow City Council Staff Survey – Glasgow City Council

Research methods

Characteristics of Qualitative Research Characteristics In-depth, open-ended interviewing with a discussion or topic guide Relatively small samples

Benefits Respondents define issues

– identify the full range of issues, Probe views in-depth (including unconscious associations)

– understand why people think/ behave as they doLimitations Small sample sizes so not statistically reliable Can never be representative

Characteristics of Quantitative Research Characteristics Relatively large samples of people (aims to be representative) Mainly ‘closed’ questioning

Benefits Measures the prevalence of attitudes/behaviours etc

– Estimate for whole population– Identify clear priorities in a reliable way– Useful for tracking

Limitations Tells you ‘what’ and ‘how many’ but not always ‘why?’ The researcher defines the range of answers given

9

Different types of data

What is primary data vs. secondary data?

– Primary data is something that you interact with – if ‘you’ didn’t instigate it, the data would not exist

– Secondary data is something that exists despite ‘you’, for example desk research (market reports, sales data etc)

Quantitative Research

Qualitative Research Desk Research

Qualitative research

11

Methodologies available – Qualitative Research

Qualitative research

Deliberative workshops

In-depth interviews

Paired-depths

Discussion groups

Standard

Conflict/ Krisis

Mini

Ethnography

OnlineBulletin boards

Testing concepts/

ads

12

Group discussions

Exposes participants to other views (ideas/consensus)

Multiplier effect Less time intensive than

depths

Exposes participants to other views (may defer/change)

One participant can dominate

Easier for participant to ‘coast’ & not engage fully

Not good for busy/dispersed audiences

Social desirability bias

13

One to one - depths

For busy professionals (B2B research)

Widely dispersed

May speak more freely

Sensitive topics

Detailed case studies

More time intensive than groups (expense per person)

Often re- scheduled/cancelled

No deliberation/ consensus (ideas?)

Telephone - harder to build rapport

Quantitative research

15

Methodologies available - quantitative

In-homeIn street

On site

Face-to-face

Quantitative surveys

Self-completion

PostalOn site

Online

Telephone surveys

16

Face to face research

Use of visual aids/show cards

Sensitive questions Longer (more detailed)

interview possible Personal rapport Good response rates Know you are speaking to

the right person

Expensive Takes time Cluster sampling (sample

bias) Interviewer bias Social desirability bias

17

Telephone research Speed

Cost effective

Easy to control quotas and sample

Especially appropriate for B2B

Good for widely dispersed audiences

Client can observe

Perceived anonymity

Difficult to ensure representative and might exclude mobile owners and/or those who are ex-directory

No visual aids (unless posted) / show cards

Difficult to explain complicated concepts or products

Higher refusal/drop out rate

Shorter interview length (less detail)

18

Postal research Can be much cheaper Large number of people can

be invited to take part Best for high interest/

involvement studies No interviewer bias Can show basic visual

material Can reach dispersed sample Anonymity Complete in own time – more

considered answers?

Low response rates are common (representative?)

Self-selecting - more critical/those with strong opinions likely to take part

No check on who actually responds

Poor qualitative information Less sophisticated questions Hard to control use of prompt

materials Cannot use complex routing

19

On-line

Cheap (No interviewer costs) Can be extended internationally Large samples Best for high interest/ involvement

studies Particularly suitable for employee

studies Complex routing and questions

possible No interviewer bias Anonymity Complete in own time – more

considered answers?

Sample issues:Internet penetrationSelf selection (panels)Out of date / incorrect email

addresses

Response rates can be low

Many of the same characteristics as postal

New methodologies – Social listening What is Social Listening?

– Capturing online ‘buzz’, conversations and opinions

– We use a tool that trawls through the internet daily to monitor mentions of particular words or phrases

– It sorts through blogs, social networking sites, forums, wikis, news sites, and video and image sites and then complies the data into a manageable format

How does it work?– The volume of mentions

– What is being said

– Where it is being said

– Who is saying what on Twitter

Questionnaire designA. Things you should do:

As a rule, keep questions short and simple

Use familiar words and concepts Start the questionnaire with easy questions they can all answer that are

relevant to what respondents have been told the survey is about

Ask sensitive and demographic questions at the end of the questionnaire

Questionnaire designB. Things to avoid:

Avoid ambiguity

Avoid leading questions

Avoid double barreled questions

Avoid double-negatives (usually with agree-disagree questions):

Question Validity A valid question is one that measures what we think it

does A question is of little use if people measure it one way

one day and another the next E.g. Self rated health question: How healthy are you?

Does this measure health or does it measure something else, such as optimism or happiness?

E.g. What is your national identity? Measure of subjective feelings or emotions towards a nation or an objective measure of citizenship or legal status?

Checking validity – cognitive testing

Cognitive testing allows an investigation into the way people understand, mentally process and respond to the question under scrutiny to explore whether the question, response options and accompanying instructions are interpreted in the way they are intended.

Is there a common understanding between researcher and respondent (which will result in valid data)?

How? Test in true-to-life conditions, small samples, in-depth discussions

2011 Census – health, household composition, qualifications, national identity, ethnicity, language, Scots!

Sampling – ensuring the results are generalisable

Definition of sampling

Learning about the views of a large group of people by speaking to a smaller number of them

On the assumption that the characteristics of the few we have interviewed (the sample) match those of the population i.e. they are to REPRESENT the population of interest

Making an INFERENCE about a….

POPULATION…….. …from a….. …..SAMPLE

Representative samples

The usual goal in sampling is to produce a representative sample (i.e. a sample that is similar to the population on all characteristics, except that it includes fewer people because it is a sample rather than the complete population) 

In other words, a representative sample is a "mirror image" of the population from which it was selected.

If a sample is representative of a known population inferences can be made to the population as a whole i.e. we can generalise the results

Unlikely to be perfectly representative – use statistical theory to help estimate how close the true population figure is likely to be to the figure obtained in the any particular sample

Types of samples Random Probability

– Simple Random

– Stratified

– Clustered

Quota ‘Snowballing’ - Specific groups not easy to find

Random probability samples Every member of the population has an equal chance of

inclusion in the sample Purest form of sampling (in theory) Most credible method Relatively easy process Usually used in postal, telephone and face to face surveys

Quota sampling Aims to control for variables that could affect the accuracy of

the sample. For example:– total of 12 per Output Area

– 5 men, 7 women

– 2 aged 16- 34, 4 aged 35-54, 6 aged 55+

– generally reflecting profile of local population

Fast / easy Mainly used for face to face surveys

Random or quota - which is best? Quota sample– cheaper than random (f2f)– can achieve representative

samples (on key factors controlling for)

BUT– interviewers have freedom of

selection– quotas not met for certain

target groups

Random sample– theoretically most pure– Everyone has an equal

chance of selectionBUT– costly (to do well f2f)– non-response bias

Generally, Random Sampling is statistically purer, but Quota Sampling is much cheaper and quicker to administer

Thank you!

Thank you for listening!

If you need any further information, please get in touch:

carolyn.black@ipsos.com