Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference

Just how comparative are comparative statistics?

Roger Jowell

Centre for Comparative Social Surveys

City University

LLAKES International Conference

www.europeansocialsurvey.org

Why bother with comparative social statistics?

Discovering differences and similarities

“Comparative sociology is not a particular branch of sociology. It is sociology itself.” Durkheim

Understanding one’s own society

Critical for cross-national governance

Helps challenge national stereotypes

But comparisons ideally needed both between countries AND over time


Obstacles to cross-national social measurement

Cultural incompatibilities

The language barrier

Breaching the ‘principle of equivalence’

Temptation of league tables

Clash between standards and consistency

Differences in methodological capacity

Differences in methodological habits


Overcoming the obstacles – at least partially

Ensuring equivalence of methods in all countries:Sampling

Mode of data collection

Language equivalence

Concept equivalence

Meticulous documentation

Consultative design, not ‘safari’ method

Contextual variables


A case history - the European Social Survey

Multinational time series, started 2001

34 countries so far

Contributes to scholarship and governance

Large training component and potential

Several substantive and methodological innovations

Widespread usage


Five main aims

To chart and explain changes in Europe’s social, political and moral climate

To achieve and spread high standards of rigour in comparative social measurement

To establish new indicators of societal well-being to stand alongside existing factual and behavioural indicators

To facilitate quantitative monitoring of value change by academics, policymakers, businesses and the public

To create and maintain a new contextual data repository

UKGreece Finland Ireland

PortugalSlovakia

Lithuania

Turkey

Switzerland Latvia

Austria Czech Republic

Luxembourg IcelandBulgaria

Ukraine

CyprusNorway

CroatiaRomania

Italy

Spain

FranceIsrael

SwedenRussia

Denmark

Belgium Estonia

HungaryPolandNetherlands

Slovenia

Germany

ESS Countries

Headquarters London, UK

ESS Countries by Round

COUNTRY R1 R2 R3 R4

Austria

Belgium

Bulgaria x x

Croatia x x x

Cyprus x x

Czech x x

Denmark

Estonia x

Finland

France

Germany

Greece x

Hungary

Iceland x x x

Ireland

Israel x x

Italy x x

COUNTRY R1 R2 R3 R4

Latvia x x

Lithuania x x x

Luxembourg x x

Netherlands

Norway

Poland

Portugal

Romania x x

Russia x x

Slovakia x

Slovenia

Spain

Sweden

Switzerland

Turkey x x

Ukraine x

UK


Funding story to date

Initiated and seed-funded by European Science Foundation

Then core-funded for five biennial rounds (to date) by European Commission

National costs of each round met by national academic funding councils

Over 30 separate funding decisions each round

Now selected as a prospective ‘ESFRI’ Research Infrastructure with prospect of long-term funding


Question clusters – some core, some rotating

Trust in institutions

Citizen engagement

Socio-political values

Immigration

Moral & social attitudes

Quality of life

Crime and security

Value orientations

Perceptions of criminal justice

National, ethnic, religious ID

Health and welfare issues

Life course perceptions

Ageism

Work and family life

Education and occupation

Financial circumstances

Household circumstances

Demographic composition


Outreach

32,000 registered data users to date (2500+ in UK)

On-line bibliography of publications based on ESS contains: 236 journal articles, 36 books and 90 chapters so far

Data increasingly deployed in policy debates

Training courses heavily over-subscribed

Influence on comparative methods well beyond Europe


A few summary findings


Education and Political Interest

Education and Interest in Politics in 22 countries

0

10

20

30

40

50

60

70

80

6 orfeweryears

7 8 9 10 11 12 13 14 15 16 17 ormoreyears

Years in education

% 'v

ery

' o

r 'f

air

ly' in

tere

ste

d in

po

liti

cs

Four Dimensions of Institutional Trust…

0

1

2

3

4

5

6

7

8Austria

BelgiumSw itzerland

Czech Republic

Germany

Denmark

Spain

Finland

France

United Kingdom

Greece

Hungary

Ireland

IsraelItaly

LuxembourgNetherlands

Norw ay

Poland

Portugal

Sw eden

Slovenia

Bulgaria

Cyprus

Estonia

Iceland

Russia

Slovakia

TurkeyUkraine

Trust in the police

Trust in the police varies but is high…


0

1

2

3

4

5

6

7

8Austria

BelgiumSw itzerland

Czech Republic

Germany

Denmark

Spain

Finland

France

United Kingdom

Greece

Hungary

Ireland

IsraelItaly


Norw ay

Poland

Portugal

Sw eden

Slovenia

Bulgaria

Cyprus

Estonia

Iceland

Russia

Slovakia

TurkeyUkraine

Trust in the legal system Trust in the police

Trust in the legal system is lower….


0

1

2

3

4

5

6

7

8Austria

BelgiumSw itzerland

Czech Republic

Germany

Denmark

Spain

Finland

France

United Kingdom

Greece

Hungary

Ireland

IsraelItaly


Norw ay

Poland

Portugal

Sw eden

Slovenia

Bulgaria

Cyprus

Estonia

Iceland

Russia

Slovakia

TurkeyUkraine

Trust in country's parliament Trust in the legal system Trust in the police

Trust in national parliaments is lower still…


0

1

2

3

4

5

6

7

8Austria

BelgiumSw itzerland

Czech Republic

Germany

Denmark

Spain

Finland

France

United Kingdom

Greece

Hungary

Ireland

IsraelItaly


Norw ay

Poland

Portugal

Sw eden

Slovenia

Bulgaria

Cyprus

Estonia

Iceland

Russia

Slovakia

TurkeyUkraine

Trust in politicians Trust in country's parliament Trust in the legal system Trust in the police

Trust in politicians is the lowest of all.


Attitudes to migration

Surges of xenophobia associated with economic downturns

New dangers of recession

But education matters

More education, less xenophobia, greater sympathy towards cultural diversity

Educated are more accepting of all newcomers, even of potential labour market competitors


10 Laws of comparative research


Law 1

Don’t confuse respect for cultural differences with tolerance of methodological anarchy


Law 2

Never design questions or interpret data about a country one knows little or nothing about


Law 3

Confine cross-national studies to the smallest number of nations compatible with the study’s intellectual needs


Law 4

Pay as much attention to collecting aggregate-level background information about each country as to individual-level variables


Law 5

Always be at least as absorbed by the limitations of the data as about their explanatory power


Law 6

Assume initially that any major ‘new’ cross-national variation one discovers is an artefact


Law 7

Resist the temptation to produce ‘gee-whiz’ league tables containing every nation in every analysis


Law 8

Undertake collective, study-specific, multi-national development work and pre-testing


Law 9

Routinely include methodological experiments in cross-national studies


Law 10

Ensure that cross-national datasets are accompanied by detailed methodological reports about procedures and outcomes in each nation

Documents

Just how comparative are comparative statistics? Roger Jowell Centre for Comparative Social Surveys City University LLAKES International Conference