raedelli

Research Design in European Studies: The Caseof Europeanization*

THEOFANIS EXADAKTYLOSUniversity of Exeter

CLAUDIO M. RADAELLIUniversity of Exeter

Abstract

In this article, we contribute to the debate on research design and causal analysis inEuropean integration studies by considering the sub-field of Europeanization. First, weexamine the awareness of research design issues in the literature on Europeanizationthrough a review of the debate on causality, concept formation and methods. Second,we analyse how much of the discussion of the trade-offs in causal analysis inmainstream political science has percolated into Europeanization studies. We thereforeconstruct a sample of the Europeanization literature, comparing it to a control group ofhighly cited articles on European integration. This enables us to control if somepatterns are specific to the Europeanization literature or reflect a more general trend inEuropean integration. We then look at trade-offs in the Europeanization sample and inthe control group. Our findings indicate that awareness of research design is still low.Europeanization articles differ from the control group in the focus on mechanisms(rather than variables) and the qualitative aspects of time in politics. Complex notionsof causality prevail in Europeanization but not in the control group and the cause-of-effects approach is preferred to effects-of-causes in the control group but not inEuropeanization – in both cases, however, the difference is slight. We conclude byexplaining differences and similarities and make proposals for future research.

* We would like to acknowledge the support of the Jean Monnet Programme. An earlier version of thearticle was delivered to the 4th ECPR General Conference in Pisa in September 2007. We would like tothank Susan Banducci, Fabrizio De Francesco, Fabio Franchino, Oliver Fritsch, Simon Hix, ThomasPlümper, Stephen Wilks and two anonymous reviewers for their constructive comments – the usualdisclaimer applies. The full set of tables and dataset is available on the Journal’s website.

JCMS 2009 Volume 47. Number 3. pp. 507–530DOI: 10.1111/j.1468-5965.2009.00820.x

© 2009 The Author(s)Journal compilation © 2009 Blackwell Publishing Ltd, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148,USA

Introduction

During the past decade, research on European integration has witnessed theemergence of research questions around the impact of the European Union(EU) on the domestic level, in terms of policies, institutional change andparty politics. This is nowadays the field known as Europeanization, con-cerned with ‘the institutional, strategic and normative adjustments generatedby European integration’ (EPPIE, 2007, p. 39). A strand of the European-ization literature, often tangential and connected to empirical concernsabout measurement and causality, has taken an interest in methods andresearch design. Specifically, this strand looks at issues in causal analysis.Scholars have discussed concept formation, mechanisms, the interplaybetween ideational and structural variables, and causal models, particularlythe difference between top-down recursive models and bottom-up researchdesigns.

One argument often aired at academic conferences is that Europeanizationhas potential for the normalization of European integration studies. Hassen-teufel and Surel (2000) and later a group of young scholars working under thesupervision of Palier and Surel (EPPIE, 2007, p. 45) have argued that Euro-peanization research can be the carrier of less ad hoc theorizing and moreintegration with mainstream political science. Thus, does Europeanizationprovide evidence of awareness of research design debates that feature promi-nently in ‘normal’ political science, especially comparative politics? If so,what are the methodological choices made by Europeanization scholars whenthey encounter classic trade-offs in causal analysis? And finally, is there anyoriginal contribution that Europeanization can make to the wider researchdesign debate? These are the three research questions that motivate thisarticle.

Up until now, most surveys of the field have dealt with the nature ofEuropeanization, its mechanisms and outcomes (Börzel and Risse, 2003;Caporaso, 2007; Lenschow, 2005; Olsen, 2002; Radaelli, 2003). Other reviewarticles are more concerned with issues of change in domestic institutions,actors, procedures and paradigms (Börzel, 2005; Bulmer and Radaelli, 2005;Kassim, 2005; Ladrech, 2005) or with the impact of the EU on new MemberStates or beyond Europe (Schimmelfennig, 2007; Sedelmeier, 2006).However, these studies do not address issues of research design. One excep-tion is Graziano and Vink (2007) where issues of methods are explicitlyaddressed by Markus Haverland, who has also treated case selection issues ina separate online paper (Haverland, 2005, 2007). Overall, the rare method-ological discussion in these review articles is not based on the systematicexploration of a sample of the literature – this is exactly what we set out to

508 THEOFANIS EXADAKTYLOS AND CLAUDIO M. RADAELLI

© 2009 The Author(s)Journal compilation © 2009 Blackwell Publishing Ltd

present here, drawing on Franchino’s (2005) template. The difference toFranchino is that he measures the propensity to test theories in Europeanintegration articles, whilst we look at causal analysis in Europeanizationarticles and in a control group of highly cited political science articles onEuropean integration.

As mentioned, we take a research design perspective, looking at causalanalysis – as defined by Mahoney and Goertz (2006, pp. 228–9). We start witha discussion of concept formation, causality and methods that has emergedsomewhat endogenously in the Europeanization literature in Section I. Wepresent the concept of trade-offs in causal analysis in mainstream politicalscience in Section II. We then build a sample of the Europeanization literaturein Section III and examine how it relates to some fundamental issues inresearch design and compare the findings with a second control sample drawnfrom the broader literature on the politics of European integration. The lastsection compares and concludes.

At the outset, however, we wish to clarify what we are not doing. First, weexclude from our analysis non-causal approaches. Second, we do not examinethe whole literature on Europeanization but only a sample including the mostcited pieces. The sample is limited to articles – monographs, edited volumesand individual book chapters are neglected. Third, we are limiting our analy-sis to political science research on Europeanization without crossing intoother related fields (for example, environmental studies, industrial relations,socio-legal studies and sociology).

To proceed systematically, we examine the literature by using anapproach informed by meta-analysis and systematic reviews. Meta-analysisis a form of extracting patterns in the findings of a selected set of studies.Systematic reviews are interrogations of samples of literature with a specificquestion in mind. The basic idea is to take articles as units, code the prop-erties of the units via a protocol (so that a quantitative treatment becomespossible) and then analyse the data – this is the template suggested by Lipseyand Wilson (2001). Data are analysed to investigate and describe the patternof findings in the sample (Lipsey and Wilson, 2001, pp. 73–88, 146–67).Meta-analysis cannot be performed to summarize theoretical papers, con-ventional research reviews, policy proposals and other reports of similarnature. We stick to this template, although we do not go beyond basicstatistics, given that our samples are small. In essence, our approach is asystematic review informed by the rules (of inclusion and exclusion) ofmeta-analysis. The patterns we are interested in concern research design.Before we present our systematic approach to the identification of thesepatterns, it is useful to provide information on how Europeanization scholarshave encountered issues of research design.

RESEARCH DESIGN IN EUROPEAN STUDIES 509


I. Europeanization Research: From Concepts to Research Design

Research design issues have emerged in the context of the debate on defini-tions (Graziano and Vink, 2007; Olsen, 1995a, b; Radaelli, 2003). Definitionsbring in the question of concept formation and in turn concepts lead toresearch design decisions about measurement (Radaelli and Pasquier, 2007).

In their discussion of the concept, Börzel and Risse (2003) suggest atheoretical framework around the issue of ‘goodness of fit’ arguing that ‘thelower the compatibility between European and domestic processes, policiesand institutions, the higher the adaptational pressure’ (Börzel and Risse, 2003,p. 61). Research design – they argue – can either be organized around socialconstructivist propositions or test more rational choice arguments. Radaelli(2003) proposes a time-sensitive definition that boils down to the idea thatEuropeanization is a process, not an outcome. In terms of research design,this invites a choice for process-tracing and Piersonian notions of causality.This definition has also been used to discuss two types of research design forempirical research, called top-down and bottom-up. Let us briefly see what isat stake here.

In top-down models (Caporaso, 2007), empirical research starts from thepresence of integration, controls the level of fit/misfit of the EU-level policyvis-à-vis the Member States and then explains the presence or absence ofdomestic change. The model is recursive, that is there are no exogenousvariables. Technically, this model can be represented by a system of linearequations that are solved simultaneously. It allows for a wide range of inter-vening variables or mediating factors, as shown by Schmidt (2002). Thebottom-up research design exogenizes the EU level (Radaelli, 2003; Radaelliand Pasquier, 2007). It starts from the set of actors, ideas, problems, rules,styles and outcomes at the domestic level at time zero – in short, the policysystem at a given time. Then it process-traces the system over the years andidentifies the critical junctures or turning points – for example, when majorideational change takes place, or the constellation of dominant actors isaltered. For each juncture, the question becomes: was the cause of this majorchange domestic, or did the change come from exogenous variables like theEU-level variables or global-level variables? In order to assess the contribu-tion of the EU variables, the researcher goes ‘up’ – controlling temporalcausal sequences from the domestic level, where the major change emergesempirically, to the EU. There is a similarity with backward mapping inimplementation research – a strategy in which we start from the implemen-tation outcome and work causality backwards (Elmore, 1999 [1982]).

Within this discussion on causality and how to draw inferences fromempirical evidence, some authors have also made progress in identifying



typologies of mechanisms (Knill and Lehmkuhl, 2002) and outcome (Héritieret al., 2001). Schmidt (2002) has made a distinction between the mechanismsinvolving structural variables and the special ways in which discourse has acausal impact on Europeanization and transforms structural variables. Dis-course can produce change ‘by altering perceptions of economic vulnerabil-ities and policy legacies and thereby enhance political institutional capacityto impose or negotiate change’ (Schmidt, 2002, pp. 899–900). This brings thewhole discussion of how to go about research design to a more sophisticatedlevel, as shown by Schmidt’s further work on discursive institutionalism(Schmidt, 2006).

Yet another stream of authors focuses on the complexity of causal chainsin the context of temporal sequences connecting major EU policies, likeeconomic and monetary union (EMU), to domestic changes (Dyson, 2000,pp. 646–7; 2002). In a carefully designed collection of case studies on EMUand domestic change, Dyson and his associates show how previous work onEMU exaggerated the influence of the EU in domestic political change(Dyson, 2002). Recent work by Stolfi (2008) demonstrates that the notion ofthe EU as an external constraint bringing about domestic change has obfus-cated the more important role of domestic policy communities. Interestingly,Stolfi uses Radaelli’s bottom-up research design to substantiate his theoreti-cal claims, thus connecting the methodological discussion with empiricalanalysis.

Finally, other authors have entered the debate on research design by usingthe notion of a control group. The question is simple: even if n EU MemberStates are experiencing change, it well may be that k countries outside the EUare also going through the same process of change – for example, becausethey are all part of a global process of diffusion (see Saurugger, 2005;Levi-Faur, 2004).

To conclude, research design issues have cropped up in this field, occa-sionally in connection with a much wider discussion on research design thathas occupied mainstream political scientists for quite a while, especially sincethe publication of Designing Social Inquiry by King, Keohane and Verba(1994). Brady and Collier’s Rethinking Social Inquiry (2004) has made thenotion of trade-offs in causal analysis quite prominent. In the next section, weshow how the trade-offs have been identified.

II. Trade-Offs in Causal Analysis

In his influential book The Comparative Method (1987), Ragin commentsthat ‘social science methodology does not concern mere technique; it



concerns the relationship between thinking and researching’ (Ragin, 1987,p. 165). For us at least, the issue at stake in causal analysis is not qualitativeversus quantitative methods; it is a different and more essential question oftrade-offs within a ‘pluralistic vision of methodology’ (Brady et al., 2006).Trade-offs provide the benchmark for evaluation and criteria-basedapproaches (Gerring, 2001).

We identified six categories of trade-offs in causal analysis. Drawing onthe Europeanization debate reviewed in the previous section, we added aseventh category (top-down versus bottom-up design). As the seventh cat-egory is specific to Europeanization, we do not use it for the control group.The six trade-offs are:

1. ‘Cause of effects’ versus ‘effects of causes’ approach;2. concept formation versus measurement;3. complex notions of causation (including multiple-conjunctural causation)

versus singular linear causation;4. omitted variables bias versus multi-collinearity – operationalized for

the purposes of the article as a rich set of independent variables versusparsimony;

5. time as a qualitative factor in politics versus time as quantity of years; and6. mechanism-oriented research versus variable-oriented analysis.

‘Cause of effects’ versus ‘effects of causes’: Mahoney and Goertz interpretthis trade-off as a manifestation of two different research cultures, qualitativeand quantitative. According to these authors, in one case the researcher islooking for the explanation of outcomes in individual cases; in the other casethe goal is to ‘estimate the average effect of one or more causes across apopulation of cases’ (Mahoney and Goertz, 2006, p. 230). We think that thereis no reason to assume a stark contrast between qualitative and quantitativecultures – otherwise we might as well have a single trade-off between quali-tative and quantitative approaches rather than six trade-offs. Hence we use‘cause of effects’ for articles that start with a dependent variable in terms ofoutcomes – for example, constrained policy autonomy – and investigate thepossible cause, be it global economic interdependence or European integra-tion. ‘Effects of causes’ is typical of studies that are interested in trackingdown how a specific cause, such as European integration, has different effects(for example, on domestic politics and policy).

Concept formation versus measurement: The choice of research designincorporates a second goal regarding descriptive inferences – more specifi-cally regarding their generalization, simplicity and meticulousness in concep-tualization and measurement (Brady and Collier, 2004, p. 222). Central to thisgoal lies a critical trade-off between concept formation and measurement.



One can, of course, envisage a sequence from concept formation to measure-ment in the historical development of a research field. For the individualresearcher, however, the problem is whether to prioritize concept formation orto develop measures. In turn, concept validity, if not adequately addressed, isa major source of measurement error (Mahoney and Goertz, 2006, p. 244).Conceptual stretching crops up in the Europeanization literature as well(Radaelli, 2003). Like in the previous trade-off, we do not make the ex anteassumption that qualitative researchers work on concepts and quantitativeresearchers privilege measurement. We will see later that indeed there arecases in which quantitative analysis is used to perfect a concept and cases inwhich a qualitative study is used to answer a question of measurement, withno sophisticated approach to concept formation.

Complex causation versus singular linear causation: Yet again, in ourview this is not a trade-off between qualitative and quantitative researchculture. For the purposes of our scoring, we included in complex causationnon-linear econometric models, such as structural model equations, multipleconjunctural causation, qualitative comparative analysis, equifinality, increas-ing returns, punctuated equilibria and models where the causal logic changesbefore and after a threshold level of a variable (Hall, 2003; Pierson, 2004;Ragin, 1987, 2000).

Omitted variables bias versus multi-collinearity: This is a trade-offbetween trying to reduce bias generated by neglecting some important vari-ables and bias arising out of the correlation between independent variables.The more one includes independent variables in the explanation, the higherthe likelihood of multi-collinearity problems – for example, two or moreexplanatory variables are correlated to such a degree that it is impossible toseparate their causal effects (King et al., 1994, p. 119). This issue can beaddressed by collecting additional observations to provide more leverage inthe differentiation of the causal effects (King et al., 1994, p. 123). However,according to Brady and Collier (2004, p. 48), increasing the number ofobservations ‘make[s] it harder to achieve other important goals, such asmaintaining independence of observations, measurement validity and causalhomogeneity’. As mentioned, we operationalize this trade-off as one betweenparsimony (the obvious way to avoid multi-collinearity) and rich explanation(the intuitive way to avoid omitted variables bias).

Time as quantity of years versus time as a qualitative factor: Here whatmatters is the consideration of time as a factor that can qualitatively affectpolitics. The reference is to the discussion on temporality in politics (Goetz,2006; Pierson, 1996, 2004). The trade-off is at the level of initial assumptions.Do we start from hypotheses about slow, complex causal chains of eventsthrough time, or from more basic assumptions? The choice is between



sophisticated approaches to time that are, however, difficult to model andraise issues of periodization and simple assumptions about ‘time as number ofyears’ that are amenable to a variety of relatively complex models includinginterrupted time-series models, but without addressing the complexity of timetypical of historical-institutional analysis (for a Piersonian approach to EUpolitics see Bulmer, 2009). We operationalize this trade-off by separatingarticles that consider qualitative effects of time from articles that treat time asquantity (number of years). We also have a residual category of articles thatare a-temporal; hence they do not deal with time at all.

Mechanism-oriented versus variable-oriented design: Here the issue isone of prioritizing one or the other. A focus on variables risks downplayingthe role of mechanisms in causal analysis. On the other hand, mechanism-oriented research designs are in danger of neglecting the necessary andsufficient conditions under which these mechanisms are triggered.

Top-down versus bottom-up approach: This is only pertinent to the studyof Europeanization because it refers specifically to the orientation of theprocess itself.

III. Sample Construction and Findings

We compiled a first selection based on the Social Science Citation Index(SSCI) (search on ‘Europeani$ation’ AND ‘Politics’; period 1997–2007),from which we extracted the most frequently cited articles of the discipline.To establish whether some features are unique to the field of Europeanizationor reflect more general trends in European Studies, a control group is needed.For the control group we searched ‘European Union’ AND ‘Politics’ in thesame index for the same years. We excluded review articles, normativearticles without any empirical analysis, industrial relations articles and sta-tistical artefacts (there are a few articles with Europeanization somewhere inthe abstract but no reference to this topic). We also excluded articles wellbelow the H-index as calculated by the SSCI. This left us with 32 European-ization articles, cited at least five times (Table 1). The corresponding totalnumber for the control group was therefore set at 32 as well (Table 2).

A quick qualitative look at Tables 1 and 2 is sufficient to appreciate thediversity between the two fields. We expected several double entries (i.e. thesame article featuring in the two tables) but this is not the case (Cole andDrake is an exception). Arguably, there is an artefact component: if an authoruses Europeanization in the abstract, title or key words, he or she may notwant to use politics AND European integration as well, thinking it is redun-dant. This way his or her article will feature in one sample but not in the other.



This may be why the top articles in the Europeanization sample do not showup in the middle of the control group table. Yet, on average, Europeanizationarticles are still several citations away from the pack of articles in the controlgroup.

The units in the Europeanization sample are disproportionately moreinterested in policy analysis (including of course regulation, transposition ofdirectives, comitology and implementation analysis), country-based case

Table 1: List of Cases in the Europeanization Sample (N = 32)

Author Journal Year Times cited

Scharpf FW JEPP 1997 39Börzel TA JCMS 1999 38Grabbe H JEPP 2001 31Benz and Eberlein JEPP 1999 29Knill and Lehmkuhl EJPR 2002 26Börzel TA JCMS 2002 21Harmsen R Governance 1999 18Marcussen; Risse;

Engelman-Martin et al.JEPP 1999 18

Lippert; Umbach; Wessels JEPP 2001 17Cole and Drake JEPP 2000 15Agh A JEPP 1999 13Bursens P Scandinavian Pol Studies 2002 13Lavenex S JCMS 2001 13Schmidt VA JEPP 2002 13Andersen MS Am Beh Scientist 2002 12Gilardi F Annals of Am Ac of Pol Soc Sci 2005 12Radaelli CM CPS 1997 12Dimitrova A WEP 2002 11Dyson K JCMS 2000 11Semetko; De Vresse; Peter J WEP 2000 11Levi-Faur D CPS 2004 10Schimmelfennig and Sedelmeier JEPP 2004 9Ladrech R Party Politics 2002 8Eyre and Lodge JEPP 2000 7Falkner G JCMS 2000 7Cole A JCMS 2001 6Warleigh A JCMS 2001 6Anderson JJ JCMS 2002 5Beyers and Trondal WEP 2004 5Fischer; Nicolet; Sciarini WEP 2002 5Lodge M WEP 2000 5Smith J PA 2001 5

Source: ISI Web of Knowledge; search terms ‘Europeani$ation’; search years 1997–2007; date of lastaccess, 1 February 2008.



studies and historical narratives. Control group units are much more diverse,with several articles on the substance of law-making and voting in the EUinstitutions, public opinion and identity, as well as public policy and modes ofgovernance. Another difference is that the Europeanization units tend to beclustered around a narrow group of journals. The control group refers to amore diverse set of journals. Moreover, 21 of the 32 Europeanization articleswere published in three years (2000, 2001 and 2002). The pattern for the

Table 2: List of Cases in the Control Group Sample (N = 32)

Author Journal Year Times cited

Alter KJ IO 1998 60Hix S EJPR 1999 54Hodson and Maher JCMS 2001 50Anderson CJ CPS 1998 50Taggart P EJPR 1998 48Keating M Environment and Planning C 1997 45Leitner H Political Geography 1997 43Garrett; Kelemen; Schulz IO 1998 41Pahre R Journal of Conflict Resolution 1997 35McLaren LM Journal of Politics 2002 32Tallberg J IO 2002 29Huysmans J JCMS 2000 29Gabel MJ AJPS 1998 29Carrubba CJ Journal of Politics 2001 26Schimmelfennig F EJIR 2000 25Bouwen P JEPP 2002 22Meyer C JCMS 1999 22Vogel D BJPS 2003 20Smith A Political Geography 2002 20Cichowski RA CPS 2000 18Friis and Murphy JCMS 1999 18Verdun A JEPP 1999 17Paarberg R IO 1997 16Mastenbroek E EUP 2003 15Cole and Drake JEPP 2000 15Pappi and Henning EJPR 1999 15Risse; Engelmann-Martin; Knopf et al. EJIR 1999 15Haas PM JEPP 1998 15Selck TJ Journal of Theoretical Politics 2004 14Selck and Steunenberg EUP 2004 13Spanou C JEPP 1998 13Smith ME JEPP 2000 12

Source: ISI Web of Knowledge; search terms ‘European Union’ AND ‘Politics’; search years 1997–2007;date of last access, 1 February 2008.



control group is more even. In both cases, it takes some five to six years foran article to gain a high number of citations.

We created a scorecard which included the six trade-offs, plus the trade-offspecific to Europeanization studies only for sample A. Each trade-off was splitinto three categories. If an observation was in accordance with one of the twooptions of the trade-off it was marked with the value ‘1’, otherwise it wasmarked with ‘0’. Observations that fell under the ‘not applicable’ option weremarked with a ‘-1’ value, although in the data analysis the category was notconsidered as a ‘missing case’. In terms of scoring the actual sample, uponconstruction of the survey protocol, we employed ourselves as the two codersto ensure inter-coder reliability. Inter-coder reliability may not increase thevalidity of the actual scoring in a sample size similar to ones of this exercise, butit increases transparency and congruence (Krippendorf, 2004). We scored thearticles once for a pilot exercise (to test our criteria and definitions on the road),then we refined our criteria and scored a second time, in April–May 2008.

The first question we raised in the Introduction was about the overallawareness of research design issues. When we started coding, it turned outthat most of the articles do not show awareness of the methodological trade-offs mentioned above. The Europeanization authors do not have explicitsections on research design; neither do they discuss whether they are morelikely to have a problem of multi-collinearity or a problem of omitted vari-ables bias. In consequence, we had to extrapolate the choice made by theauthors in terms of research design by carefully considering the empiricalevidence. We used the empirical analysis presented in the articles as revealedpreference of the methodological choices made by the authors. To defend thischoice, we argue that no matter how much an author is aware of a method-ological trade-off, in order to perform analysis he or she must have madesome implicit choices.

The same problem applies to the control group of European integrationarticles: it is hard to detect an interest in causal analysis and trade-offs and anacknowledgement of the methodological issues at stake in most of the papers.In this respect, Europeanization and the wider category of European integra-tion have the same features. In some cases, however, the trade-offs aregenuinely not applicable to the study in question, and are therefore assigneda value of -1.

Turning to the findings, Table 3 compares the results for the two samples.Tables 4 and 5 show the correlation between the trade-offs. Following theargument of Mahoney and Goertz that all trade-offs reflect a major fracturebetween qualitative versus quantitative, one would expect strong correlationbetween the six trade-offs (the seventh trade-off used in the Europeanizationsample is not interesting in this respect). However, this is not the case in our



Tabl

e3:

Com

pari

ngth

eTw

oSc

orec

ards

IDSA

MP

LE

Trad

e-of

fsin

caus

alan

alys

is

12

34

56

Causeofeffects

Effectsofcauses

N/A

Concept

Measurement

N/A

Complexcausation

Singularcausation

N/A

Parsimony

Richsetofindependentvariables

N/A

Time-quantity

Time-quality

N/A

Mechanism-oriented

Variable-oriented

N/A

1E

urop

eani

zatio

n(N

=32

)16

160

923

016

124

919

412

155

1814

02

Con

trol

grou

p(N

=32

)17

141

824

014

144

1220

016

133

1217

3

Sour

ce:A

utho

rs’

own

data

.



Tabl

e4:

Cor

rela

tions

Mat

rix

for

Eur

opea

niza

tion

Sam

ple

Cau

sev

effe

ctC

once

ptv

mea

sure

Com

plex

vsi

ngul

arP

arsi

mon

yv

rich

Tim

equ

ant

vqu

alM

echa

nv

vari

able

Cau

sev

effe

ctPe

arso

nco

rrel

atio

n1

.070

-.09

0.0

51-.

135

.252

Sig.

(2-t

aile

d).7

05.6

25.7

83.4

62.1

64N

3232

3232

3232

Con

cept

vm

easu

rePe

arso

nco

rrel

atio

n.0

701

.362

(*)

-.38

3(*)

.203

.412

(*)

Sig.

(2-t

aile

d).7

05.0

42.0

30.2

65.0

19N

3232

3232

3232

Com

plex

vsi

ngul

arPe

arso

nco

rrel

atio

n-.

090

.362

(*)

1.0

09-.

040

.294

Sig.

(2-t

aile

d).6

25.0

42.9

61.8

26.1

02N

3232

3232

3232

Pars

imon

yv

rich

Pear

son

corr

elat

ion

.051

-.38

3(*)

.009

1-.

298

-.18

5Si

g.(2

-tai

led)

.783

.030

.961

.098

.311

N32

3232

3232

32

Tim

equ

ant

vqu

alPe

arso

nco

rrel

atio

n-.

135

.203

-.04

0-.

298

1.0

06Si

g.(2

-tai

led)

.462

.265

.826

.098

.975

N32

3232

3232

32

Mec

han

vva

rPe

arso

nco

rrel

atio

n.2

52.4

12(*

).2

94-.

185

.006

1Si

g.(2

-tai

led)

.164

.019

.102

.311

.975

N32

3232

3232

32

Sour

ce:A

utho

rs’

calc

ulat

ions

onSP

SS.

Not

e:*

Cor

rela

tion

issi

gnifi

cant

atth

e0.

05le

vel

(2-t

aile

d).



Tabl

e5:

Cor

rela

tions

Mat

rix

for

Con

trol

Gro

upSa

mpl

e

Cau

sev

effe

ctC

once

ptv

mea

sure

Com

plex

vsi

ngul

arP

arsi

mon

yv

rich

Tim

equ

ant

vqu

alM

echa

nism

vva

riab

le

Cau

sev

effe

ctPe

arso

nco

rrel

atio

n1

-.25

8.4

10(*

).0

00-.

128

-.49

3(**

)Si

g.(2

-tai

led)

.154

.020

1.00

0.4

85.0

04N

3232

3232

3232

Con

cept

vm

easu

rePe

arso

nco

rrel

atio

n-.

258

1-.

370(

*).2

98.0

83.2

02Si

g.(2

-tai

led)

.154

.037

.097

.653

.267

N32

3232

3232

32

Com

plex

vsi

ngul

arPe

arso

nco

rrel

atio

n.4

10(*

)-.

370(

*)1

.024

-.07

4-.

060

Sig.

(2-t

aile

d).0

20.0

37.8

98.6

86.7

46N

3232

3232

3232

Pars

imon

yv

rich

Pear

son

corr

elat

ion

.000

.298

.024

1-.

086

.271

Sig.

(2-t

aile

d)1.

000

.097

.898

.639

.133

N32

3232

3232

32

Tim

equ

ant

vqu

alPe

arso

nco

rrel

atio

n-.

128

.083

-.07

4-.

086

1-.

050

Sig.

(2-t

aile

d).4

85.6

53.6

86.6

39.7

85N

3232

3232

3232

Mec

han

vva

rPe

arso

nco

rrel

atio

n-.

493(

**)

.202

-.06

0.2

71-.

050

1Si

g.(2

-tai

led)

.004

.267

.746

.133

.785

N32

3232

3232

32

Sour

ce:A

utho

rs’

own

calc

ulat

ions

onSP

SS.

Not

es:

*C

orre

latio

nis

sign

ifica

ntat

the

0.05

leve

l(2

-tai

led)

;**

corr

elat

ion

issi

gnifi

cant

atth

e0.

01le

vel

(2-t

aile

d).



two samples. Tables 6 and 7 provide the more fine-grained information usedto compile the summative results of Table 3. Table 6, in particular, shows thatmost Europeanization articles are based on the top-down research design (21against five).

Let us now look at the Europeanization and the control group together(Table 3). Both samples show a balanced distribution between cause of effectsand effects of causes – the control group with a slight orientation towards theformer. Turning to the trade-off between concept and measurement, weexpected to find Europeanization articles more interested in developing theconcept rather than measurement and the opposite for the control group. Thisis because the field of Europeanization is relatively new. Hence – we reasoned– researchers spend more time in discussing their concepts. In addition, thereview articles on Europeanization (Lenschow, 2005; Olsen, 2002; Radaelli,2003) suggest that definitional issues and concept formation have been some-what prominent. The sample, by contrast, seems to suggest that there is apreference for measurement. This characteristic features also in the controlgroup.

On the type of causation (Table 3: column 3), yet again the two samples donot show much difference, although Europeanization articles have a slightpreference for complex notions of causation. This is arguably the result of afield that was literally created around a causation puzzle: how can one grasp theessence of the Europeanization process if there are multiple feedback loopsbetween domestic and EU variables in this process? Unsurprisingly, research-ers are working with complex notions of causation in this area. Singularcausation is preferred by those who think of Europeanization as implementa-tion of EU decisions – for these authors causation is more straightforward.

Both Europeanization and control group units prefer a rich set of explana-tory variables to parsimony (Table 3: column 4). Regarding time (Table 3:column 5) the Europeanization sample deals with the qualitative aspects ofthis variable more than the control group. The (relatively small) differencecan be explained by noting that several articles on Europeanization are basedon policy analysis, longitudinal case studies and/or process tracing. Since theEuropeanization scholars have spent much time discussing how exactly thisprocess works, there is a major emphasis on mechanisms that we do not findin the control group (Table 3: column 6).

Conclusion

In this article, we have examined the field of Europeanization by looking attrade-offs in causal analysis. Previous work has been more interested in either



Tabl

e6:

Scor

ecar

dof

the

Eur

opea

niza

tion

Sam

ple

IDA

utho

r(y

ear)

Trad

e-of

fsin

caus

alan

alys

is

12

34

56

7

Causeofeffects

Effectsofcauses

N/A

Concept

Measurement

N/A

Top-down

Bottom-up

N/A

Complexcausation

Singularcausation

N/A

Parsimony

Richsetofindepvar.

N/A

Time-quantity

TimeQuality

N/A

Mech-orient

Var-orient

N/A

1Á

gh20

020

10

01

01

00

01

00

10

10

00

10

2A

nder

sen

2002

01

00

10

10

00

10

01

01

00

01

03

And

erso

nJJ

2002

01

00

10

10

00

10

01

00

10

01

04

Ben

zan

dE

berl

ein

1999

10

00

10

10

01

00

01

01

00

01

05

Bey

ers

and

Tro

ndal

2004

10

00

10

01

00

10

10

00

0-1

10

06

Bör

zel

1999

01

01

00

10

01

00

01

01

00

10

07

Bör

zel

2002

10

01

00

10

00

0-1

00

-11

00

10

08

Bur

sens

2002

10

00

10

10

01

00

01

01

00

10

09

Col

e20

011

00

01

01

00

01

01

00

01

01

00

10C

ole

and

Dra

ke20

000

10

10

00

10

10

00

10

01

00

10

11D

imitr

ova

2002

01

00

10

10

00

10

01

00

0-1

01

012

Dys

on20

000

10

01

01

00

00

-10

0-1

01

01

00

13E

yre

and

Lod

ge20

000

10

01

01

00

10

01

00

00

-11

00

14Fa

lkne

r0

10

01

01

00

10

00

10

01

01

00

15Fi

sche

r;N

icol

et;

Scia

rini

2002

01

00

10

10

00

10

10

01

00

01

0



16G

ilard

i20

051

00

01

00

0-1

01

01

00

01

01

00

17G

rabb

e20

010

10

10

01

00

10

00

10

10

01

00

18H

arm

sen

1999

10

00

10

10

00

0-1

01

01

00

01

019

Kni

llan

dL

ehm

kuhl

2002

10

01

00

10

01

00

00

-10

10

10

020

Lad

rech

2002

10

01

00

00

-11

00

01

00

10

10

021

Lav

enex

2001

10

00

10

01

00

0-1

01

00

10

01

022

Lev

i-Fa

ur20

040

10

01

00

0-1

01

01

00

10

00

10

23L

ippe

rt,U

mba

ch,W

esse

ls20

011

00

10

01

00

10

01

00

01

01

00

24L

odge

2000

01

01

00

00

-11

00

00

-11

00

10

025

Mar

cuss

en,R

isse

etal

.199

91

00

01

00

10

01

01

00

01

00

10

26R

adae

lli19

971

00

01

00

10

01

00

10

01

01

00

27Sc

harp

f19

970

10

01

00

0-1

10

00

10

10

01

00

28Sc

him

mel

fenn

igan

dSe

delm

eier

2004

10

00

10

10

01

00

01

00

10

10

029

Schm

idt

2002

10

01

00

10

01

00

01

00

10

10

030

Sem

etko

;D

eV

rees

e;Pe

ter

J20

000

10

01

01

00

10

00

10

01

00

10

31Sm

ithJ

2001

01

00

10

10

00

10

10

00

0-1

01

032

War

leig

hA

2001

10

00

10

00

-11

00

01

00

0-1

01

0T

OTA

LO

BSE

RV

AT

ION

S(N

=32

)16

160

923

021

56

1612

49

194

1215

518

140

Sour

ce:A

utho

rs’

own

data

.



Tabl

e7:

Scor

ecar

dof

the

Con

trol

Gro

upSa

mpl

e

IDA

utho

r(y

ear)

Trad

e-of

fsin

caus

alan

alys

is

12

45

67

Causeofeffects

Effectsofcauses

N/A

Concept

Measurement

N/A

Complexcausation

Singularcausation

N/A

Parsimony

Richsetofindepvariables

N/A

Time-quantity

Time-quality

N/A

Mechanism-oriented

Variable-oriented

N/A

1A

lter

1998

01

00

10

10

00

10

10

01

00

2A

nder

son

CJ

1998

01

00

10

01

00

10

10

00

10

3B

oyle

2000

10

00

10

10

00

10

01

01

00

4C

arru

bba

2001

10

00

10

01

00

10

10

00

10

5C

icho

wsk

i20

001

00

01

01

00

01

01

00

01

06

Col

ean

dD

rake

2000

01

01

00

10

00

10

01

00

10

7Fr

iisan

dM

urph

y19

990

10

01

01

00

01

00

10

10

08

Gab

el19

981

00

01

00

10

01

01

00

01

09

Gar

rett,

Kel

emen

,Sch

ulz

1998

10

01

00

01

01

00

01

01

00

10H

aas

PM19

981

00

01

01

00

01

01

00

01

011

Hix

1999

10

00

10

10

00

10

10

00

10

12H

odso

nan

dM

aher

2001

01

01

00

01

01

00

10

01

00

13H

uysm

ans

2000

01

00

10

10

01

00

10

01

00



14K

eatin

g19

970

10

01

00

0-1

01

00

10

01

015

Lei

tner

1997

10

00

10

10

00

10

01

00

10

16M

aste

nbro

ek20

030

10

01

00

10

01

00

10

01

017

McL

aren

LM

2002

10

00

10

01

00

10

00

-10

10

18M

eyer

C19

991

00

01

00

10

10

01

00

01

019

Paar

lber

g19

971

00

01

01

00

10

00

10

01

020

Pahr

e19

970

10

01

00

10

01

01

00

10

021

Papp

ian

dH

enni

ng19

991

00

01

00

10

01

00

0-1

01

022

Ris

se,E

ngel

man

n-M

artin

,eta

l.19

990

10

01

01

00

10

00

10

01

023

Schi

mm

elfe

nnig

2000

10

01

00

10

01

00

01

01

00

24Se

lck

2004

01

01

00

00

-10

10

10

00

10

25Se

lck

and

Steu

nenb

erg

2004

01

00

10

01

01

00

00

-11

00

26Sm

ithA

2002

10

00

10

10

00

10

01

00

0-1

27Sm

ithM

E20

000

10

10

00

0-1

10

00

10

10

028

Span

ou19

980

10

01

00

10

10

01

00

10

029

Tagg

art

1998

10

00

10

01

00

10

10

00

0-1

30Ta

llber

g20

020

0-1

10

00

0-1

01

01

00

10

031

Ver

dun

1999

10

01

00

01

01

00

10

00

0-1

32V

ogel

2003

10

00

10

10

01

00

01

00

10

TO

TAL

OB

SER

VA

TIO

NS

(N=

32)

1714

18

240

1414

412

200

1613

312

173

Sour

ce:A

utho

rs’

own

data

.



discussing outcomes (that is, whether country A is Europeanized or not) orissues of research design at a fairly general level. Surveys of the literaturehave not used basic quantitative tools such as meta-analysis and systematicreviews. With this article, instead, we provide a synthetic overview of whatgoes on in the field in terms of trade-offs in causal analysis.

We addressed three issues: what is the level of awareness of researchdesign in Europeanization? What are the methodological choices made by thescholars in this field? And finally, is there any original contribution thatEuropeanization can make to the research design debate?

Awareness of research design issues is still low in the field of European-ization. As coders, one of our major problems was to find out just how anauthor had gone about the trade-offs, since in most cases awareness of thistype of choice was not evident. Hence in most cases we had to infer thechoices made by the author in terms of causal analysis from the substantivesections of the article.

Europeanization scholars do not systematically prefer cause-of-effectsanalysis to effects-of-causes. Measurement features more prominently thanconceptual development. Complex notions of causality prevail in European-ization but not in the control group – the difference, however, is slight. On thetrade-off between parsimony and rich set of variables, Europeanization schol-ars go for the latter. They are also engaged with time making qualitativeimpacts on politics and public policy. Finally, the design shows slightpreference for mechanism-oriented analysis.

To what extent are these features unique to Europeanization? Or do theyreflect general patterns in the study of EU politics? Europeanization articlesdiffer from the control group in the focus on mechanisms (rather than vari-ables) and the qualitative aspects of time in politics. Complex notions ofcausality prevail in Europeanization but not in the control group and thecause-of-effects approach is preferred to effects-of-causes in the controlgroup but not in Europeanization – in both cases, however, the difference isslight.

The methodological discussions within Europeanization have generated afew innovative ideas, such as the insights on mechanisms through whichcausality works; how to measure the net impact by looking at control groupsand test cases from outside the EU; and the notion of bottom-up researchdesign. However, the vast majority of scholars work with rather standardtop-down designs.

Future work will have to establish if the trade-offs are independent or ifsome of them are correlated. Our analysis suggests that they are not correlatedand do not represent manifestations of a more fundamental fracture betweenquantitative methods and qualitative approaches.



Correspondence:Theofanis Exadaktylos and Claudio M. RadaelliDepartment of PoliticsAmory BuildingRennes DriveUniversity of ExeterEX4 4RJ, UKTel + 44 (0) 1392 263176Fax + 44 (0) 1392 263305email [email protected]; [email protected]

References

Börzel, T.A. (2005) ‘Mind the Gap! European Integration between Level and Scope’.Journal of European Public Policy, Vol. 12, No. 2, pp. 217–36.

Börzel, T.A. and Risse, T. (2003) ‘Conceptualizing the Domestic Impact’. In Feath-erstone, K. and Radaelli, C. (eds) The Politics of Europeanization (Oxford:Oxford University Press).

Brady, H. and Collier, D. (2004) Rethinking Social Inquiry: Diverse Tools, SharedStandards (Oxford: Rowman & Littlefield).

Brady, H., Collier, D. and Seawright, J. (2006) ‘Towards a Pluralistic Vision ofMethodology’. Political Analysis, Vol. 14, p. 353.

Bulmer, S. (2009) ‘Politics in Time Meets the Politics of Time: Historical Institution-alism and the EU Timescape’. Journal of European Public Policy, forthcoming.

Bulmer, S. and Radaelli, C. (2005) ‘The Europeanization of National Policy’. InBulmer, S. and Lequesne, C. (eds) The Member States of the European Union(Oxford: Oxford University Press).

Caporaso, J.A. (2007) ‘The Three Worlds of Regional Integration Theory’. In Gra-ziano, P. and Vink, M.P. (eds) Europeanization: New Research Agendas (Basing-stoke: Palgrave/Macmillan).

Dyson, K. (2000) ‘EMU as Europeanization: Convergence, Diversity and Contin-gency’. JCMS, Vol. 38, No. 4, pp. 645–66.

Dyson, K. (2002) (ed.) European States and the Euro (Oxford: Oxford UniversityPress).

Elmore, R.F. (1999 [1982]) ‘Background Mapping: Implementation Research andPolicy Decisions’. In Miyakawa, T. (ed.) The Science of Public Policy (New York:Routledge).

EPPIE (2007) ‘Analyser l’européanisation des politiques publiques’. In Palier, B. andSurel, Y. (eds) L’Europe en action: L’Européanisation dans une perspectivecomparée (Paris: L’Harmattan).

Franchino, F. (2005) ‘The Study of EU Public Policy – Results of a Survey’. Euro-pean Union Politics, Vol. 6, No. 2, pp. 243–52.

Gerring, J. (2001) Social Science Methodology: A Criterial Framework (Cambridge:Cambridge University Press).



mailto:[email protected]

mailto:[email protected]

Goetz, K. (2006) ‘Temporality and the European Administrative Space’. Paper pre-sented at the CONNEX Thematic Conference ‘Towards a European Administra-tive Space’, London, 16–18 November.

Graziano, P. and Vink, M.P. (2007) Europeanization: New Research Agendas(Basingstoke: Palgrave/Macmillan).

Hall, P.A. (2003) ‘Aligning Ontology and Methodology in Comparative Research’. InMahoney, J. and Rueschmeyer, D. (eds) Comparative Historical Analysis in theSocial Science (Cambridge: Cambridge University Press).

Hassenteufel, P. and Surel, Y. (2000) ‘Des politiques publiques comme les autres?Construction de l’objet et outils d’analyse des politiques européennes’. Politiqueeuropéenne, Vol. 1, pp. 8–24.

Haverland, M. (2005) ‘Does the EU Cause Domestic Developments? The Problem ofCase Selection in Europeanization Research’. European Integration OnlinePapers (EIoP), Vol. 9, No. 2.

Haverland, M. (2007) ‘Methodology’. In Graziano, P. and Vink, M.P. (eds) Europe-anization: New Research Agendas (London: Palgrave/MacMillan).

Héritier, A., Kerwer, D., Knill, C., Lehmkuhl, D. and Teutsch, M. (2001) DifferentialEurope – New Opportunities and Restrictions for Policy Making in MemberStates (Lanham, MD: Rowman & Littlefield).

Kassim, H. (2005) ‘The Europeanization of Member State Institutions’. In Bulmer,S. and Lequesne, C. (eds) The Member States of the European Union (Oxford:Oxford University Press).

King, G., Keohane, R.O. and Verba, S. (1994) Designing Social Inquiry: ScientificInference in Qualitative Research (Princeton, NJ: Princeton UniversityPress).

Knill, C. and Lehmkuhl, D. (2002) ‘The National Impact of European Union Regu-latory Policy: Three Europeanization Mechanisms’. European Journal of Politi-cal Research, Vol. 41, No. 2, pp. 255–80.

Krippendorf, K. (2004) Content Analysis: An Introduction to its Methodology(London: Sage Publications).

Ladrech, R. (2005) ‘The Europeanization of Interest Groups and Political Parties’. InBulmer, S. and Lequesne, C. (eds) The Member States of the European Union(Oxford: Oxford University Press).

Lenschow, A. (2005) ‘Europeanization of Public Policy’. In Richardson, J.J. (ed.)European Union: Power and Policy-Making (Abingdon: Routledge).

Levi-Faur, D. (2004) ‘On the “Net Impact” of Europeanization: the EU’s Telecomand Electricity Regimes between the Global and the National’. ComparativePolitical Studies, Vol. 37, No. 1, pp. 3–29.

Lipsey, M.W. and Wilson, D.B. (2001) Practical Meta-Analysis (Thousand Oaks,CA: Sage Publications).

Mahoney, J. and Goertz, G. (2006) ‘A Tale of Two Cultures: Contrasting Quantitativeand Qualitative Research’. Political Analysis, Vol. 14, pp. 227–49.

Olsen, J.P. (1995a) ‘European Challenges to the Nation-State’. ARENA, WorkingPaper No. 14, University of Oslo.



Olsen, J.P. (1995b) ‘Europeanization and Nation-State Dynamics’. ARENA,Working Paper No. 9, University of Oslo.

Olsen, J.P. (2002) ‘The Many Faces of Europeanization’. JCMS, Vol. 40, No. 5, pp.921–52.

Pierson, P. (1996) ‘The Path to European Integration. A Historical InstitutionalAnalysis. Comparative Political Studies, Vol. 29, No. 2, pp. 123–63.

Pierson, P. (2004) Politics in Time: History, Institutions and Social Analysis(Princeton, NJ: Princeton University Press).

Radaelli, C. (2003) ‘The Europeanization of Public Policy’. In Featherstone, K. andRadaelli, C. (eds) The Politics of Europeanization (Oxford: Oxford UniversityPress).

Radaelli, C. and Pasquier, R. (2007) ‘Conceptual Issues’. In Vink, M.P. and Graziano,P. (eds) Europeanization: New Research Agendas (Basingstoke: Palgrave/Macmillan).

Ragin, C. (1987) The Comparative Method: Moving Beyond Qualitative and Quan-titative Strategies. (Berkeley, CA: University of California Press).

Ragin, C. (2000) Fuzzy-Set Social Science (Chicago, IL: University of ChicagoPress).

Saurugger, S. (2005) ‘Europeanization as a Methodological Challenge: The Case ofInterest Groups’. Journal of Comparative Policy Analysis, Vol. 7, No. 4, pp.291–312.

Schimmelfennig, F. (2007) ‘European Regional Organizations, Political Condition-ality and Democratic Transformation in Eastern Europe. East European Politicsand Societies, Vol. 21, No. 1, pp. 126–41.

Schmidt, V.A. (2002) ‘Europeanization and the Mechanics of Economic PolicyAdjustment’. Journal of European Public Policy, Vol. 9, No. 6, pp. 894–912.

Schmidt, V.A. (2006) ‘Give Peace a Chance: Reconciling Four (Not Three) “NewInstitutionalisms” ’. Annual Meeting of the American Political Science Associa-tion. Marriot, Lowes Philadelphia and the Pennsylvania Convention Center,Philadelphia, PA.

Sedelmeier, U. (2006) ‘Europeanisation in New Member and Candidate States’.Living Reviews in European Governance, Vol. 1, No. 3.

Stolfi, F. (2008) ‘The Europeanization of Italy’s Budget Institutions in the 1990s’.Journal of European Public Policy, Vol. 15, No. 4, pp. 550–6.

Supporting Information

Additional Supporting Information may be found in the online version of thisarticle:

Table S1. List of most cited articles on Europeanization (cut-off point at 5citations)Table S2. List of most cited articles on European Union and Politics (cut-offpoint at 12 citations)



Table S3. List of cases in the Europeanization Sample (N = 32)Table S4. List of cases in the Control Group sample (N = 32)Table S5. Comparative table of the two scorecardsTable S6. Correlations matrix for Europeanization sampleTable S7. Scorecard of the Europeanization sampleTable S8. Scorecard of the Control Group sample

Please note: Wiley-Blackwell are not responsible for the content or function-ality of any supporting materials supplied by the authors. Any queries (otherthan missing material) should be directed to the corresponding authors for thearticle.



Documents

raedelli