24
1 Qualitative Methods John Gerring Department of Political Science Boston University [email protected] Forthcoming: Annual Review of Political Science 20 (May/June 2017) Draft: 1 June 2016 Estimated pages: 23 (http://www.annualreviews.org/page/authors/article-length-estimator-1) Comments welcome! Please do not cite without permission

Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

Embed Size (px)

Citation preview

Page 1: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

1

Qualitative Methods

John Gerring Department of Political Science

Boston University [email protected]

Forthcoming:

Annual Review of Political Science 20 (May/June 2017)

Draft: 1 June 2016

Estimated pages: 23 (http://www.annualreviews.org/page/authors/article-length-estimator-1)

Comments welcome!

Please do not cite without permission

Page 2: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

2

Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis (however that might be dated).1 Selfconscious reflection on those methods, however, is comparatively recent. The first methodological statements of contemporary relevance grew out of the work of logicians, philosophers, and historians in the nineteenth century, most importantly J.S. Mill (1843). To be sure, these scholars were in a quest for science, understood as a unified venture. So the notion of a method that applies only to qualitative data would have made little sense to them.

At the turn of the twentieth century, a bifurcation appeared between quantitative and qualitative methods (Platt 1992). The natural sciences, along with economics, moved fairly quickly and without much fuss into the quantitative camp, while the humanities remained largely qualitative in orientation. The social sciences found themselves in the middle – divided between scholars aligned with each camp, and some who embraced both. For this reason, the qual/quant distinction has assumed considerable importance in these fields, and very little importance outside these fields.

Perhaps it is not coincidental that the quest for a method to qualitative inquiry has proceeded further in the social sciences than in the humanities. And among the social sciences one might argue that political science has gone further than any other in developing the field of qualitative methods. Accordingly, this review article focuses primarily on work produced by political scientists, with an occasional glance at neighboring disciplines.

I begin by discussing the time-honored qualitative/quantitative distinction. What is qualitative data and analysis and how does it differ from quantitative data and analysis? I propose a narrow definition for “qualitative” and then explore the implications of that definition. I also explore in a speculative vein some of the factors underlying the ongoing Methodenstreit between scholars who identify with quantitative and qualitative approaches to social science. In the remainder of the article I discuss areas of qualitative research that seem especially fecund, judging by output over the past decade. This includes case-selection, causal inference, and multimethod research.

In treating these subjects I try to represent the current state of the field. However, representing every position in every debate would not be possible in the short space of this review. There are simply too many ways to cut the cake. Instead, I endeavor to reduce the material in a fashion that incorporates work conducted by many scholars but – inevitably – imposes my own views on the subject matter.

Even so, many vitally important subjects are neglected in this short review. I do not address concept formation (Collier & Gerring 2009; Goertz 2005), typological methods (Collier, LaPorte & Seawright 2012; Elman 2005; George & Bennett 2005), set theory and qualitative comparative analysis (Mahoney 2010; Mahoney & Vanderpoel 2015; Rihoux 2013), data archiving, transparency, and replication (Elman & Kapiszewski 2014; Elman, Kapiszewski & Vinuela 2010; Lieberman 2010), comparative historical analysis (Mahoney & Thelen 2015), path dependence (Bennett & Elman 2006a; Boas 2007; Page 2006), the organizational features of qualitative methods (Collier & Elman 2008), interpretivism and ethnography (Schatz 2009; Yanow & Schwartz-Shea 2013), or other methods of data collection grouped together under the rubric of field research (Kapiszewski, MacLean & Read 2015). Fortunately, these topics are amply covered in recent work, as the foregoing citations attest. I should also signal that the following discussion pertains mostly to causal inference, leaving aside many knotty questions pertaining to descriptive inference (Gerring 2012).

Having acknowledged my biases and omissions – especially important in a review focused on a subject as contested as qualitative methods – let us begin.

1 I am grateful to Colin Elman, Evan Lieberman, Jim Mahoney, and David Waldner for comments and suggestions on this manuscript.

Page 3: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

3

Qual and Quant

Although the qual/quant distinction is ubiquitous in social science today, the distinction is viewed differently by scholars identified with each camp. As a rule, scholars whose work is primarily quantitative tend to view social science as a unified endeavor, following similar rules and assumptions. The naturalistic ideal centers on goals such as replication, cumulation, and consensus – all of which point toward a single logic of inference (Beck 2006, 2010; King, Keohane & Verba 1994).

By contrast, scholars whose work is primarily qualitative tend to view the two modes of inquiry as distinctive, perhaps even incommensurable. They are more likely to believe that knowledge of the world is embedded in theoretical, epistemological, or ontological frameworks from which we can scarcely disentangle ourselves. They may also identify with the phenomenological idea that all human endeavor, including science, is grounded in human experience. Since experiences – inevitably couched in positions of differential power and status – vary, one can reasonably expect that the methods and goals of social science might also vary. The apparent embeddedness of knowledge reinforces qualitative scholars’ predilection toward pluralism, as it suggests that there are fundamentally – and legitimately – different ways of going about business (Ahmed & Sil 2012; Bennett & Elman 2006b: 456-57; Goertz & Mahoney 2012; Hall 2003; Mahoney & Goertz 2006; Shapiro, Smith & Masoud 2004; Sil 2000; Yanow & Schwartz-Shea 2013).

Following the axiom that where one sits determines where one stands, we must also consider the stakes in this controversy. Over the past century, quantitative work has been on the ascendant and qualitative work has been cast in a defensive posture. Qualitative researchers are at pains to explain their work in ways that those in the quantitative tradition can understand. Uncomfortable with the prospect of absorption into a “quantitative template,” one may surmise that many qualitative scholars have sought to emphasize the distinctiveness of what they do for strategic reasons – establishing a nature preserve for endangered species, as it were.

Whatever its intellectual and sociological sources, the question of unity or dis-unity depends upon how one chooses to define similarity and difference. Any two objects will share some characteristics and differ in others. It follows that they may be either compared or contrasted, depending upon the author’s point of view. Quantitatively inclined scholars may choose to focus on similarities while qualitatively inclined scholars choose to focus on differences. Both are correct, as far as they go. The half-empty/half-full conundrum seems difficult to overcome in this particular context.2 To put the matter in a more specific frame: most political scientists probably agree with Brady & Collier (2010) that there are “diverse tools” (the pluralistic angle) as well as “shared standards” (the monist angle).3 But they do not necessarily agree on what those shared standards are or to what extent they should discipline the work of social science.

Any attempt to resolve the monism/pluralism question that begins with high-level concepts (e.g., monism and pluralism, logic of inquiry, epistemology, commensurability, naturalism, interpretivism) is probably doomed to failure. These words are loaded, and once they have been uttered the die is cast. Those who identify with either camp are likely to dig in their heels.

I propose, therefore, to take a ground-level approach that seeks to avoid diffuse – and loaded – concepts from philosophy of science, focusing instead on matters of definition. What, exactly, is qualitative data? And what, by contrast, is quantitative data? We shall then explore the

2 This is nicely illustrated in recent arguments about causation (Reiss 2009). 3 A more radical pluralist view, associated with post-structuralism (Rosenau 1992)), denies the existence of shared standards. I suspect that few political scientists hold that view.

Page 4: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

4

repercussions of this distinction, working toward some tentative conclusions which may resolve some (though not all) aspects of the qual/quant debate.

Definitions Qualitative and quantitative are usually understood as antonyms. The resulting polar concepts may be viewed as a continuum (a matter of degrees) or as a set of crisp concepts (with clear-cut boundaries). In either case, the two terms are defined in opposition to each other. Let us consider some of the attributes commonly associated with these contrasting approaches.

Qualitative work is expressed in natural language while quantitative work is expressed in numbers and in statistical models. Qualitative work employs small samples, while quantitative work is large-n. Qualitative work draws on cases chosen in an opportunistic or purposive fashion while quantitative work employs systematic (random) sampling. Qualitative work is often focused on particular individuals, events, and contexts, lending itself to an idiographic style of analysis. Quantitative work is more likely to be focused on features that (in the researcher’s view) can be generalized across a larger population, lending itself to a nomothetic style of analysis.

I shall suppose that all of the foregoing contrasts contain some truth; that is, they describe patterns found in the work of social scientists, even if there are many exceptions. And let us further suppose that they resonate with common usage of these terms, as reflected in extant work on the subject (e.g., Bennett & Elman 2006b; Brady 2010; Caporaso 2009; Collier & Elman 2008; Glassner & Moreno 1989; Goertz & Mahoney 2012; Hammersley 1992; King, Keohane & Verba 1994; McLaughlin 1991; Levy 2007; Morgan 2012; Patton 2002; Schwartz & Jacobs 1979; Shweder 1996; Snow 1959/1993; Strauss & Corbin 1998). If so, we have usefully surveyed the field. But we have not provided anything more than a semantic map of this rugged terrain. And because the foregoing attributes are multidimensional, the subject remains elusive. We cannot bring methodological clarity to it because “it” remains ambiguous.

My goal is to arrive at a minimal definition that bounds our subject in a fairly crisp fashion, that resonates with extant understandings (subsuming many of the meanings contained in the passage above), and that does not trespass on other well-established terms. (It would be inefficient, semantically speaking, to conflate qualitative with idiographic, ethnographic, or some other term in this family of concepts.) In addition, it would be helpful if the proffered definition accounts for (in a loosely causal sense) the various attributes commonly associated with the terms “qualitative” and “quantitative” as surveyed above.

With these goals in mind, I propose that the defining feature of qualitative work is its use of non-comparable observations – observations that pertain to different aspects of a causal or descriptive question. As an example, one may consider the clues in a typical detective story. One clue concerns the suspect’s motives; another concerns his location at the time the crime was committed; a third concerns a second suspect; and so forth. Each observation, or clue, draws from a different population. This is why they cannot be arrayed in a matrix (rectangular) dataset and must be dealt with in prose (aka narrative analysis). It is also why we have difficulty counting such observations. The time-honored question of quantitative research – What is the n? – is impossible to answer in a definitive fashion. Likewise, styles of inference based on qualitative data operate somewhat differently than styles of inference based on quantitative data.

I therefore define quantitative observations as comparable (along whatever dimensions are relevant) and qualitative observations as non-comparable, regardless of how many there are. When qualitative observations are employed for causal analysis they may be referred to as causal-process observations (Brady 2010), though I shall continue to employ the more general (and less bulky) term, qualitative observation, which applies to both descriptive and causal inferences.

The notion of a qualitative or quantitative analysis is, accordingly, an inference that rests on one or the other sort of data. If the work is quantitative, it enlists patterns of covariation

Page 5: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

5

found in a matrix of observations and analyzed with a formal model (e.g., set theory/QCA, frequentist statistics, Bayesian probabilities, randomization inference) to reach a descriptive or causal inference. If the work is qualitative, the inference is based on bits and pieces of non-comparable observations that address different aspects of a problem. Traditionally, these are analyzed in an informal fashion, an issue taken up below.

Some strategies of data collection seem inherently qualitative, e.g., unstructured interviews, participant-observation (ethnography), and archival work. This is because researchers are likely to incorporate a wide variety of clues drawn from different kinds of sources and addressing different aspects of a problem. The different-ness of the evidence makes them non-comparable, and hence qualitative. Other data collection strategies such as standardized surveys are inherently quantitative, as they involve counting large numbers of observations that are comparable by assumption. Of course, they might not actually be comparable. We are speaking here of assumptions about the data generating process, not about the truth with a capital T. But we cannot avoid assumptions about the world, and these assumptions – quite rightly – lead researchers to adopt one or the other method of apprehending that reality.

Before moving on I want to call attention to the fact that the proposed definition of qualitative/quantitative offered here imposes a somewhat narrower purview on the subject than is common in everyday speech. It encompasses much of what scholars refer to as qualitative, but not everything. It excludes qualitative comparative analysis, for example. Note that QCA utilizes multiple observations that are comparable to each other, as suggested by the matrix format that underlies QCA algorithms (crisp, fuzzy, and so forth). It runs orthogonal to the notion of a “qualitative” level of measurement, i.e., a binary or ordinal scale. Qualitative, as the term is used here, refers to a type of data and analysis, not a type of scale.

Other contrasts might be drawn with extant usages of “qualitative/quantitative,” which is extremely loose. The point is, in order to say anything about our subject we need to circumscribe it. And in circumscribing it we necessarily include some phenomena and exclude others. There is no getting around the somewhat arbitrary act of definition. Let us now turn to the payoff: What might we learn about qualitative (and quantitative) research when defined in this manner?

Converting Words to Numbers No qualitative observation is immune from quantification. Interviews, pictures, ethnographic notes, and texts drawn from other sources may be coded, either through judgments exercised by coders or through mathematical algorithms (Grimmer & Stewart 2013). By coding I refer to the systematic measurement of the phenomenon at hand – reducing the information at hand to a small number of dimensions, consistently defined across the units of interest. All that is required, following our definition, is that multiple observations of the same kind be produced and (voila!) quantitative observations are born. These may then be represented in the matrix format familiar to those who work with rectangular datasets.

Of course, there are often practical obstacles to quantification. Perhaps additional sources (informants, pictures, texts) are unavailable. Perhaps, if available, they are not really comparable, or they introduce problems of causal identification (e.g., heterogeneity across cases that could pose a problem of noise or confounding). Alternatively, it may be possible to generate additional (comparable) observations but not worthwhile, e.g., because the first observation is sufficient to prove the point at issue. Sometimes, one clue is decisive. Nonetheless, in principle, if the researcher’s assumptions of comparability are justified, qualitative data can become quantitative data. The plural of anecdote is data.

Something is generally lost in the process of reducing qualitative information to quantitative data. One must ignore the unique aspects of each qualitative observation in order render them comparable. If one wishes to generalize across a population, ignoring idiosyncratic features of the data is desirable. But if one wishes to shed light on these heterogeneous features

Page 6: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

6

the conversion of qualitative to quantitative data will iron out the ruggedness of the landscape – obscuring variation of theoretical interest. Information loss must be reckoned with.4

Finally, and perhaps most importantly, there is an asymmetry between qual and quant. One can convert qualitative data to quantitative data but not the reverse. It is a one-way street. Once a piece of information is rendered in a matrix template whatever unique aspects may have adhered to that observation have been lost. Data reduction is possible, but not expansion. The singular of data is not anecdote, which is to say one can never recover an anecdote from a data point.

Contrasting Affinities It follows from our discussion that the utility of qualitative and quantitative data varies according to the researcher’s goals, i.e., whether the research is exploratory and whether it is case-based. Other features of a theory or an analysis do not seem to have a direct bearing on the relative utility of these varying approaches.5

First, qualitative data is likely to be more important at early stages of research, when not much is known about a subject or when the goal is to uncover a new angle or hypothesis. Qualitative data are ideal for exploratory analysis, and often indispensable. Arguably, social science knowledge begins at a qualitative level and then (sometimes) proceeds to a quantitative level. This is implicit in the notion that data can be converted from qual to quant, but not the reverse.

Granted, the qualitative component of research sometimes follows the quantitative component, as when a cross-case study is combined with a case-based investigation of causal mechanisms. Here, the causal effect may be well-understood but not the pathway(s) by which X is connected to Y. Again, case-based investigation plays an exploratory role.

Second, qualitative data is likely to be more useful insofar as a study is focused on a single case (or event), or a small number of cases (or events). Such investigations bear close resemblance, methodologically speaking, to a detective’s quest to explain a crime, which may be thought of as a single event or a small number of associated events (if it is a string of crimes committed by the same person or group). The reason that these investigations often rest on qualitative data is that the researcher wishes to know a lot about the chosen case/event, and this requires a supple mode of investigation that allows one to draw different kinds of observations from different populations.

Whether case-level analysis is warranted may rest on other, more fundamental aspects of the analysis. For example, case-level analysis is more plausible if the cases of theoretical interest are heterogeneous and scarce (e.g., nation-states) rather than homogeneous and plentiful (e.g., firms or individuals), if the causal factor cannot be manipulated by the researcher, if the causal factor or outcome is extremely rare, if the theory is focused on a single case or a small set of cases, and so forth.

4 Of course, any rendering of a complex phenomenon involves some loss of information. This is true even for the most faithful – and detailed – descriptions of reality such as those produced by ethnomethodologists (Garfinkel 1967). 5 For example, the long-standing distinction between research that seeks a complete explanation of an outcome (“causes-of-effects”) and research that narrows its scope to a single hypothesis (“effects-of-causes”) seems to bear ambivalently on the qual/quant divide. Note that a causes-of-effects explanation may be provided solely on the basis of quantitative data, e.g., a “full” regression model. Likewise, an effects-of-causes explanation may be provided based solely on qualitative data, i.e., a process tracing analysis).

Page 7: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

7

Case Selection

We have observed that case-based analysis is likely to contain qualitative observations (even if it also incorporates quantitative observations). Consequently, the question of case-selection – how a case, or a small number of cases, is chosen from a large number of potential cases – is central to qualitative analysis.

Quite a number of case-selection typologies have been proposed over the years, with a noticeable acceleration in the past decade. Mill (1843/1872) proposes the method of difference (aka most-similar method) and method of agreement (aka most-different method), along with several others that have not gained traction. Lijphart (1971: 691) proposes six case study types: a-theoretical, interpretative, hypothesis-generating, theory-confirming, theory-infirming, and deviant. Eckstein (1975) identifies five species: configurative-idiographic, disciplined-configurative, heuristic, plausibility probes, and crucial-case. Skocpol & Somers (1980) identify three logics of comparative history: macro-causal analysis, parallel demonstration of theory, and contrast of contexts. Gerring (2007) and Seawright & Gerring (2008) identify nine techniques: typical, diverse, extreme, deviant, influential, crucial, pathway, most-similar, and most-different. Levy (2008) identifies five case study research designs: comparable, most and least likely, deviant, and process tracing. Rohlfing (2012: ch3) identifies five case-types – typical, diverse, most-likely, least-likely, and deviant – which are applied differently according to the purpose of the case study. Blatter & Haverland (2012: 24-26) identify three explanatory approaches – covariational, process tracing, and congruence analysis – each of which offers a variety of case-selection strategies.

Building on these efforts, Gerring & Cojocaru (2016) propose a new typology that (arguably) qualifies as the most comprehensive to date, incorporating much of the foregoing literature. Its organizing feature is the goal that a case study is intended to serve, identified in the first column of Table 1. Column 2 specifies the number of cases (n) in the case study. It will be seen that case studies enlist a minimum of one or two cases, with no clearly defined ceiling (though at a certain point the defining goal of a case study – intensive analysis of a case – becomes dubious). Column 3 clarifies which dimensions of the case are relevant for case-selection, i.e., descriptive features (D), causal factors of theoretical interest (X), background factors (Z), and/or the outcome (Y). Column 4 specifies the criteria used to select a case(s) from a universe of possible cases. Column 5 offers an example of each case-selection strategy. In what follows, I offer a brief resume of the resulting typology.

Page 8: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

8

Table 1: Case-Selection Strategies

Goals/Strategies N Factors Criteria for cases Examples

I. DESCRIPTIVE (to describe)

● Typical 1+ D Mean, mode, or median of D Lynd & Lynd (1929) Middletown ● Diverse 2+ D Typical sub-types Fenno (1977, 1978) Home Style

II. CAUSAL (to explain Y)

1. Exploratory (to identify HX) ● Extreme 1+ X or Y Maximize variation in X or Y Skocpol (1979) States and Social Revolutions ● Index 1+ Y First instance of ∆Y Pincus (2011) 1688: First Modern Revolution ● Deviant 1+ Z Y Poorly explained by Z Alesina et al (2001) Why Doesn’t US Have Welfare State? ● Most-similar 2+ Z Y Similar on Z, different on Y Epstein (1964) A Comparative Study of Canadian Parties ● Most-different 2+ Z Y Different on Z, similar on Y Karl (1997) Paradox of Plenty ● Diverse 2+ Z Y All possible configurations of Z (assumption: X ∈ 𝒁) Moore (1966) Social Origins of Dictatorship and Democracy

2. Estimating (to estimate HX)

● Longitudinal 1+ X Z X changes, Z constant or biased against HX Friedman & Schwartz (1963) Monetary History of US ● Most-similar 2+ X Z Similar on Z, different on X Posner (2004) Political Salience of Cultural Difference

3. Diagnostic (to assess HX)

● Influential 1+ X Z Y Greatest impact on P(HX) Ray (1993) Wars between Democracies ● Pathway 1+ X Z Y X→ 𝑌 strong, Z constant or biased against HX Mansfield & Snyder (2005) Electing to Fight ● Most-similar 2+ X Z Y Similar on Z, different on X & Y, X→ 𝑌 strong Walter (2002) Committing to Peace D = descriptive features (other than those to be described in a case study). H X = causal hypothesis of interest. P(H X) = the probability of HX. X = causal factor(s) of theoretical interest. X→ 𝑌 = apparent or estimated causal effect. Y = outcome of interest. Z = vector of background factors that may affect X and/or Y.

Page 9: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

9

Many case studies are primarily descriptive, which is to say they are not organized around a central, overarching causal hypothesis. Although writers are not always explicit about their selection of cases, most of these decisions might be described as following a typical or diverse case strategy. That is, they aim to identify a case, or cases, that exemplify a common pattern (typical) or patterns (diverse). This follows from the minimal goals of descriptive analysis. Where the goal is to describe there is no need to worry about the more complex desiderata that might allow one to gain causal leverage on a question of interest. Other case studies are oriented toward causal analysis. A good case (or set of cases) for purposes of causal analysis is generally one that exemplifies quasi-experimental properties, replicating the virtues of a true experiment even while lacking a manipulated treatment (Gerring & McDermott 2007). Specifically, for a given case (observed through time) or for several cases (compared to each other), variation in X should not be correlated with other factors that are also causes of Y, which might serve as confounders (Z), generating a spurious (non-causal) relationship between X and Y.

Exploratory case studies aim to identify a hypothesis. Sometimes, the researcher begins with a factor that is presumed to have fundamental influence on a range of outcomes. The research question is, what outcomes (Y) does X affect? More commonly, the researcher works backward from a known outcome to its possible causes. The research question is, therefore, what accounts for variation in Y? Or, if Y is a discrete event, Why does Y occur? The researcher may also have an idea about background conditions, Z, that influence Y but are not of theoretical interest. The purpose of the study, in any case, is to identify X, regarded as a possible or probable cause of Y. Specific exploratory techniques may be classified as extreme, index, deviant, most-different, most-similar, or diverse, as specified in Table 1.

Estimating cases aim to test a hypothesis by estimating a causal effect. That might mean a precise point estimate along with a confidence interval (e.g., from a time-series or synthetic matching analysis), or an estimate of the “sign” of a relationship, i.e., whether X has a positive, negative, or no relationship to Y. The latter is more common, not only because of the small size of the sample (at the case level) but also because it is more likely to be generalizable across a population of cases. In either situation, case selection rests on information about X and Z (not Y). Two general approaches are viable – longitudinal and most similar – as outlined in Table 1.

Diagnostic case studies help to confirm, disconfirm, or refine a hypothesis (garnered from the literature on a subject or from the researcher’s own ruminations) and identify the generative agent (mechanism) at work in that relationship. All the elements of a causal model – X, Z, and Y – are generally involved in the selection of a diagnostic case. Specific strategies may be classified as influential, pathway, or most-similar, as shown in Table 1.

Note that virtually all of these case selection strategies may be executed in an informal, qualitative fashion or by employing a quantitative algorithm. For example, a deviant case could be chosen based on a researcher’s sense about which case(s) is poorly explained by extant theories. Or it might be chosen by looking at residuals from a regression model. Discussion of the pros and cons of algorithmic case selection can be found in Gerring (2017).

Validation The reader may wonder, how does one know whether a designated strategy will achieve what it is intended to achieve? Given a research goal, which is the best way to choose cases? Why these strategies (listed in Table 1) and not others? Evidently, there are serious problems of validation to wrestle with.

With this nagging question in mind, several attempts have been made to assess varying case selection strategies using simulation techniques. Herron & Quinn (2016) assess estimating strategies, i.e., where the case is intended to measure causal effects. Seawright (2016) assesses diagnostic strategies, where the case is designed to help confirm or disconfirm a causal hypothesis. Lucas & Szatrowksi (2014) assess QCA-based strategies of case-selection.

Page 10: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

10

It would take some time to discuss these complex studies, so I shall content myself with several summary judgments. First, case selection techniques have different goals, so any attempt to compare them must focus on the goals that are appropriate to that technique. A technique whose purpose is exploratory (to identify a new hypothesis about Y) cannot be judged by its efficacy in identifying causal mechanisms, for example. Second, among these goals, estimating causal effects is the least common – and, by all accounts, the least successful – of these goals, so any attempt to gauge the effectiveness of case selection methods should probably focus primarily on exploratory and diagnostic functions. Third, case selection techniques are best practiced when taking into account change over time in the key variables, rather than static cross-sectional analyses – as most of the simulation exercises appear to do. Finally, and most importantly, it is difficult and perhaps impossible to simulate the complex features involved in an in-depth case analysis. The question of interest – which case(s) would best serve my purpose if I devoted a case study of it? – is hard to model without introducing assumptions that pre-judge the results of the case study and are in this respect endogenous to the case-selection strategy.6

In my opinion, testing the viability of case selection strategies in a rigorous fashion would involve a methodological experiment of the following sort. First, assemble a panel of researchers with similar background knowledge of a subject. Second, identify a subject deemed ripe for case study research, i.e., it is not well-studied or has received no authoritative treatment and is not amenable to experimental manipulation. Third, select cases algorithmically, following one of the protocols laid out in Table 1. Fourth, randomly assign these cases to the researchers with instructions to pursue all case study goals – exploratory, estimating, and diagnostic. Fifth, assemble a panel of judges, who are well-versed in the subject of theoretical focus, to evaluate how well each case study achieved each of these goals. These could be scored on a questionnaire using ordinal, Likert-style categories. Judges would be instructed to decide independently (without conferring), though there might be a second round of judgments following a deliberative process in which they shared their thoughts and their preliminary decisions.

Such an experiment would be time-consuming and costly (assuming participants receive some remuneration). And it would need to be iterated across several research topics and with several panels of researchers and judges in order to make strong claims of generalizability. Nonetheless, it might be worth pursuing given the possible downstream benefits.7

Causal Inference

Having discussed case selection, we proceed to case analysis, with a focus on the qualitative components of that inquiry. Can causal inference be reached with qualitative data? Here, we encounter the most mysterious, and most contested, aspect of qualitative methods. Causal inference in a quantitative context usually refers to the estimation of a fairly precise causal (treatment) effect. In qualitative contexts, the meaning of causal inference is more

6 For example, Herron & Quinn (2016: 9) make the assumption that the potential outcomes inherent in a case (i.e., the unit-level causal relationship) will be discovered by the case study researcher in the course of an intensive analysis of the case. Yet, “discoverability” is the very thing that case selection techniques are designed to achieve. That is, a case selection technique is regarded as superior insofar as it offers a higher probability of discovering an unknown feature of a case. 7 Note, however, that this experiment disregards qualitative judgments by researchers that might be undertaken after an algorithmic selection of cases. These qualitative judgments might serve as mediators. It could be, for example, that some case-selection strategies work better when the researcher is allowed to make final judgments – from among a set of potential cases that meet the stipulated case-selection criteria – based on knowledge of the potential cases. One must also consider a problem of generalizability that stems from the use of algorithmic procedures for selecting cases. It could be that subjects for which algorithmic case selection is feasible (i.e., where values for X, Z, and Y can be measured across a large sample) are different from subjects for which algorithmic case selection is infeasible. If so, we could not generalize the results of this experiment to the latter genre of case study research.

Page 11: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

11

complicated. First, inferences about a causal effect are apt to be looser, less precise (unless the relationship is deemed to be deterministic). Typically, an author will attempt to determine whether X is a cause of Y and whether its effect is positive or negative. Sometimes, an attempt will be made to account for all the causes of an outcome (a causes-of-effects style of research). Invariably, there will be an attempt to identify a mechanism. Indeed, the latter may form the main focus of analysis, as it would be in situations where a causal effect is presumed at the outset (perhaps as a product of quantitative analysis). For qualitative inquiry, the distinction between internal validity (causal relationships for the studied cases) and external validity (causal relationships inferred for a broader population) is especially critical. This is because the studied cases are usually small in number and not chosen randomly from a known population. When we speak of causal inference in this section we are concerned about inferences drawn for the studied cases, not for a larger population.

Rules of Thumb Over the past several decades, scholars have attempted to identify a set of loosely framed rules to guide the process of qualitative inquiry where the goal is causal inference (loosely defined).8 These may be summarized as follows…

• Analyze sources according to their relevance (to the question of theoretical interest), proximity (whether the source is in a position to know what s/he is claiming), authenticity (the source is not fake or reflecting the influence of someone else), validity (the source is not biased), and diversity (collectively, sources represent a diversity of viewpoints on the question at hand).

• When identifying a new causal factor or theory, look for one (a) that is potentially generalizable to a larger population, (b) that is neglected in the extant literature on your subject, (c) that greatly enhances the probability of an outcome (if binary) or explains a lot of variation on that outcome (if interval-level), and (d) that is exogenous (not explained by other factors).

• Canvas widely for rival explanations, which also serve as potential confounders. Treat them seriously (not as “straw men”), dismissing them only when warranted. Utilize this logic of elimination, where possible, to enhance the strength of the favored hypothesis.

• For each explanation, construct as many testable hypotheses as possible, paying close attention to within-case opportunities – e.g., mechanisms and alternative outcomes.

• Enlist counterfactual thought-experiments in an explicit fashion, making clear which features of the world are being altered, and which are assumed to remain the same, in order to test the viability of a theory. Also, focus on periods when background features are stable (so they don’t serve as confounders) and minimize changes to the world (the minimal-rewrite rule) so that the alternate scenario is tractable.

• Utilize chronologies and diagrams to clarify temporal and causal interrelationships among complex causal factors. Include as many features as possible so that the time-line is continuous, uninterrupted.

These are the loose guidelines – “rules of thumb” – that students are taught, and that scholars follow (we hope). Because of their informal nature, qualitative evidence is often regarded with suspicion. It’s hard to articulate what a convincing inference might consist of and how to know it when one sees it. Are there methodological standards applying to qualitative data analysis (aka process tracing)?

Inferential Frameworks To remedy this situation a number of recent studies try to make sense of qualitative data, imposing order on the seeming chaos. Proposed frameworks include set theory (Mahoney 2012;

8 See Beach & Pedersen (2013), Bennett & Checkel (2015), Brady & Collier (2004), Collier (2011), George (1979), Hall (2006), Jacobs (2015), Mahoney (2012), Roberts (1996), Schimmelfennig (2015), Waldner (2012, 2015a, 2015b).

Page 12: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

12

Mahoney & Vanderpoel 2015), acyclic graphs (Waldner 2015b), or – most commonly – Bayesian inference (Beach & Pedersen 2013: 83-99; Bennett 2008, 2015; Crandell et al. 2011; George & McKeown 1985; Gill et al. 2005; Humphreys & Jacobs 2015, forthcoming; McKeown 1999; Rohlfing 2012: 180-99). These efforts have performed an invaluable service to the cause of qualitative inquiry, fitting them into frameworks that are already well-established for quantitative inquiry. It should be no surprise that there are multiple frameworks, just as there are multiple frameworks for quantitative methodology. Scholars may debate whether, or to what extent, these frameworks are compatible with each other; this important debate is orthogonal to the present topic. The point to stress is that qualitative inquiry can be understood within the rubric of general causal frameworks. There is, in this sense, a unifying logic of inquiry. Thus far, applications of set theory, acyclic graphs, and bayesianism to qualitative methods have focused on making sense of the activity rather than providing a practical guide to research. It remains to be seen whether these can be developed in such a way as to alter the ways that qualitative researchers go about their business. Let me illustrate. Some years ago, Van Evera (1997) proposed a fourfold typology of tests that has since been widely adopted (e.g., Bennett & Checkel 2015: 17; George & Bennett 2005; Mahoney & Vanderpoel 2015; Waldner 2015a). A “hoop” test is necessary (but not sufficient) for demonstrating Hx. A “smoking-gun” test is sufficient (but not necessary) for demonstrating Hx. A “doubly-decisive” test is necessary and sufficient for demonstrating Hx. A “straw-in-the-wind” test is neither necessary nor sufficient, constituting weak or circumstantial evidence. These concepts, diagramed in Table 2, are useful for classifying the nature of evidence according to a researcher’s judgment. However, the hard question – the judgment itself – is elided. When does a particular piece of evidence qualify as a hoop, smoking-gun, doubly-decisive, or straw-in-the-wind test (or something in between)?

Table 2: Qualitative Tests and their Presumed Inferential Role

Inferential Role

Necessary Sufficient

Tests

Hoop Smoking-gun

Doubly-decisive Straw-in-wind

Likewise, Bayesian frameworks are useful for combining evidence from diverse quarters in a logical fashion with the use of subjective assessments, e.g., the probability that a hypothesis is true, ex ante, and assessments of the probability that the hypothesis is true if a piece of evidence (stipulated in advance) is observed. The hard question, again, is the case-specific judgment. Consider the lengthy debate that has ensued over the reasons for electoral system choice in Europe (Kreuzer 2010). Humphreys & Jacobs (2015) use this example to sketch out their application of Bayesian inference to qualitative research. In particular, they explore the “left threat” hypothesis, which suggests that the presence of a large left-wing party explains the adoption of proportional representation (PR) in the early twentieth century (Boix 1999). The authors point out that “for cases with high left threat and a shift to PR, the inferential task is to determine whether they would have, or would not have, shifted to PR without left threat” (Humphreys & Jacobs 2015: 664). Bayesian frameworks do nothing to ease this inferential task,

Page 13: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

13

which takes the form of a counterfactual thought-experiment. Similar judgments are required by other frameworks – set theory, acyclic graphs, and so forth. To get a feel for the level of detail required in qualitative research let us take a closer look at a particular inquiry. Helpfully, Tasha Fairfield (2013: 55-6; see also 2015) provides a scrupulous blow-by-blow account of the sleuthing required to reach each case-level inference in her study of how policymakers avoid political backlash when they attempt to tax economic elites. One of her three country cases is Chile, which is observed during and after a recent presidential election. Fairfield explains,

During the 2005 presidential campaign, right candidate Lavı´n blamed Chile’s persistent inequality on the left and accused President Lagos of failing to deliver his promise of growth with equity. Lagos responded by publicly challenging the right to eliminate 57 bis, a highly regressive tax benefit for wealthy stockholders that he called “a tremendous support for inequality.” The right accepted the challenge and voted in favor of eliminating the tax benefit in congress, deviating from its prior position on this policy and the preferences of its core business constituency. The following three hypotheses encompass the main components of my argument regarding why the right voted in favor of the reform: Hypothesis 1. Lagos’ equity appeal motivated the right to accept the reform, due to concern over public opinion. Hypothesis 2. The timing of the equity appeal—during a major electoral campaign—contributed to its success. Hypothesis 3. The high issue-salience of inequality contributed to the equity appeal’s success. The following four observations, drawn from different sources, provide indirect, circumstantial support for Hypothesis 1: Observation 1a (p. 48): The Lagos administration considered eliminating 57 bis in the 2001 Anti-Evasion reform but judged it politically infeasible given business-right opposition (interview: Finance Ministry-a, 2005). Observation 1b: The Lagos administration subsequently tried to reach an agreement with business to eliminate 57 bis without success (interview, Finance Ministry-b, 2005). Observation 1c: Initiatives to eliminate the exemption were blocked in 1995 and 1998 due to right opposition. (Sources: congressional records, multiple interviews) Observation 1d: Previous efforts to eliminate 57 bis did not involve concerted equity appeals. Although Concertacio´n governments had mentioned equity in prior efforts, technical language predominated, and government statements focused much more on 57 bis’ failure to stimulate investment rather than its regressive distributive impact (congressional records, La Segunda, March 27, 1998, El Mercurio, April 1, 1998, Interview, Ffrench-Davis, Santiago, Chile, Sept. 5, 2005). Inference: These observations suggest that right votes to eliminate 57 bis would have been highly unlikely without some new, distinct political dynamic. Lagos’ strong, high-profile equity appeal, in the unusual context of electoral competition from the right on the issue of inequality, becomes a strong candidate for explaining the right’s acceptance of the reform.

The appendix continues in this vein for several pages, focused relentlessly on explaining the behavior of one particular set of actors in one event, i.e., the motivation of the right-wing in favoring the reform. This event is just one of a multitude of events discussed in connection with the Chilean case study, to which must be added the equally complex set of events occurring in Argentina and Bolivia in Fairfield’s three-country study. Clearly, reaching case-level inferences is complicated business. One may conclude that if researchers agreed on case-level judgments then general frameworks could successfully cumulate those judgments into higher-level inferences, accompanied by a

Page 14: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

14

(very useful!) confidence interval. But if one cannot assume case-level consensus, conclusions based on qualitative judgments combined through a Bayesian (or other) framework represent one researcher’s views, which might vary appreciably from another’s. Readers who are not versed in the intricacies of Chilean politics will have a hard time ascertaining whether Fairfield’s judgments are correct.

A Crowd-based Approach In principle, this sort of problem could be overcome with a crowd-based approach. Specifically, one might survey a panel of experts – chosen randomly or with an aim to represent diverse perspectives – on each point of judgment. One could then cumulate these judgments into an overall inference, in which the confidence interval reflects the level of disagreement among experts (among other things). Unfortunately, not just any crowd will do. The extreme difficulty of case study research derives in no small part from the expertise that case study researchers bring to their task. I cannot envision a world in which lay coders, recruited through Amazon Turk or Facebook, would replace that expertise, honed through years of work on a particular problem and in a particular site (a historical period, country, city, village, organization,…). To be credible, a crowd-based approach to the problem of judgment would need to enlist the small community of experts who study a subject and can be expected to make knowledgeable judgments about highly specific questions such as the “left wing threat.” In the previous example, it would entail enlisting scholars versed in the politics of early twentieth century Europe. This procedure is conceivable, but difficult to implement. How would one identify a random, or otherwise representative, sample? (What is the sampling frame?) How would one motivate scholars to undertake the task? How would one elicit honest judgments about the specific questions on a questionnaire, uncorrupted by broader judgments about the theoretical question at hand (which they would probably be able to infer)? Likewise, if one goes to the trouble of constructing a common coding frame (a questionnaire), an on-line system for recording responses, a system of recruitment, and a Bayesian (or some other) framework for integrating judgments, the considerable investment in time and expense of such a venture would probably justify extending the analysis to many cases, chosen randomly, so that a representative sample can be attained and stochastic threats to inference minimized. In this fashion, procedures to integrate qualitative data into a quantitative framework seem likely to morph from case studies into cross-case coding exercises. This is not to argue against the idea. It is simply to point out that any standardization of procedures tends to work against the intensive focus on one or several cases which (by my definition) characterizes case study research.

Multimethod Research

In multimethod research both qual and quant styles of evidence are brought to bear on the same general research question (Brewer & Hunter 2006; Goertz 2015; Harrits 2011; Lieberman 2005; Seawright 2017). While multimethod research is increasingly common, there are serious questions about its effectiveness (Lohmann 2007). Doing more than one thing might mean doing multiple things poorly, by dint of limited time or expertise. Nor is clear whether qualitative and quantitative analysis can speak to one another productively (Ahmed & Sil 2012). In discussing this question it is important not to confuse disagreement with incommensurability. If qual and quant tests of a proposition are truly independent there is always the possibility that they will elicit different, perhaps even directly contradictory, answers. For example, the most common style of multimethod analysis combines a quantitative analysis of many units with an in-depth, qualitative (or at least partially qualitative) analysis of a single case or a small set of cases, which Lieberman (2005) refers to as a nested analysis. Occasionally, these

Page 15: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

15

two analyses reach different conclusions about a causal relationship (though, one suspects, authors do not always bring these disagreements to the fore). However, the same disagreements also arise from rival quantitative analyses (e.g., conducted with different samples or specifications) and rival qualitative analyses (e.g., focused on different research sites or generated by different researchers). Disagreement about whether X causes Y, or about the mechanisms at work, does not entail that multimethod research is unavailing. Sometimes, triangulation does not confirm one’s hypothesis. It is still useful information; and for those worried about confirmation bias, it is critical. In any case, Seawright (2017) points out that when qualitative and quantitative evidence is combined these analyses are usually oriented toward somewhat different goals. Typically, a large-n cross-case analysis is focused on measuring a causal effect while a small-n within-case analysis is focused on identifying a causal mechanism. As such, the two styles of evidence cannot directly conflict since their objectives are different. They nonetheless inform each other in a useful fashion.

This leaves open another way of viewing multimethod research. Sometimes, the qualitative and quantitative aspects of research are profitably united within a larger “research cycle” that includes a diversity of methods and authors (Lieberman 2016). This allows scholars with a qual or quant bent to do what they do best, concentrating their efforts on their particular skill-set and on one particular context that they can become intimately acquainted with. The research cycle also mitigates a presentational problem – stuffing results from myriad analyses into a 10,000-word article. Unfortunately, the research cycle approach to multimethod research also encounters obstacles. In particular, one must wonder whether cumulation can occur successfully across diverse studies utilizing diverse research methods. Note that political science work is not highly standardized, even when focused on the same research question and when utilizing the same quantitative method. This inhibits the integration of findings, and helps to account for the scarcity of meta-analyses in political science. Qualitative studies are even less likely to be standardized in a way that allows for their integration into an ongoing research trajectory. Inputs and outputs may be defined and operationalized in disparate ways, or perhaps not clearly operationalized at all. And because samples are not randomly chosen, any aggregation of studies cannot purport to represent a larger population in an unbiased fashion. There is yet another angle on this topic that offers what is perhaps a more optimistic – not to mention realistic – reading of the multimethod ideal. Rather than conceptualizing qualitative and quantitative research as separate research designs we might regard them as integral components of the same design.

Nowadays, it is my impression that there are fewer purely qualitative studies. Although the main burden of inference may be carried by qualitative data, this is often supplemented by a large-n cross-case analysis or a large-n within-case analysis (where observations are drawn from a lower level of analysis). Likewise, there are few purely quantitative analyses since the former is usually (always?) accompanied by qualitative observations of one sort or another. At a minimum, qualitative data is trotted out by way of illustration. At a maximum, qualitative data is essential to causal inference.

In this vein, a number of recent studies highlight the vital role played by qualitative data, even when the research design is experimental or quasi-experimental. Although we tend to think of these designs as being quantitative – since they generally incorporate a large number of comparable units – they may also contain important qualitative components. There is, to begin with, the problem of research design. Without an ethnographic understanding of the research site and the individuals who are likely to serve as subjects it is impossible to design an experiment that adequately tests a hypothesis of interest. It is impossible to define a confounder “in the abstract.” In-depth case-based understanding is especially

Page 16: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

16

important in the context of field experimentation, where one context is likely to influence how subjects react to a given treatment.

Second, one must assess potential threats to inference. Where the assignment is randomized, ex ante comparability is assured. But ex post comparability remains a serious threat to inference. For example, experiments often face problems of compliance, so it is incumbent on the researcher to ascertain whether subjects adhered to the prescribed protocol and, if not, which subjects violated the protocol. Where significant numbers of subjects attrit (withdraw from participation) there is an important question about what motivated their withdrawal and what sort of subjects were inclined to withdraw. In field experiments, where a significant time lag often separates the treatment and the outcome of theoretical interest, one must try to determine whether subjects under study may have communicated with one another, introducing potential problems of interference and/or contamination (interference across treatment and control groups). Third, there is a question of causal mechanisms. Assuming a treatment effect can be measured without bias, what is it that accounts for the connection between X and Y? Finally, there are questions of generalizability. In order to determine the external validity of an experiment one must have a good sense of the research site and the subjects who have been studied. Specifically, one must be able to assess the extent to which these individuals, and this particular treatment effect, can be mapped across other – potentially quite different – settings. These issues – of research design, inferential threats, causal mechanisms, and generalizability – are often assessable with qualitative data. Indeed, they may only be assessable by means of a rich, contextual knowledge of a research project as it unfolds on a particular site.

Paluck (2010) argues, further, that experimental designs may be combined with qualitative measurement to access outcomes that would not be apprehended with traditional quantitative measures. As an example, she explores Chattopadhyay & Duflo’s (2004) study of women leaders in India. While praising this landmark study, Paluck (2010: 61) points out,

participant observation of women leaders outside of the council settings—such as in their homes, where they visit with other women—could have revealed whether they were influenced by women constituents in these more informal settings. Intensive interviews could compare social processes in villages with female or male council leaders to reveal how beliefs about women leaders’ efficacy shift. For example, did other council members, elders, or religious leaders make public statements about female leaders or the reservation system? Was there a tipping point at which common sentiment in villages with female leaders diverged from villages with male leaders? Such qualitatively generated insights could have enabled this study to contribute more to general theories of identity, leadership, and political and social change. Moreover, ethnographic work could compare understandings of authority and political legitimacy in villages with female- and male-led councils. Do the first female leaders inspire novel understandings of female authority and legitimacy, or are traditional gender narratives invoked just as frequently to explain women’s new power and position?

Paluck concludes that experiments provide an opportunity for qualitative analysis, one that is grossly under-utilized. Quantitative scholars who are enamored of experiments are well-advised to pursue a parallel investigation along ethnographic lines. Qualitative scholars who wish to understand causal relationship are well-advised to conduct experiments to facilitate this analysis.

For example, suppose one is interested in the impact of modernization across range of measurable outcomes. To address this question, one might construct a field experiment in which an agent of modernization – e.g., a bridge, road, harbor, radio tower – is randomized across sites, allowing for an opportunity to systematically compare treatment and control groups over time. This design not only allows for unbiased estimates of a causal effect; it also affords an occasion for participant-observation – focused, one supposes, on how subjects respond to the treatment, what sense they make of their changing world, and what mechanisms are at work.

Page 17: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

17

Where the treatment is not randomly assigned (i.e., in observational research) there are additional issues pertaining to potential assignment (or selection) bias. Here, qualitative data often comes into play (Dunning 2012). For example, Jeremy Ferwerda & Nicholas Miller (2014) argue that devolution of power reduces resistance to foreign rule. To do so, they focus on France during World War Two, when the northern part of the country was ruled directly by German forces and the southern part was ruled indirectly by the “Vichy” regime headed by Marshall Petain. The key methodological assumption of their regression discontinuity design is that the line of demarcation was assigned in an as-if random fashion. For the authors, and for their critics (Kocher & Monteiro 2015), this assumption requires in-depth qualitative research – research that promises to uphold, or call into question, the author’s entire analysis.

As a second example, we may consider Romer & Romer’s (2010) analysis of the impact of tax changes on economic activity. Because tax changes are non-random, and likely to be correlated with the outcome of interest, anyone interested in this question must be concerned with bias arising from the assignment of the treatment. To deal with this threat, Romer & Romer make use of the narrative record provided by presidential speeches and congressional reports to elucidate the motivation of tax policy changes in the postwar era. This allows them to distinguish policy changes that might have been motivated by economic performance from those that may be considered as-if random. By focusing solely on the latter, they claim to provide an unbiased test of the theory that tax increases are contractionary.

Conclusions

I began by stipulating a definition for quantitative and qualitative research. If the work is quantitative, it enlists patterns of covariation found in a matrix of observations and analyzed with a formal model to reach a descriptive or causal inference. If the work is qualitative, the inference is based on bits and pieces of non-comparable observations that address different aspects of a problem. Traditionally, these are analyzed in an informal fashion. If one accepts this definition, it follows that one can convert qualitative data to quantitative data (“coding”) but not the reverse. And it follows that each approach to social science has characteristic strengths and weaknesses. Qualitative data is generally (but not always!) more useful insofar as a study is exploratory and insofar as it is focused on a single case or a small number of cases.

In the second section, I presented a typology of case-selection strategies whose organizing feature is the goal that a case study is intended to serve. I argued (implicitly) that methods of case selection are considerably more differentiated than extant work suggests.

In the third section, I discussed the application of qualitative data to the goal of causal inference – beginning with common “rules of thumb” and proceeding to general frameworks such as set theory, acyclic graphs, and Bayesian probability. These general frameworks have demonstrated (at least to my mind) that qualitative and quantitative observations can be incorporated into a unified framework in the pursuit of causal inference. They have not yet provided practical tools for the conduct of qualitative inquiry (more on this below).

In the fourth section, I discussed multimethod research – which in this context refers to the combination of quantitative and qualitative data in the same analysis, the same study, or in various studies devoted to the same research question (a “research cycle”). This pluralistic approach seems to offer the possibility of combining the strengths of both styles of research, while avoiding their respective weaknesses. And it seems to be reflected in current trends within the discipline. Acknowledging the burdens imposed upon authors (who must master a diverse range of skills), the limitations imposed by journals with stringent word-counts, and the problem of cumulating results across diverse methods, I would argue that the multi-method ideal nonetheless offers a plausible solution to the pervasive conflict between quantitative and qualitative styles of research.

Page 18: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

18

By way of conclusion to this short review, I want to invoke a fundamental tradeoff in scientific endeavor – between a context of discovery (aka exploration, innovation) and a context of justification (aka appraisal, demonstration, proof, verification/falsification) (Reichenbach 1938). While both are acknowledged to be essential to scientific progress, the field of methodology is strongly aligned with the latter. This is because the task of justification is amenable to systematic rules which can be presented in academic journals, summarized in textbooks, and taught in courses. By contrast, the task of discovery is a comparatively anarchistic affair. There are no rules for finding new things. There may be some informal rules of thumb, analogies, pieces of advice, but nothing one could build an academic field around.

I am exaggerating, to be sure. But the dichotomy is useful in illustrating a core feature of qualitative inquiry. If exploratory work is inherently hostile to systematic method (Feyerabend 1975), and if qualitative approaches are uniquely insightful at early stages of research, then it may be a mistake to suppose that systematic rules of method can apply, or should apply, to this genre of research. Indeed, the very features that inhibit the achievement of falsifiability, replicability, and cumulation may enhance the achievement of discovery. For example, qualitative work is often derided as providing multiple angles on a subject, all of which are plausible and none of which can be definitively proven or disproven. This flows from the narrow but intensive manner of study, which may be summarized as large-K (variables), small-N (observations – in the classic sense of comparable/dataset observations).” Qualitative work is also seen as “post hoc,” adjusting theories to fit the facts or adjusting facts to fit the theories (i.e., looking for settings in which Theory X might be true). These are indeed vices if the researcher’s goal is to avoid Type A errors. But they are virtues insofar as one wishes to discover new – and potentially true – things about the world.

I do not wish to ghetto-ize qualitative inquiry as purely exploratory. Non-comparable bits of evidence have a vital role to play in confirming, and disconfirming, theories, as the foregoing discussion illustrates. However, insofar as qualitative inquiry contributes to the discovery of new concepts, new hypotheses, and new frameworks of analysis we must come to terms with the nature of that inquiry, which is often at odds with current trends in social science methodology. To honor the contributions of qualitative research is to honor the role of exploratory research in the progress of social science.

Page 19: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

19

References

Ahmed, Amel, Rudra Sil. 2012. “When Multi-Method Research Subverts Methodological Pluralism - Or, Why We Still Need Single-Method Research.” Perspectives on Politics 10:4 (December) 935-953.

Alesina, Alberto, Edward Glaeser, Bruce Sacerdote. 2001. “Why Doesn’t the US Have a European-Style Welfare State?” Brookings Papers on Economic Activity 2, 187-277.

Beach, Derek, Rasmus Brun Pedersen. 2013. Process-Tracing Methods: Foundations and Guidelines. Ann Arbor, MI: University of Michigan Press.

Beck, Nathaniel. 2006. “Is Causal-Process Observation An Oxymoron?” Political Analysis 14(3): 347–52.

Beck, Nathaniel. 2010. “Causal Process ‘Observations’: Oxymoron or (Fine) Old Wine.” Political Analysis 18(4): 499–505.

Bennett, Andrew. 2008. “Process Tracing: A Bayesian Approach.” In Janet Box-Steffensmeier, Henry Brady, & David Collier (eds), Oxford Handbook of Political Methodology (Oxford: Oxford University Press) 702-21.

Bennett, Andrew. 2015. “Disciplining our Conjectures: Systematizing Process Tracing with Bayesian Analysis.” In Andrew Bennett & Jeffrey T. Checkel (eds), Process Tracing: From Metaphor to Analytic Tool (Cambridge: Cambridge University Press) 276-98.

Bennett, Andrew, Colin Elman. 2006a. “Complex Causal Relations and Case Study Methods: The Example of Path Dependence.” Political Analysis 14:3, 250-67.

Bennett, Andrew, Colin Elman. 2006b. “Qualitative Research: Recent Developments in Case Study Methods.” Annual Review of Political Science 9:455–76.

Bennett, Andrew, Jeffrey T. Checkel (eds). 2015. Process Tracing: From Metaphor to Analytic Tool. Cambridge: Cambridge University Press.

Blatter, Joachim, Markus Haverland. 2012. Designing Case Studies: Explanatory Approaches in Small-n Research. Palgrave Macmillan.

Boas, Taylor C. 2007. “Conceptualizing Continuity and Change: The Composite-Standard Model of Path Dependence.” Journal of Theoretical Politics 19:1, 33–54.

Boix, Carles. 1999. “Setting the Rules of the Game: The Choice of Electoral Systems in Advanced Democracies.” American Political Science Review 93:3, 609-624.

Brady, Henry E. 2010. “Data-Set Observations versus Causal-Process Observations: The 2000 U.S. Presidential Election.” In Henry E. Brady & David Collier (eds), Rethinking Social Inquiry: Diverse Tools, Shared Standards. 2nd ed. (Lanham, MD: Rowan & Littlefield) 237–42.

Brady, Henry E., David Collier (eds). 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham: Rowman & Littlefield.

Brady, Henry E., David Collier (eds). 2010. Rethinking Social Inquiry: Diverse Tools, Shared Standards. 2nd ed. Lanham, MD: Rowan & Littlefield.

Brewer, John, Albert Hunter. 2006. Foundations of Multimethod Research: Synthesizing Styles. Thousand Oaks, CA: Sage.

Caporaso, James. 2009. “Is There a Quantitative-Qualitative Divide in Comparative Politics?” In Todd Landman and Neil Robinson (eds), Sage Handbook of Comparative Politics (Thousand Oaks: Sage).

Chattopadhyay, Raghabendra, Esther Duflo. 2004. “Women as policy makers: Evidence from a randomized policy experiment in India.” Econometrica 72 (5): 1409-43.

Collier, David. 2011. “Understanding Process Tracing.” PS: Political Science and Politics 44 (04): 823–30.

Collier, David, Colin Elman. 2008. “Qualitative and Multimethod Research: Organizations, Publications, and Reflections on Integration.” In Janet M. Box-Steffensmeier, Henry Brady, and David Collier (eds), The Oxford Handbook for Political Methodology (Oxford: Oxford University Press) 779–95.

Page 20: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

20

Collier, David, Jody LaPorte, Jason Seawright. 2012. “Putting Typologies to Work: Concept Formation, Measurement, and Analytic Rigor.” Political Research Quarterly 65(1) 217–232.

Collier, David, John Gerring (eds). 2009. Concepts and Method in Social Science: The Tradition of Giovanni Sartori. Routledge.

Crandell, Jamie L., Corrine I. Voils, YunKyung Chang, Margarete Sandelowski. 2011. “Bayesian data augmentation methods for the synthesis of qualitative and quantitative research findings.” Quality & Quantity 45, 653–669.

Crasnow, Sharon. 2012. “The Role of Case Study Research in Political Science: Evidence for Causal Claims.” Philosophy of Science 79:5 (December) 655-66.

Dunning, Thad. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach. Cambridge: Cambridge University Press.

Eckstein, Harry. 1975. “Case Studies and Theory in Political Science.” In Fred I. Greenstein and Nelson W. Polsby (eds), Handbook of Political Science, vol. 7. Political Science: Scope and Theory (Reading, MA: Addison-Wesley).

Elman, Colin. 2005. “Explanatory Typologies in Qualitative Studies of International Politics.” International Organization 59:2 (April) 293-326.

Elman, Colin, Diana Kapiszewski. 2014. “Data Access and Research Transparency in the Qualitative Tradition.” PS: Political Science & Politics 47/1: 43–47.

Elman, Colin, Diana Kapiszewski, Lorena Vinuela. 2010. “Qualitative Data Archiving: Rewards and Challenges.” PS: Political Science & Politics 43, 23-27.

Epstein, Leon D. 1964. “A Comparative Study of Canadian Parties.” American Political Science Review 58 (March) 46-59.

Fairfield, Tasha. 2013. “Going Where the Money Is: Strategies for Taxing Economic Elites in Unequal Democracies.” World Development 47, 42–57.

Fairfield, Tasha. 2015. Private Wealth and Public Revenue in Latin America: Business Power and Tax Politics. Cambridge: Cambridge University Press.

Fenno, Richard F., Jr. 1977. “U.S. House Members in Their Constituencies: An Exploration.” American Political Science Review 71:3 (September) 883-917.

Fenno, Richard F., Jr. 1978. Home Style: House Members in their Districts. Boston : Little, Brown. Ferwerda, Jeremy, Nicholas Miller. 2014. “Political Devolution and Resistance to Foreign Rule:

A Natural Experiment.” American Political Science Review 108:3 (August) 642-60. Feyerabend, Paul. 1975. Against Method. London: New Left Books. Friedman, Milton, Anna Jacobson Schwartz. 1963. A Monetary History of the United States, 1867-

1960. Princeton: Princeton University Press. Garfinkel, Harold. 1967. Studies in Ethnomethodology. Englewood Cliffs: Prentice-Hall. George, Alexander L. 1979. “Case Studies and Theory Development: The Method of Structured,

Focused Comparison.” In Paul Gordon Lauren (ed), Diplomacy: New Approaches in History, Theory, and Policy (New York: The Free Press).

George, Alexander L., Andrew Bennett. 2005. Case Studies and Theory Development. Cambridge: MIT Press.

George, Alexander L., Timothy J. McKeown. 1985. “Case Studies and Theories of Organizational Decision-making.” In Robert F. Coulam & Richard A. Smith (eds), Advances in Information Processing in Organizations (Greenwich, Conn.: JAI Press) 21–58.

Gerring, John. 2007. Case Study Research: Principles and Practices. Cambridge: Cambridge University Press.

Gerring, John. 2012. “Mere Description.” British Journal of Political Science 42:4 (October) 721-46. Gerring, John. 2017. Case Study Research: Principles and Practices, 2d ed. Cambridge: Cambridge

University Press. Gerring, John, Jason Seawright. 2016. “The Inference in Causal Inference: A Psychology for

Social Science Methodology.” Unpublished manuscript, Department of Political Science, Boston University.

Page 21: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

21

Gerring, John, Lee Cojocaru. 2016. “Selecting Cases for Intensive Analysis: A Diversity of Goals and Methods.” Sociological Methods & Research (forthcoming).

Gerring, John, Rose McDermott. 2007. “An Experimental Template for Case-Study Research.” American Journal of Political Science 51:3 (July) 688-701.

Gill, Christopher J, Lora Sabin, Christopher H. Schmid. 2005. “Why Clinicians are Natural Bayesians.” BMJ 330:1080-3 (May 7).

Glassner, Barry; Jonathan D. Moreno (eds). 1989. The Qualitative-Quantitative Distinction in the Social Sciences (Boston Studies in the Philosophy of Science, 112).

Goertz, Gary. 2005. Social Science Concepts: A User’s Guide. Princeton: Princeton University Press. Goertz, Gary. 2017. Multimethod Research, Causal Mechanisms, and Selecting Cases: The Research Triad.

Princeton: Princeton University Press. Goertz, Gary, James Mahoney. 2012. A Tale of Two Cultures: Qualitative and Quantitative Research in

the Social Sciences. Princeton: Princeton University Press. Grimmer, Justin, Brandon M. Stewart. 2013. “Text as Data: The Promise and Pitfalls of

Automatic Content Analysis Methods for Political Texts.” Political Analysis 21:3, 267-97. Hall, Peter A. 2003. “Aligning Ontology and Methodology in Comparative Politics.” In James

Mahoney and Dietrich Rueschemeyer (eds), Comparative Historical Analysis in the Social Sciences (Cambridge: Cambridge University Press).

Hall, Peter A. 2006. “Systematic process analysis: when and how to use it.” European Management Review 3, 24–31

Hammersley, Martyn. 1992. “Deconstructing the Qualitative-Quantitative Divide.” In Julie Brannen (ed), Mixing Methods: Qualitative and Quantitative Research (Aldershot: Avebury).

Harrits, Gitte Sommer. 2011. “More Than Method? A Discussion of Paradigm Differences within Mixed Methods Research.” Journal of Mixed Methods Research 5(2): 150–66.

Herron, Michael C., Kevin M. Quinn. 2016. “A Careful Look at Modern Case Selection Methods.” Sociological Methods & Research (forthcoming).

Humphreys, Macartan, Alan M. Jacobs. 2015. “Mixing Methods: A Bayesian Approach.” American Political Science Review 109:4 (November) 653-73.

Humphreys, Macartan, Alan M. Jacobs. Forthcoming. Integrated Inferences: A Bayesian Integration of Qualitative and Quantitative Approaches to Causal Inference. Cambridge: Cambridge University Press.

Jacobs, Alan. 2015. “Process Tracing the Effects of Ideas.” In Andrew Bennett, Jeffrey T. Checkel (eds), Process Tracing: From Metaphor to Analytic Tool (Cambridge: Cambridge University Press) 41-73.

Kapiszewski, Diana, Lauren M. MacLean, Benjamin L. Read. 2015. Field Research in Political Science: Practices and Principles. Cambridge: Cambridge University Press.

Karl, Terry Lynn. 1997. The Paradox of Plenty: Oil Booms and Petro-States. Berkeley: University of California Press.

King, Gary, Robert O. Keohane, Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.

Kocher, Matthew, Nuno Monteiro. 2015. “What’s in a Line? Natural Experiments and the Line of Demarcation in WWII Occupied France.” Unpublished manuscript, Department of Political Science, Yale University.

Kreuzer, Markus. 2010. “Historical Knowledge and Quantitative Analysis: The Case of the Origins of Proportional Representation.” American Political Science Review 104:369–92.

Levy, Jack S. 2007. “Qualitative Methods and Cross-Method Dialogue in Political Science.” Comparative Political Studies 40(2): 196–214.

Levy, Jack S. 2008. “Case Studies: Types, Designs, and Logics of Inference.” Conflict Management and Peace Science 25:1–18.

Lieberman, Evan S. 2005. “Nested Analysis as a Mixed-Method Strategy for Comparative Research.” American Political Science Review 99:3 (August) 435-52.

Page 22: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

22

Lieberman, Evan S. 2010. “Bridging the Qualitative-Quantitative Divide: Best Practices in the Development of Historically Oriented Replication Databases.” Annual Review of Political Science 13, 37-59.

Lieberman, Evan S. 2016. “Improving Causal Inference through Non-Causal Research: Can the Bio-Medical Research Cycle Provide a Model for Political Science?” Unpublished manuscript, Department of Political Science, MIT.

Lijphart, Arend. 1971. “Comparative Politics and the Comparative Method.” American Political Science Review 65, 682-93.

Lohmann, Susanne. 2007. “The Trouble with Multi-Methodism.” Newsletter of the APSA Organized Section on Qualitative Methods 5(1): 13–17.

Lucas, Samuel R., Alisa Szatrowski. 2014. “Qualitative Comparative Analysis in Critical Perspective.” Sociological Methodology 44:1, 1–79.

Lynd, Robert Staughton, Helen Merrell Lynd. 1929/1956. Middletown: A Study in American Culture. New York: Harcourt, Brace.Mahoney, James, Gary Goertz. 2006. “A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research.” Political Analysis 14:227–49.

Mahoney, James. 2010. “After KKV: The New Methodology of Qualitative Research.” World Politics 62 (1): 120–47.

Mahoney, James. 2012. “The Logic of Process Tracing Tests in the Social Sciences.” Sociological Methods & Research 41:4 (November) 566-590.

Mahoney, James, Gary Goertz. 2006. “A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research.” Political Analysis 14:3 (Summer) 227-49.

Mahoney, James, Kathleen Thelen (eds). 2015. Advances in Comparative-Historical Analysis. Cambridge: Cambridge University Press.

Mahoney, James, Rachel Sweet Vanderpoel. 2015. “Set Diagrams and Qualitative Research.” Comparative Political Studies 48:1 (January) 65-100.

Mansfield, Edward D., Jack Snyder. 2005. Electing to Fight: Why Emerging Democracies go to War. Cambridge: MIT Press.

McKeown, Timothy J. 1999. “Case Studies and the Statistical World View.” International Organization 53 (Winter) 161-190.

McLaughlin, Eithne. 1991. “Oppositional Poverty: The Quantitative/Qualitative Divide and Other Dichotomies.” The Sociological Review 39 (May): 292-308.

Mill, John Stuart. 1843/1872. The System of Logic, 8th ed. London: Longmans, Green. Moore, Barrington, Jr. 1966. Social Origins of Dictatorship and Democracy: Lord and Peasant in the

Making of the Modern World. Boston: Beacon Press. Morgan, Mary. 2012. “Case Studies: One Observation or Many? Justification or Discovery?”

Philosophy of Science 79:5 (December) 655-66. Page, Scott E. 2006. “Essay: Path Dependence.” Quarterly Journal of Political Science 1: 87–115. Paluck, Elizabeth Levy. 2010. “The Promising Integration of Qualitative Methods and Field

Experiments.” The ANNALS of the American Academy of Political and Social Science 628, 59-71. Patton, Michael Quinn. 2002. Qualitative Research & Evaluation Methods. Sage. Pincus, Steve. 2011. 1688: The First Modern Revolution. New Haven: Yale University Press. Platt, Jennifer. 1992. “’Case Study’ in American Methodological Thought.” Current Sociology 40:1,

17-48. Posner, Daniel. 2004. “The Political Salience of Cultural Difference: Why Chewas and

Tumbukas are Allies in Zambia and Adversaries in Malawi.” American Political Science Review 98:4 (November) 529-46.

Ray, James Lee. 1993. “Wars between Democracies: Rare or Nonexistent?” International Interactions 18:251–76.

Reichenbach, Hans 1938. Experience and Prediction: An Analysis of the Foundations and the Structure of Knowledge. University of Chicago Press.

Page 23: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

23

Reiss, Julian. 2009. “Causation in the Social Sciences: Evidence, Inference, and Purpose.” Philosophy of the Social Sciences 39(1): 20–40.

Rihoux, Benoit. 2013. “Qualitative Comparative Analysis (QCA), Anno 2013: Reframing The Comparative Method’s Seminal Statements.” Swiss Political Science Review 19:2, 233–45.

Roberts, Clayton. 1996. The Logic of Historical Explanation. University Park: Pennsylvania State University Press.

Rohlfing, Ingo. 2012. Case Studies and Causal Inference: An Integrative Framework. Palgrave Macmillan.

Romer, Christina D., David H. Romer. 2010. “The Macroeconomic Effects of Tax Changes: Estimates Based on a New Measure of Fiscal Shocks.” American Economic Review 100 (June) 763–801.

Rosenau, Pauline Marie. 1992. Post-Modernism and the Social Sciences: Insights, Inroads, and Intrusions. Princeton: Princeton University Press.

Schatz, Edward (ed). 2009. Political Ethnography: What Immersion Contributes to the Study of Power. Chicago: University of Chicago Press.

Schimmelfennig, Frank. 2015. “Efficient Process Tracing: Analyzing the Causal Mechanisms of European Integration.” In Andrew Bennett, Jeffrey T. Checkel (eds), Process Tracing: From Metaphor to Analytic Tool (Cambridge: Cambridge University Press) 98-125.

Schwartz, Howard, Jerry Jacobs. 1979. Qualitative Sociology: A Method to the Madness. New York: Free Press.

Seawright, Jason. 2016. “The Case for Selecting Cases that are Deviant or Extreme on the Independent Variable.” Sociological Methods & Research (forthcoming).

Seawright, Jason. 2017. Multi-Method Social Science: Combining Qualitative and Quantitative Tools. Cambridge: Cambridge University Press, forthcoming.

Seawright, Jason, John Gerring. 2008. “Case-Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options.” Political Research Quarterly 61:2 (June) 294-308.

Shapiro, Ian, Rogers Smith, Tarek Masoud (eds). 2004. Problems and Methods in the Study of Politics. Cambridge: Cambridge University Press.

Shweder, Richard A. 1996. “Quanta and Qualia: What is the ‘Object’ of Ethnographic Method?” In Richard Jessor, Anne Colby, and Richard A. Shweder (eds), Ethnography and Human Development: Context and Meaning in Social Inquiry (Chicago: University of Chicago Press).

Sil, Rudra. 2000. “The Division of Labor in Social Science Research: Unified Methodology or ‘Organic Solidarity’?” Polity 32:4 (Summer) 499-531.

Skocpol, Theda. 1979. States and Social Revolutions: A Comparative Analysis of France, Russia, and China. Cambridge: Cambridge University Press.

Skocpol, Theda, Margaret Somers. 1980. “The Uses of Comparative History in Macrosocial Inquiry.” Comparative Studies in Society and History 22:2 (April) 147-97.

Snow, C.P. 1959/1993. The Two Cultures. Cambridge: Cambridge University Press. Strauss, Anselm, Juliet Corbin. 1998. Basic of Qualitative Research: Techniques and Procedures for

Developing Grounded Theory. Thousand Oaks: Sage. Van Evera, Stephen. 1997. Guide to Methods for Students of Political Science. Ithaca: Cornell University

Press. Waldner, David. 2012. “Process Tracing and Causal Mechanisms.” In Harold Kincaid (ed),

Oxford Handbook of Philosophy of Social Science (Oxford: Oxford University Press) 65-84. Waldner, David. 2015a. “Process Tracing and Qualitative Causal Inference.” Security Studies 24:2,

239-50. Waldner, David. 2015b. “What Makes Process Tracing Good? Causal Mechanisms, Causal

Inference, and the Completeness Standard in Comparative Politics.” In Andrew Bennett, Jeffrey T. Checkel (eds), Process Tracing: From Metaphor to Analytic Tool (Cambridge: Cambridge University Press) 126-52.

Page 24: Qualitative Methods - BU Blogsblogs.bu.edu/jgerring/files/2016/06/Text_9.pdf2 Qualitative methods, broadly construed, extend back to the very beginnings of social and political analysis

24

Walter, Barbara. 2002. Committing to Peace: The Successful Settlement of Civil Wars. Princeton: Princeton University Press.

Yanow, Dvora, Peregrine Schwartz-Shea (eds). 2013. Interpretation and Method: Empirical Research Methods and the Interpretive Turn, 2d ed. Armonk, NY: M E Sharpe.