TARROW_Bridging the Quantitative-qualitative Divide

8/12/2019 TARROW_Bridging the Quantitative-qualitative Divide

1/5

Bridging the Quantitative-Qualitative Divide in Political Science

Designing Social Inquiry: Scientific Inference in Qualitative Research by Gary King; Robert O.Keohane; Sidney VerbaReview by: Sidney TarrowThe American Political Science Review, Vol. 89, No. 2 (Jun., 1995), pp. 471-474Published by: American Political Science AssociationStable URL: http://www.jstor.org/stable/2082444.

Accessed: 21/06/2014 14:39

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at.http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

.

American Political Science Associationis collaborating with JSTOR to digitize, preserve and extend access to

The American Political Science Review.

http://www.jstor.org

This content downloaded from 181.177.248.115 on Sat, 21 Jun 2014 14:39:05 PMAll use subject to JSTOR Terms and Conditions
http://www.jstor.org/action/showPublisher?publisherCode=apsahttp://www.jstor.org/stable/2082444?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/2082444?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=apsa


2/5

American Political Science Review Vol. 89, No. 2 June 1995

BRIDGINGHEQUANTITATIVE.QUALITATIVEDIVIDENPOLITICALCIENCESIDNEY TARROWCornell UniversityIn Designing Social Inquiry, Gary King, Bob Keo-hane and Sidney Verba (KKV) have performed areal service to qualitative researchers. I, for one,will not complain if I never again have to look into theuncomprehending eyes of first-year graduate stu-dents when I enjoin them (pace Przeworski and Teune)to "turn proper names into variables." The book is briefand lucidly argued and avoids the weighty, muscle-bound pronouncements that are often studded onto thepages of methodological manuals.But following KKV's injunction that "a slightlymore complicated theory will explain vastly more ofthe world" (p. 105), I will praise them no more butfocus on an important weakness in the book. Theircentral argument is that the same logic that is "expli-cated and formalized clearly in discussions of quan-titative research methods" underlies-or should-thebest qualitative research (p. 4). If this is so, then theyreally ought to have paid more attention to therelations between quantitative and qualitative ap-proaches and what a rigorous use of the latter canoffer quantifiers. But while they offer a good deal ofgenerous (at times patronizing) advice to qualitativelyoriented scholars, they say very little about howqualitative approaches can be combined with quanti-tative research. Especially with the growth of choice-theoretic approaches, whose users often illustratetheir theories with stories, there is a need for a set ofground rules on how to make intelligent use ofqualitative data.KKV do not address this issue. Rather, they use themodel of quantitative research to advise qualitativeresearchers on how best to approximate good modelsof descriptive and causal inference. (Increasing thenumber of observations is their cardinal operationalrule.) But in today's social science world, how manysocial scientists can be simply labeled "qualitative" or-quantitative"? How often, for example, do we findsupport for sophisticated game-theoretic models rest-ing on the use of anecdotal reports or on secondaryevidence lifted from one or two qualitative sources?More and more frequently in today's social sciencepractice, quantitative and qualitative data are inter-larded within the same study. A recent work thatKKV warmly praise illustrates both that their distinc-tion between quantitative and qualitative researchersis too schematic and that we need to think moreseriously about the interaction of the two kinds of data.Marinating PutnamIn Robert Putnam's (1993) analysis of Italy's creationof a regional layer of government, Making DemocracyWork,countless elite and mass surveys and ingeniousquantitative measures of regional performance arearrayed for a 20-year period of regional development.

On top of this, he conducted detailed case studies ofthe politics of six Italian regions, gaining, in theprocess, what KKV recommend as "an intimateknowledge of the internal political manoeuvering andpersonalities that have animated regional politicsover the last two decades" (p. 5) and Putnam calls"marinating yourself in the data" (Putnam 1993, 190).KKV use MakingDemocracyWork o praise the virtuesof "soaking and poking," in the best Fenno tradition(p. 38).But Putnam's debt to qualitative approaches ismuch deeper and more problematic than this; forafter spending two decades administering surveys toelites and citizens in the best Michigan mode, he wasleft with the task of explaining the sources of the vastdifferences he had found between Italy's north-cen-tral and southern regions. To find them, his quanti-tative evidence offered only indirect evidence; and heturned to history, repairing to the halls of Oxford,where he delved deep into the Italian past to fashiona provocative interpretation of the superior perfor-mance of the northern Italian regional governmentsvis-A-vis the southern ones. This he based on the civictraditions of the (northern) Renaissance city-states,which, according to him, provide "social capital" thatis lacking in the traditions of the South (chap. 5). Aturn to qualitative history (probably not even inPutnam's mind when he designed the project) wasused to interpret cross-sectional, contemporary quan-titative findings.Putnam's procedure in MakingDemocracyWorkpin-points a problem in melding quantitative and quali-tative approaches that KKV's canons of good scien-tific practice do not help to resolve. For in delvinginto the qualitative data of history to explain ourquantitative findings, by what rules can we choosethe period of history that is most relevant to ourproblem? And what kindof history are we to use; thetraditional history of kings and communes or thehistory of the everyday culture of the little people?And how can the effect of a particular historicalperiod be separated from that of the periods thatprecede or follow it? In the case of Making DemocracyWork,for example, it would have been interesting toknow (as Suzanne Berger asked at the 1994 APSAroundtable devoted to the book) by what rules ofinference Putnam chose the Renaissance as determin-ing of the North's late twentieth-century Italian civicsuperiority. Why not look to its sixteenth-centurycollapse faced by more robust monarchies, its nine-teenth-century military conquest of the South, or its1919-21 generation of fascism (not to mention its1980s corruption-fed pattern of economic growth)?None of these are exactly "civic" phenomena; bywhat rules of evidence are they less relevant in"explaining" the northern regions' civic superiority

471

http://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsp


3/5

Review Symposium June 1995over the South than the period of the Renaissancecity-states? Putnam does not tell us; nor do KKV.To generalize from the problem of Putnam's book,qualitative researchers have much to learn from themodel of quantitative research. But their quantitativecousins who wish to profit from conjoining theirfindings with qualitative sources need, for the selec-tion of qualitative data and the intersection of the twotypes, rules just as demanding as the rules putforward by KKV for qualitative research on its own. Ishall sketch some useful approaches to bridging thequantitative-qualitative gap from recent examples ofcomparative and international research.Tracing Processes To Interpret DecisionsOne such rule that KKV cite favorably is the practiceof process racing,in which the researcher looks closelyat 'the decision process by which various initialconditions are translated into outcomes'" (p. 226,quoting George and McKeown 1985, 35). But evenhere, KKV interpret the advantages of process tracingnarrowly, assimilating it to their favorite goal ofincreasing the number of theoretically relevant obser-vations (p. 227). As George and McKeown actuallyconceived it, the goal of process tracing was not toincrease the number of discrete decision stages andaggregate them into a larger number of data pointsbut to connect the phases of the policy process andenable the investigator to identify the reasons for theemergence of a particular decision through the dy-namic of events (George and McKeown 1985, 34-41).Process tracing is different in kind from observationaccumulation and is best employed in conjunctionwith it-as was the case, for example, in the study ofcooperation on economic sanctions by Lisa Martin(1992) that KKV cite so favorably.Systematic and NonsystematicVariable DiscriminationKKV give us a second example of the uses of quali-tative data but, once again, underestimate its partic-ularity. They argue that the variance between differ-ent phenomena "can be conceptualized as arisingfrom two separate elements: systematicand nonsystem-atic differences," the former more relevant to fashion-ing generalizations than the latter (p. 56). For exam-ple, in the case of conservative voting in Britain,systematic differences include such factors as theproperties of the district, while unsystematic differ-ences could include the weather or a flu epidemic atthe time of the election. "Had the 1979 British elec-tions occurred during a flu epidemic that sweptthrough working-class houses but tended to sparethe rich," they conclude, "our observations might berather poor measures of underlying Conservativestrength" (pp. 56-57).Right they are, but this piece of folk wisdom hardlyexhausts the importance of nonsystematic variablesin the interpretation of quantitative data. A goodexample comes from how the meaning and extensionof the strike changed as systems of institutionalized

industrial relations developed in the nineteenth cen-tury. At its origins, the strike was spontaneous, unin-stitutionalized, and often accompanied by whole-community "turnouts." As unions developed andgovernments recognized workers' rights, the strikebroadened to whole sectors of industry, became aninstitutional accompaniment to industrial relations,and lost its link to community collective action. Thesystematic result of this change was permanently toaffect the patterns of strike activity. Quantitativeresearchers like Michelle Perrot (1986) documentedthis change. But had she regarded it only as a case of"nonsystematic variance" and discarded it from hermodel, as KKV propose, Perrot might well havemisinterpreted the changes in the form and incidenceof the strike rate. Because she was as good a historianas she was a social scientist, she retained it as a crucialchange that transformed the relations between thestrike incidence and industrial relations.

To put this more abstractly, distinct historicalevents often serve as the tipping points that explainthe interruptions in an interrupted time-series, per-manently affecting the relations between the vari-ables (Griffin 1992). Qualitative research that turns up"nonsystematic variables" is often the best way touncover such tipping points. Quantitative researchcan then be reorganized around the shifts in variableinteraction that such tipping points signal. In otherwords, the function of qualitative research is notonly, as KKV seem to argue, to peel away layers ofunsystematic fluff from the hard core of systematicvariables but also to assist researchers to understandshifts in the value of the systematic variables.Framing Qualitative Research withinQuantitative ProfilesThese two uses of qualitative data pertain largely toaiding quantitative research. But this is not the onlyway in which social scientists can combine quantita-tive and qualitative approaches. Another is to focuson the qualitative data, using a systematic quantita-tive data base as a frame within which the qualitativeanalysis is carried out. Case studies have been validlycriticized as being based on often dramatic but fre-quently unrepresentative cases. Studies of successfulsocial revolutions often possess characteristics thatmay also be present in unsuccessful revolutions,rebellions, riots, and ordinary cycles of protest (Tilly1993, 12-14). In the absence of an adequate sample ofrevolutionary episodes, no one can ascribe particularcharacteristics to a particular class of collective action.The representativity of qualitative research cannever be wholly assured until the cases become sonumerous that the analysis comes to resemble quan-titative research (at which point the qualitative re-search risks losing its particular properties of depth,richness and process tracing). But framing it withina quantitative data base makes it possible to avoidgeneralizing on the occasional "great event" andpoints to less dramatic but cumulative- historicaltrends.Scholars working in the "collective action event"

472



4/5

American Political Science Review Vol. 89, No. 2

history tradition have used this double strategy withsuccess. For example, in his 1993 study of over sevenhundred revolutionary years in over five hundredyears of European history, Charles Tilly assembleddata that could have allowed him to engage in alarge-N study of the correlates and causes of revolu-tion. Tilly knows how to handle large time-series datasets as well as anybody. But he did not believe thatthe concept of revolution had the monolithic qualitythat other social scientists had assigned to it (1993,chap. 1). So he resisted the temptation for quantifi-cation, using his data base, instead, to frame a seriesof regional time-series narratives that depended asmuch on his knowledge of European history as onthe data themselves. When a problem cried out forsystematic quantitative analysis (e.g., when it cameto periodizing nationalism), Tilly (1994) was happy toexploit the quantitative potential of the data. But thequantitative data set served mainly as a frame forqualitative analysis of representative regional and tem-poral revolutionary episodes and series of episodes.Puffing Qualitative Flesh on Quantitative BonesThese examples are possibly exotic to the traditions ofmuch of American social science practice. But anAmerican sociologist, Doug McAdam, has shownhow social science can be enriched by combiningquantitative and qualitative approaches to the samedata base. McAdam's 1988 study of Mississippi Free-dom Summer participants was based on a treasure-trove of quantifiable data-the original question-naires of the prospective Freedom Summervolunteers. While some of these young people even-tually stayed home, others went south to registervoters, teach in "freedom schools" and risk thedangers of Ku Klux Klan violence. Two decades later,both the volunteers and the no-shows could be inter-viewed by a researcher with the energy and the imag-ination to go beyond the use of canned data banks.McAdam's main analytic strategy was to carry outa paired comparison between the questionnaires ofthe participants and the stay-at-homes and to inter-view a sample of the former in their current lives.This systematic comparison formed the analyticalspine of the study and of a series of technical papers.But except for a table or two in each chapter, thetexture of FreedomSummeris overwhelmingly quali-tative. McAdam draws on his interviews with formerparticipants, as well as on secondary analysis of otherpeople's work, to get inside the Freedom Summerexperience and to highlight the effects that participa-tion had on their careers and ideologies and theirlives since 1964. With this combination of quantitativeand qualitative approaches, he was able to tease aconvincing picture of the effects of Freedom Summeractivism from his data.As I write this, I imagine KKVexclaiming, "But thisis preciselythe direction we would like to see qualita-tive research moving-toward expanding the numberof observations and respecifying hypotheses to allowthem to be tested on different units " (see chap. 6).But would they argue, as I am, that it is the combina-

tion of quantitative and qualitative methods trainedon the same problem (not a move toward the logic ofquantitative analysis alone) that is desirable? Twomore ways of combining these two logics illustratemy intent.

Sequencing Quantitative and Qualitative ResearchThe growth industry of qualitative case studies thatfollowed the 1980-81 Solidarity movement in Polandlargely took as given the idea that Polish intellectualshad the most important responsibility for the birthand ideology of this popular movement. There wasscattered evidence for this propulsive role of theintellectuals; but since most of the books that ap-peared after the events were written by them or bytheir foreign friends, an observer bias might havebeen operating to inflate their importance in themovement vis-A-vis the working class that was at theheart of collective action in 1980-81 and whose voicewas less articulate.Solid quantitative evidence came to the rescue. In asharp attack on the "intellectualist" interpretationand backed by quantitative evidence from the strikedemands of the workers themselves, Roman Labashowed that their demands were overwhelminglyoriented toward trade union issues and showed littleor no effect of the proselytizing that Polish intellectu-als had supposedly been doing among the workers ofthe Baltic coast since 1970 (1991, chap. 8). This findingdovetailed with Laba's own qualitative analysis of thedevelopment of the workers' movement in the 1970sand downplayed the role of the Warsaw intellectualswho had been at the heart of a series of books by theirforeign friends.The response of those who had been responsiblefor the intellectualist interpretation of Solidarity waspredictably violent. But there were also more mea-sured responses that shed new light on the issue. Forexample, prodded by Laba's empirical evidence ofworker self-socialization, Jan Kubik returned to theissue with both a sharper analytical focus and betterqualitative evidence than the earlier intellectualisttheorists had employed, criticizing Laba's conceptu-alization of class and reinterpreting the creation ofSolidarity as "a multistranded and complicated socialentity ... created by the contributions of variouspeople" whose role and importance he proceeded todemonstrate (1994, 230-38). Moral: a sequence ofcontributions using different kinds of evidence led toa clearer and more nuanced understanding of the roleof different social formations in the world's firstsuccessful confrontation with state socialism.TriangulationI have left for last the research strategy that I thinkbest embodies the strategy of combining quantitativeand qualitative methods-the triangulation of differ-ent methods on the same problem. Triangulation isparticularlyappropriatein cases in which quantita-tive data are partial and qualitativeinvestigation isobstructedby politicalconditions. For example, Val-

473



5/5

Review Symposium June 1995erie Bunce used both case methodology and quanti-tative analysis to examine the policy effects of lead-ership rotation in Western and socialist systems. InDo New Leaders Make a Difference?, she wrote, "Idecided against selecting one of these approaches tothe neglect of the other [the better] to test the impactof succession on public policy by employing bothmethodologies" (1981, 39).Triangulation is also appropriate in specifying hy-potheses in different ways. Consider the classicalTocquevillian insight that regimes are most suscepti-ble to a political opportunity structure that is partiallyopen. The hypothesis takes shape in two comple-mentary ways: (1) that liberalizing regimes are moresusceptible to opposition than either illiberal or liberalones; and (2) that within the same constellation ofpolitical units, opposition is greatest at intermediatelevels of political opportunity. Since there is noparticular advantage in testing one version of thehypothesis over the other, testing both is optimal (ascan be seen in the recent social movement study,Kriesi et al. 1995).My final example of triangulation comes, withapologies, from my own research on collective actionand social movements in Italy. In the course of aqualitative reconstruction of a left-wing Catholic"base community" that was active in a peripheraldistrict of Florence in 1968, I found evidence thatlinked this movement discursively to the larger cycleof student and worker protest going on in Italy at thesame time (Tarrow 1988). Between 1965 and 1968, itsmembers had been politically passive, focusingmainly on neighborhood and educational issues. Butas the worker and student movements explodedaround it in 1968, their actions became more confron-tational, organized around the themes of autonomyand internal democracy that were animating thelarger worker and student movements around them.Researchers convinced of their ability to under-stand political behavior by interpreting "discourse"might have been satisfied with these observations;but I was not. If nothing else, Florence was only onecase among potential thousands. And in today'sglobal society, finding thematic similarity among dif-ferent movements is no proof of direct diffusion,since many movements around the world select fromthe same stock of images and frames without the leastconnection among them (Tarrow 1994, chap. 11).As it happened, quantitative analysis came to therescue to triangulate on the same problem. For alarger study, I had collected a large sample of nationalcollective action events for a period that bridged the1968 Florentine episode. And as it also happened,two Italian researchers had collected reliable data onthe total number of religious "base communities" likethe Florentine one throughout the country (Sciubbaand Pace 1976). By reoperationalizing the hypothesiscross-sectionally, I was able to show a reasonablyhigh positive correlation (R = .426) between thepresence of Catholic base communities in variouscities and the magnitude of general collective actionin each city (Tarrow1989, 200). A longitudinal, local,and qualitative case study triangulatedwith the re-

sults of cross-sectional, national, and quantitativecorrelations to turn my intuitive hunch that Italy inthe 1960s underwent an integrated cycle of protestinto a more strongly supported hypothesis.KKV are not among those social scientists whobelieve that quantification is the answer to all theproblems of social science research. But their single-minded focus on the logic of quantitative research(and of a certain kind of quantitative research) leavesunderspecified the particular contributions that qual-itative approaches make to scientific research, espe-cially when combined with quantitative research. Asquantitatively trained researchers shift to choice-the-oretic models backed up by illustrative examples(often containing variables with different implicitmetrics), the role of qualitative research grows moreimportant. We are no longer at the stage when publicchoice theorists can get away with demonstrating atheorem with an imaginary aphorism. We need todevelop rules for a more systematic use of qualitativeevidence in scientific research. Merely wishing that itwould behave as a slightly less crisp version ofquantitative research will not solve the problem.This is no plea for the veneration of historicaluniqueness and no argument for the precedence of"interpretation" over inference. (For an excellentanalysis of the first problem, see KKV pp. 42-3 and ofthe second, pp. 36-41.) My argument, rather, is thata single-minded adherence to either quantitative orqualitative approaches straightjackets scientificprogress. Whenever possible, we should use qualita-tive data to interpret quantitative findings, to getinside the processes underlying decision outcomes,and to investigate the reasons for the tipping pointsin historical time-series. We should also try to usedifferent kinds of evidence together and in sequenceand look for ways of triangulating different measureson the same research problem.KKV have given us a spirited, lucid, and well-bal-anced primer for training our students in the essentialunity of social science work. Faced by the clouds ofphilosophical relativism and empirical nominalismthat have recently blown onto the field of socialscience, we should be grateful to them. But theirtheoretical effort is marred by the narrowness of theirempirical specification of qualitative research and bytheir lack of attention to the qualitative needs ofquantitative social scientists. I am convinced that hada final chapter on combining quantitative and quali-tative approaches been written by these authors, itsspirit would not have been wildly at variance withwhat I have argued here. As it is, someone else willhave to undertake that effort.

NotesI wish to thank Henry Brady, Miriam Golden, Peter Katzen-stein, David Laitin, Peter Lange, Doug McAdam, WalterMebane, RobertPutnam, Shibley Telhamiand CharlesTillyfor their comments on drafts of this review.

474


Documents

TARROW_Bridging the Quantitative-qualitative Divide