utions agreeable to all impacted parties Ð and some-thing like equality of `argumentative capacity' are alsorequired. While House and Howe suggest that thoseentering into deliberation be committed to these ideals,it seems crucial that these requirements be furtherunderscored. This is particularly true with prescrip-tions for evaluation that dictate the way people argueor limit admissible claims. Clearly, at times, seeminglynon-rational, impassioned pleas are the very means ofopening up a deliberative process (Gutmann &Thompson, 1996). This is perhaps where House andHowe's dialogical requirement requires further elabor-ation. Moreover, while it is obvious that within adeliberative democratic framework rational standardsare required for judging the merit or worth of moralclaims, it is necessary that the standards themselves beopen for debate. Without this safeguard, those chargedwith making decisions about what quali®es as`rational' can restrain dialogue and control the dis-course.

The authors fully acknowledge that a deliberativedemocratic framework is by its very nature contentiousand often demands compromise on matters of value.But the framework itself is uncompromising in itsoverarching commitment to the rational decidability ofvalue claims. House and Howe might well endorseGutmann's (1990) observation that the standards of

democratic deliberation often do not yield eithersimple or single answers to questions, but the fact``that democratic standards do not yield simpleanswers is a necessity, given the complexity of our col-lective life; that they do not yield single answers is avirtue, which underscores the democratic critique ofthe `one best system''' (p. 17).

References

Gutmann, A. (1990). Democratic education in di�cult times.

Teachers College Record, 92(1), 7±20.

Gutmann, A., & Thompson, D. (1990). Moral con¯ict and political

consensus. Ethics, 101, 64±88.

Gutmann, A., & Thompson, D. (1996). Democracy and disagreement.

Cambridge, MA: Belknap Press.

Sheila A. ArensIndiana University, Bloomington, IN, USA

Thomas A. SchwandtUniversity of Illinois at Urbana Champaign, Department

of Educational Psychology, Education 210C, 1310 SSixth Street, Champaign, IL 61820-6990, USATel.: +1-217-333-5350; fax: +1-217-244-7620

E-mail address: tschwand@uiuc.edu

PII: S0149-7189(00 )00019 -7

Qualitative EvaluationIan Shaw; Sage Publications, 1999

Ian Shaw's Qualitative Evaluation (1999) is the ®rstvolume in Sage Publication's new series, `IntroducingQualitative Methods'. The problem that Shaw's quali-tative evaluation is intended to counter is a `horses-for-courses' approach to evaluation design. Shawwrites, ``[W]hen the ground is hard or ®rm, outcome-oriented evaluation designs can safely be jockeyed intoposition, whereas when the ground is soft we shouldopt for mounts which can safely negotiate an under-standing of sticky institutional processes'' (p. 2). Main-stream evaluators, Shaw asserts, have pairedquantitative studies with conditions of `hard or ®rm'and qualitative studies with the `soft' terrain. The con-sequences of this horses-for-courses approach, Shawcontends, has ``left [us] with an impression of qualitat-ive evaluation as an imprecise, ill-focused, descriptive,inductive exercise, strong on vicarious experience, butchronically at risk of failed credibility in the eyes of

the people who count'' (p. 123). To address this pro-blem Shaw promotes a version of qualitative evaluationthat ``[1] promises distinct but coherent perspectives onpolicy, program, and practitioner evaluation, . . . [2]o�ers credible partial solutions to problems of causalanalysis and outcome evaluation, . . . [and 3] is able toavoid sentimental failures [of other approaches] to pro-blematize its own analysis and solutions'' (pp. 5±6).Shaw's qualitative evaluation further acts as ``a correc-tive redistribution in the light of the [current] enthu-siasms for outcomes, evidence, and accountability''(p. 15). He uses the rest of the book to convince thereader of the rightness of his claims.

In the ®rst half of the book, Shaw lays out thetheoretical, methodological, and political-moral(Schwandt, 1989) foundations signalling the need forthe kind of qualitative evaluation that he envisions.Chapter 2 introduces the reader to perennial andrecent dilemmas and debates in evaluation, drawing onfour primary literatures: evaluation theory (Scriven,Campbell, House, Weiss, critical evaluation), edu-cational evaluation (Cronbach, Eisner), social work

Book reviews / Evaluation and Program Planning 23 (2000) 329±335 333

evaluation (Reid), and qualitative research method-ology (Stake, Guba and Lincoln). Within the span of20 pages, Shaw highlights problems with paradigms,politics, advocacy, validity, ®eldwork, design, andre¯exivity. Crafted selectively, Shaw uses this review todevelop an agenda to which his brand of qualitativeevaluation is particularly well-suited to address.

From speci®c evaluation theorists, Shaw extends hisdiscussion to paradigms of evaluation. After examiningcritical postpositivism, critical evaluation, constructi-vism, and pragmatism, he tackles the twin tensions ofrealism versus relativism and rigor versus relevance. Ina unique move that I have not often encountered intexts on qualitative inquiry, Shaw excoriates Guba andLincoln and constructivism in general for its relativismand opportunistic `ontological gerrymandering' (Wool-gar and Pauluch, 1985 in Shaw, p. 56). Not surpris-ingly, Shaw argues for a qualitative evaluation closerto the fallible realist post-positivist position.

Shaw uses problems of scienti®c rigor vs practicalrelevance to transition to questions of values, validity,and evaluation use. He organizes his presentationbased on ®ve orientations: internal and external val-idity; plausibility and credibility; evaluation as a moraland practical activity; reformist evaluation; and openlyideological standards. De®ning each orientation arethe primary values driving the quality criteria its avers.Importantly, Shaw makes explicit that the evaluator'sworldview assumptions about social change underlieone's orientation to validity, her values, and the waysshe considers evaluation use.

The second part of Qualitative Evaluation focusesmore directly on evaluation practice. Chapters 5 and 6delineate program, policy, and practitioner evaluationbased on di�erences in evaluative purpose, informationneeds, and intended use. Shaw illuminates how pur-pose and use in program, policy, and practitionerevaluations intimate particular orientations to validityand valuable knowledge. Particularly illuminative isShaw's analysis of how evaluation by practitioners,evaluation for practitioners, evaluation of practitioners,and evaluation with practitioners di�er.

In the last three chapters (design, ®eldwork, andanalysis), Shaw returns directly to the promises hemakes about qualitative evaluation that open thisreview. For example, Shaw says qualitative evaluationdoes not put a misplaced faith in rationality of pro-gram provision or policy development; rather, qualitat-ive evaluation problematizes practices such asoutcome-based studies and evaluation for accountabil-ity. With regard to outcomes and causality, Shawargues that qualitative evaluation provides a better ®tfor evaluation purposes because it attends to micro-processes, local theory, and contextual variables. Theprimary forms of methodology that Shaw seems topromote toward these ends are ethnographic case stu-

dies in di�erent combinations (single, multi-site, com-parative) and various sampling frameworks (i.e.,critical cases, revelatory cases, extreme cases) as wellas participatory and action-oriented evaluation strat-egies.

Chapter 8 translates design decisions into ®eldworkissues including several ®ne examples of how di�erentqualitative methods (interviewing, observation, anddocument review) can be used for evaluative studies.The ethical issues raised in this chapter, for the mostpart, do not diverge from conventional human subjectsstandards until the last two pages, where (for the ®rsttime!!) Shaw directly recognizes the importance of re-lationships in qualitative evaluation. In the ®nal chap-ter, Shaw squarely takes up the topic of establishingevaluative claims in qualitative evaluation. Shaw pre-sents several qualitative analysis strategies applicableto di�erent evaluative purposes and ways in whichevaluators can strengthen evaluative claims.

Overall, I applaud Shaw's e�orts to show how quali-tative methods can contribute to our understandingsof causality and generalizability in evaluation. For thenovice evaluator or uninformed evaluation user,Shaw's book provides several ®ne examples of usingqualitative methods in ®eldwork. His discussion of thesimilarities and di�erences across policy, program, andpractitioner evaluation is useful as are his e�orts tohighlight the centrality of evaluative purpose in designdecisions. Finally, the sustained attention Shaw paysto validity, both conventional and alternative forms,captures well the ways in which evaluation as a®eld has grappled with tensions between rigor andrelevance.

Although successful in the ways highlighted above,Shaw's presentation su�ers from several problems thatseem endemic to the ®eld of evaluation (as well asmany other academic ®elds). The ®rst problem makesitself evident with the very title of Shaw's book, Quali-tative Evaluation. Is it useful, valuable, or even logicalto assert that there is a qualitative evaluation? As thereader may have guessed, I believe that what Shawpromotes in his book is one kind of evaluation thatrelies on qualitative methods; however it is not theonly approach to evaluation that uses qualitativemethods and qualitative thinking. The qualitativeevaluation that Shaw describes sounds to me a lot likeethnography in evaluation. I would like to point outthat there are many approaches to qualitative inquirythat can inform evaluation Ð life histories and narra-tive (Abma, 1999), phenomenology (Moustakas, 1994;Polkinghorne, 1989), and grounded theory (Strauss &Corbin, 1990) are just three of them. Indeed, Glaserand Strauss have been making claims about theory-building, causality, and generalizability in qualitativestudies since 1967 (Glaser & Strauss, 1967). AndRobert Stake's naturalistic generalizability (1995)

Book reviews / Evaluation and Program Planning 23 (2000) 329±335334

sounds very close to the kinds of claims of generaliz-ability that Shaw describes. I believe that Shaw's bookwould be far improved had he articulated the breadthof qualitative approaches available today, approachesthat have rich disciplinary histories and well-developedlogics for conducting inquiry that can inform evalu-ation.

My second concern with Shaw's book goes to hispromise that his qualitative evaluation ``is able toavoid sentimental failures to problematize its ownanalysis and solutions'' (p. 6). I believe that Shaw vio-lates this promise. For example, Shaw insists that inthe ``everyday world of evaluation'' (p. 2) ``qualitativeevaluation will usually be the methodology of choice''(p. 15). To me, it seems logically inconsistent to assertsimultaneously that evaluation studies should bedriven by evaluative purpose and that most studieswill use the same methodology. In a ®eld driven by ahost of practical purposes, political agendas, andhuman values of all sorts (such as the evaluation®eld portrayed in Shaw's book), how can one claimmethodological superiority?

Shaw unrepentantly advocates for his brand ofevaluation, using the wider literature selectively to bol-ster his points. Personally, as an instructor of evalu-ation, I want my students to think critically about thequestions that they ask, the methods they select, andthe interests served in di�erent formulations of evalu-ation questions. I want them to learn how to do rigor-ous, thoughtful, and relevant work based onevaluating the options that they have for conductingevaluations and the needs of the speci®c study. Iwould like them to learn that good `scienti®c citizenry'(Greene, 1996 in Shaw, p. 194) depends on the ques-tions they ask and the ways that they practice, not

simply the method that they employ; that social justiceis not just a matter of fairness and equity (althoughcertainly important), but is also concerned with howwe treat others who participate in and are a�ected byour studies (Davis, 1999; Ryan, 1993). Based on myreading, Qualitative Evaluation will not lend itself tothese ends.

References

Abma, T. A. (1999). Telling tales: On evaluation and narrative. In

Advances in program evaluation, Vol. 6. Stamford: JAI Press.

Davis, J. E. (1999). Commentary: Advocacy, care, and power. The

American Journal of Evaluation, 20(1), 19±122.

Glaser, B. G., & Strauss, A. (1967). The discovery of grounded theory.

Chicago: Aldine.

Moustakas, C. (1994). Phenomenological research methods. Thousand

Oaks: Sage.

Polkinghorne, D. E. (1989). Phenomenological research methods. In

R. S. Valle, & S. Halling, Existential-phenomenological perspec-

tives in psychology. New York: Plenum.

Ryan, K. E. (1993). Evaluation ethics and issues of social justice:

Contributions from feminine moral thinking. A paper presented

at the Annual Meeting of the American Evaluation Association,

Dallas, TX.

Schwandt, T. A. (1989). Recapturing moral discourse in evaluation.

Educational Researcher, 18(8), 11±16.

Stake, R. (1995). The art of case study research. Thousand Oaks:

Sage.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research:

Grounded theory procedures and techniques. Newbury Park: Sage.

Alexis KaminskyStanford University, School of Education, 485 Laseun

Mall, Stanford, CA 94305-3096, USAE-mail address: ak38@leland.stanford.edu

PII: S0149-7189(00 )00020 -3

Book reviews / Evaluation and Program Planning 23 (2000) 329±335 335